The Protein Data Bank (PDB) is the definitive repository of macromolecular structural information. The availability of structure factors for most entries in the PDB has made it possible to continuously improve the models in the PDB by reinterpreting the primary data for existing structures as new methods of analysis, new biological information, and new ways of describing structures become available. This continuous improvement will be even more powerful once the diffraction images associated with each entry become accessible. The key factor is that depositing raw images will stimulate the improvement of integration and processing software in the same way as the deposition of merged X-ray data hugely stimulated progress in refinement software. Revisiting deposited images with that improved software will deliver more accurate data (especially, free from the currently inadequate treatment of contamination by multiple lattices) against which to re-refine the deposited structures themselves. With the initial interpretation of a structure, the original structure factors and the raw images, it will become possible both to carry out extensive validation of structures and to apply new algorithms for structure determination and analysis as they become available, leading to structures of ever-increasing accuracy and completeness.
 So it's my pleasure to introduce Tom Tawiliga, the final speaker of the day. So Tom is a founder member of the fraction data deposition working group. He was the agreed nominee of the Commission on Biological Molecules of the IUCR and I believe he's now the chairman. He's also the incoming president of the ACA and I think you will agree with me with the vision for realizing the living PDB and how raw diffraction data and its metadata can help. Tom has his finger on all the pulses. Thanks John. Yes, it's a delight to be here to talk to you about this. You know, this talk is on continuous improvement of data, of interpretation of data in the protein data bank and actually when I started off pushing this idea, there was a lot of resistance that this just didn't seem like a practical thing and there's all these challenges to doing it. But a couple weeks ago I was at a workshop that the protein data bank organized was on a validation of ligand structures in fact and this issue came up and it was obvious that of course we should do this. It was like, wow, it's really catching on. So maybe I'm just going to be preaching to the choir here for the next minutes which would be the easier thing to do than usual. So this talk is basically coming from a paper that Gerard Braconia and I published last year. There was a series of papers that were published based on the diffraction data deposition working group and this was just one of those. And so let me give you a little background here. The idea of reanalysis of deposited data goes back a long ways and here's one quotation from Gerard Klevek and Alan Jones group. After they developed a server, the electron density server, there's basically a tool to allow anybody and it's there today. It allows anybody to look at the electron density for any structure in the PDB for which structure factors have been deposited. So basically looking at those maps allows you, the user, to evaluate with your own eyes as we heard earlier today whether or not a structure is likely to be right or whether not a particular feature of that structure is right and you don't have to own any fancy tools in order to do this. You just go to the web and you can look right there or you can download the files and look at them on your own computer. So this is a very nice tool and they came up with the idea that yes, perhaps this whole structure database should be dynamic, not just a fixed, here's a structure, take it or leave it, but maybe we should be updating these things as things go on. More recently, Robbie Justin, Braconia and Gert Vried and others developed a pipeline that they call PDB Redu that was alluded to earlier today also in which the structures in the PDB for which data or structure factors are available are continuously re-refined and actually remodeled at a certain level as well. So you can go to the PDB Redu site and you can download a version of any structure that's in the PDB that is basically the best automated structure you could get using the tools that they're using that starts off with the structures in the PDB. And many of these are demonstrably less inaccurate than the structures that were originally in the PDB. So there's already significant merit to going to that site to get your structure rather than one from the PDB. I wouldn't necessarily take everyone from there yet, but certainly the direction is the right one. And they made some arguments why this is a good idea and amongst them the last one there, it's good for the depositors. So this is kind of one of the worries like well I deposit my structure that's my great wonderful structure, how can anybody ruin it, but in fact these guys are going to make your structure even better and you still get credit for it, so what's wrong with that? So lots of ideas to do that. So why are we even talking about this? It's obvious, right? I already told you people are feeling it's obvious. It's because the current paradigm is not what I just described. So the protein data bank is and for the foreseeable future will remain the definitive repository of macromolecular structure. It's really unique in the structural biology, in the biology world that there's really only one place. This is the place where if you want to get a structure it's going to be there. And the crystallographic community sees the PDB as its archive of models and data and the biological community which is much, much bigger uses the PDB as its essentially only entry into structures. They want a structure that's where they're going to get them. And other repos- it's very important to recognize that other repositories like PDB Redo would have to add major value to these structures to be widely viewed and they're not widely viewed. Anything like the PDB is at the moment. And that's the important issue is that the protein data bank is totally special. Now how does it work in terms of structure determination and analysis? Well right now it's really one time thing. A crystallographer or their team determines a structure. They deposit the data now that goes with that structure and their model which is their interpretation of the data. And that's basically it. And it is possible to update these structures. It's easier for the depositor to update the structure than for anybody else. Basically the original depositor can remove the original and replace it with another deposit. And then that becomes the definitive new structure. Anybody else can reanalyze and deposit another version of that structure. But doing so requires a publication to go with it, be it's a lot of work. And C, the new structure is for all intents and purposes not associated with the old structure. It gets a new PDB identifier. Of course there will be a link that says that this is highly related to a previous one. But there's no link that says this is the exact same data except in text. So it's basically like a new structure. It's poorly linked. That's today's version. This will all be changing. The PDB also is on board that it's obvious this has to change. This is all very good. Now another important aspect of this is the historical focus has been on the model, on the interpretation, not on the data itself. So you deposit a PDB structure. It's the structure. It's the coordinates that are the thing that everybody thinks about. It's the coordinates that originally got the identifier. Later as an afterthought with a lot of work, the structure factors started getting deposited along with that. Okay, great. But those are like an annotation of the model rather than the other way around which you would imagine. It makes much more sense. And there's only rare reference to the data itself. It's really the reference to the structure of the coordinates. In the community, the biological community doesn't see the data at all. So why should this be different? Why doesn't this make sense to the current state? Well, to me, it doesn't make any sense at all. And one is the reinterpretation idea is totally feasible. So PDB redo is demonstrated that you can automatically reanalyze structures at a high level and with high throughput. So it's feasible to do that automatically even though it's not perfect yet. But surely that'll get better. And as software improves, that capability will be extended. Third, validation procedures exist to evaluate model quality. I'll mention this later. And the protein data bank itself is spending a lot of effort on developing standards for validation of protein structures and of the data. And importantly for this meaning, the availability of raw images will extend the capability of reinterpretation even further. So an expert reanalyzing images can often obtain data that are far superior to the process data that are obtained by routine application of even their own tools. And just an example of that. Recently, I was working on methods for solving structures by Sulfur SAD. And we had some structures that were very challenging to solve. We obtained the data from the original depositors and we tried to solve that. And we could solve some of them but only with great difficulty by merging together. And then we sent the same data, or the same data went to Spiesic Ostrinowski, who is Vladek's companion who works on data processing and Spiesic reanalyzed all that data. And then suddenly a lot of these structures could be solved, that basically couldn't be solved before. So an expert but not super expert is not the same as a super expert. And these processes will always be getting better. So it is surely true that down the road we will be able to take the exact same images and obtain much better, more accurate structure factors and therefore more accurate phases and we will get better models. And this, you know, might say it doesn't really matter, you only can achieve small things, but actually at lower resolution a little bit of difference can make a huge impact. And so our structures may well be dramatically improved by doing this. And reinterpretation is a good thing to happen. It is not only feasible, but why would we do this? Well, you can fix minor errors. Many people, including some of the room, Vladek has published papers particularly on validation of metal ions, for example, in structures. Others here have also published structures about errors in structures. Surely we can fix the little things. And maybe we will fix the big things as well. So why leave these errors in there if we know how to fix them? Another thing is you might be able to get consistency amongst related structures by using consistent methods. So just imagine a situation where you are solving the structures of a whole bunch of proteins that are more or less the same with different ligands or something like that. And what you are really interested in is the differences between these. Well if you use a different software packet for each one and you allow the side chains to refine to different confirmations in each one, you get a whole bunch of difference that are totally superfluous, that are irrelevant. But if you did everything totally consistently and refined all those structures together and only allowed differences amongst them that are required by the data, surely you would get more accurate differences that way. So that kind of thing is the kind of thing you might imagine people doing in a systematic way if you are basically allowed to reinterpret all these structures. Of course you can do that today, but you can't make it available to everybody in an easy way. You might have new formulations about how you even describe structures, multi-model descriptions of structures. And the other ones I really talked about, joint refinement of structures. So why hasn't this happened already a long time ago? So it's really a cultural thing. And the interpretation of a structure is basically owned by the person who solved it. So I solved my structure, I spent a whole lot of effort getting it there, I deposited in the PDB and then everybody in the world knows that's Tom's structure. And then if there's something wrong with it, it's my fault and everything. But basically somebody else just goes and takes my data and reinterprets that there might be conflict between me and them. And we don't want that, right? And that's just a cultural thing, there's no reason why that should be that way. And why is it? It's because I spent a lot of effort making that structure. And if somebody else fixes it, I might look at that as criticism of me instead of an improvement in my model. So we need to fix that way of thinking in order for this whole thing to change. So I need to feel that when you fix my structure, you have just made all this better and you've made put more value into my structure and so I feel good too. That's how we have to have it. There's some practical issues too. So if there's multiple, suppose there was some way to have lots of interpretations over the same data. Well you, the user, the biologist, well which one should I look at? They're all kind of similar but everyone is different and I might get a different interpretation of the thing I'm actually interested in depending on which one I look at. So how do I decide? So that's a big issue, not totally solved and that's a really important one. And along with that, if I make a new structure, which one's better? How do you decide what's improved? The minor point is that the original positor, they might have access to some specialized information that they didn't specify as metadata. And if so, maybe an automated repeat or my own repeat later might not do as well. So we need to keep that kind of thing in mind. Another reason this doesn't happen is because deposition is a big job. Okay so if we're going to continuously improve structures, what might it look like? Well it could be systematic optimization of all the structures. That's PDB redo, just continue that and make it better. Great. Redetermination of groups of related structures, I talked about that kind of idea. Or your determination of certain groups of structures where you're focusing on specific questions. So look at just metal containing proteins and look at all the bond lengths that are in those and really refine every structure in order to obtain that information most accurately. So that would be a perfectly fine thing you might want to save and let other people have down the road. And so what do we have to do if we want to make this all happen? So we really need a way to validate the data and have available to us validated process data and metadata so that a person can start now starting off with the original structure perhaps or maybe starting from scratch and redo everything and have all the information that's necessary that was also available to the original depositor. So we talked here a lot about collecting metadata and this is just totally critical, right? Because you have to have a way to transfer the information that the original person had to a person later so they can do the optimal job in developing a new model. And I think it would really be ideal for the deposited data at deposition time basically to one of the validation checks would be taking the deposited data, deposited metadata and eventually just redoing the structure determination with those and validating that the answer you get is pretty much the same thing as the depositor got. So today it's not very feasible, it takes too much computer time, blah, blah, blah, but that won't be a limiting factor down the road. So no particular reason why you couldn't do that. So that says yes, at least using today's technology would get pretty much the same thing they did. They must have deposited the right amount of metadata. If they didn't do that then we wouldn't be able to get the answer and they just send it back to them and say give us enough data so we can reproduce what you did, right? It seems pretty obvious, right? You also need unmerged data. So John Westbrook talked about this a little bit before. It's really essential to have unmerged data or the images themselves in order to do this validation process well. So we already talked about it so much today, but basically the estimation of errors in the intensities of reflections is not as difficult to do accurately. And particularly for, say, measuring anomalous differences and estimating the uncertainties and anomalous differences that go into phasing and so forth, it's very hard to get those numbers right. The most powerful way of getting that number pretty accurate is to have multiple measurements of the same things and in order to validate whether or not somebody did that correctly, you need all those multiple measurements of the same things or you need the raw images so you can redo it from the beginning. So these things are totally critical. So the raw diffraction images that we're talking about today are by far the best way to do this. Okay, so just a couple more slides here on what do we need to do to make this all happen. So if the original depositor's structure is not going to be the only structure that the protein data bank or somebody else serves up, you have to have a way to decide which one to look at. And so I make a point here about the views, the protein data bank is the protein data data bank, RCSV, and Europe and Japan, they're three different portals and they actually give slightly different views of the same information. The PDB, the www.pdb.org is by far the most hit, I think, and so whatever they put on their website, that's what a lot of people are going to see. So which structures they serve up and same thing for the other versions of the PDB make, have a huge influence on what people are going to think or what they're going to do. And the model that's appropriate for a particular use may depend on what that use is. So I'll just give you some examples before. If your use is what are the bond lengths for metal atoms to their ligands in proteins, then you want a certain way of having handled that data and a certain pruning of all the data in order to do that. On the other hand, if you're just interested in what's the overall shape of my molecule, then you don't care about those things nearly as much, right? And so eventually it will be necessary to know what is in the user's mind, what am I going to do with that structure, and then serve them up a structure that is most appropriate for that use. And there will presumably always be some default thing, which might be just, for example, the highest resolution structure analyzed most recently with the best statistics in some measure, right? So you might serve that up. And of course, the PDB would also serve up the very original one if people want the original interpretation, which is always going to be there, right? And so how are we going to go about doing this? So as I say, we need a way to serve up the right model, and you have to have metrics for making those decisions. And so this is what many people are working on right now, is the idea of you have a list of structures, a list of possible interpretations of the same data. How do you rank these different interpretations for different uses, or just for validation, is this which structure is most likely to be closest to the right answer, right? And there's no firm answer to any of these things. The PDB is working very hard on developing tools for this validation, though. And so in this slide, I'm illustrating some of the things that are happening along these lines. So number one, there's a lot of discussion now about the idea of versioning of structures in the PDB, instead of having a new PDB deposit for a new interpretation of the same thing, there's just one data and one PDB identifier goes with the data, and then versions of that same thing. And those could be versions that would be the same depositor replacing it, might be a version, or somebody outside making a new version that might be a new version, et cetera. So it's just a conceptual thing. Now everything is associated with data, and there are different interpretations of that data. So that makes more sense in terms of serving things up. And then validation, as I've been talking about, in the protein data bank now, the PDB has task forces for each overall class of data, like X-ray data, EM data, and so forth. And for each one of these, the task forces have the responsibility of identifying criteria, actual measures that can be used as criteria to identify are there problems or not problems with this structure. So the protein, or X-ray, validation task force went through that exercise a number of years ago, came up with a long list of calculations to be made, things such as bond lengths, clashes, agreement with the density, et cetera, a long list of these things. And when you go to the PDB today, you can download, by clicking, a long sheet that lists the results of all those things. Now one might say these reports need a little bit more work so that they're easier to interpret. That's just a minor technical aspect of it. The reports themselves are very, very valuable. And the PDB is continuing to upgrade and update these validation reports. And the basic idea is these validation reports will be available to users of the PDB. They'll be available, we hope, to reviewers always before accepting papers. And then down the road, the same information can be used for serving up the right structure for each purpose. So that's where I think we're going to be going here, is we'll have metrics that say, what are the numerical criteria for how the quality of the structure in many different ways, and then use those as some mixture of those as a way to serve up the structure that's most appropriate for particular users. And we've been talking about images a lot, and I just want to mention that there's actually several places right now where people can deposit or have their aura images made available. And one is Blottic's site that he talked about earlier today, which is just super great, and it's really wonderful to see that. It's going to be a way to download images from any of a number of different places with a lot of analysis that goes with them. Also the SBGrid, which is an organization that supplies crystallographic software, many different types of crystallographic software as a big package to many structural biology groups, and which also has access to cloud computing. They also have a system for depositing a raw diffraction data for their users. So those users can also do that. So in summary, I think everything I said is perfectly obvious, and I'm really glad that today it's obvious, even if it wasn't a few years ago. And I think there's a really great future for continually updating what's in the PDB, making the structures more and more accurate, making them more and more comprehensive, and serving up the right structures to the right users. Thanks. Thank you very much. The idea of versioning of entries in the PDB is a good idea, but it's almost a paradigm shift for the PDB because, I guess, up to now the basic unit of the PDB has been the model, right? It's been the coordinates. And for the longest time, even structure factors were not required, were not recorded. And so what you're proposing is basically making the structure factor the basic unit of the PDB, which I think is a good idea and will be very helpful. So can I respond to that one first? So the PDB has been giving very positive indications that they really like this approach and would like to do something along these lines. So I'm anticipating this will happen soon. Okay. And so the second comment is something going back to the whole metadata idea. You said that, and everyone knows that, I guess, with more modern software, more modern approaches, you get better models, better structures from the same data. And that, I guess, flies in the face of the idea of the user doing the time of the experiment, making a decision of which metadata to record and which compression algorithm to use for his project, because in the end, the whole long-term storage is not about this person's project, but again, it's more about the data themselves, right? So how do you make this decision of what you want to record, how you want to compress for it to be useful down the road? Yeah. I think that's a really important question. I don't know the answer to that, of course. So the user at the time of collecting the data or at the time of deposition doesn't know what 10 years from now, 20, 50 years from now, somebody will want to do with that data. So the more accurately, the more comprehensively we can save the information that we do have about the data, the better off we'll be. As I sort of maybe facetially said earlier, today the default is highly lossy compression, which is basically interpretation of the data and saving merged structure factors. That's the default. And that's not so bad. We're doing pretty well. And it's way better than not having structure factors at all. So every incremental thing that we add beyond that is just a plus. And the more of those we have, the better off it would be. And yeah, I can see somebody a long time from now having some idea about how to interpret all these things. And they'll go through the whole protein data banks of 10 million structures, and they'll redo them all, analyzing something that we never thought about at all. And I think that would just be fantastic. Tom, I think that you made extremely good point that we really do not know what we should do because we are working in fast prototyping mode. How I know that it's 192 frame formats, at least 192, because I encounter 192, but somewhere there may be more. And the same we have with structures and metadata. We do not, as long as we do not have even basic metadata set for all PDB, we really do not know what we can do with that. Yes, and to our images. Agreed. So this is a repeat of a comment I made earlier, but outside of the session. Instead of worrying about the fact that there are so many different frame formats, why not make it part of the archiving process that everything gets transformed into the same frame format? Then it's just one place that has the same frame formats, and you don't need to worry about it. Right? Then the PDB has only CBF. I know it doesn't have to be CBF, but the PDB only stores one type of frame, and then everyone can reprocess it using whichever software they want. So you essentially turn the frame format problem over to the archivers. Yeah, actually, I am personally totally in favor of this idea. And my interpretation of that is that the software developers like Volatic and Company have a mechanism for taking 192 different whatever, and turning them into something internal that is one version. So yeah, looking for Volatic to answer this. And then presumably that thing could be written out. And presumably that thing could be standardized because there's only five people doing it instead of 300 people doing it. And I think if that would be possible, it would be really fantastic. So I think that future proofs the problem at least a little bit. And I had one second comment, which is really minor. And that's if you're starting to talk about taking multiple versions of a file and then giving data to choose which one is the best one, do you then need to introduce a new phraseology? I mean, you've got data about data about data. Maybe that's not metadata, it's hyper data. Yeah, perhaps so. And you know, better is the wrong word to be using for sure. Yes, obviously, internally we have our own frame format. Yes. Otherwise we would have gone crazy. Long time ago. I mean, I am crazy. I realize that. But that's great. But and we have a procedure to write in that format. Unfortunately, it's not so easy, yes, to write because it's relatively easy to change the Goniostad outside if you are learning after some time that this Goniostad is not accurate. You would have to rewrite all this data. Yes, you would have to rewrite all this. I see. So your interpretation, your internal data is actually interpreted data that depends upon parameters, which may or may not be. All parameters which are in Dev side. So it's a little far from the original, which is a bit, you don't think it's risky. It sounds risky to me, yes. Yeah. And you see that it's even more complicated because look, we are getting data from, let's say, BIM line A. Yes, and we are doing, and they usually, they sent one data set, they are saying it's extremely urgent because users are coming tomorrow. We have to have something. We are doing that. We are giving them, three months later, we are learning that it's really not the one frame format. There are two frame formats. Yes, and already some people use this other frame format. Yes, I mean, you see that as an example of the company which does not exist as a valid company is ADSC detector, extremely good detector, which had software billing and hardware billing. And companies, they do not know which one is better. So they are saying, okay, we will leave that to users, but they are not saying anything about that. Yes, we are getting data and error density, so-called error density parameter is different for this too. It's different by the factor of five. And overnight, you are learning that some people who use, let's say, software billing when you got hardware billing, they have worse resolution than they obtained before. But these are the problems which we have, I mean, we, the people who develop software, processing software in real life. So can I propose an alternative version? So still, a user who takes your software and produces intensities out of it, you and the user have gone through a process there with a little handshake that says you understand their inputs and are able to interpret those inputs in terms of parameters that you need, namely all the angles, all the geometry, whatever. Could you not write out that information, which I think you are doing, that information which is sufficient to reproduce the experiment, the analysis with your software, and also with another, you are writing it out so that XDS or somebody else can also do that. That is, that's halfway there and it seems like depositing that would be very, very beneficial. Michael? Right, Michael, I'll ask Alamos. So the question I have has more to do with my perspective, viewing the protein structure as an ensemble rather than a single structure. So in a kind of Bayesian way, you can define a likelihood for different models depending on the data and some models more likely than others. So the approach that you described seems fine of reporting one structure to a user assuming that this likelihood function is very sharp. But if it is broader, then it might be good to actually force a user to pay attention to multiple models instead of presenting one, especially if what they are going to do with it will yield a different answer depending on which one they choose. So what you described is like a huge advance over the way things are now. What I am describing is something that I guess would go somehow beyond that. So how far are we from doing this next step? So Mike, I totally agree. I didn't talk about that today, but actually that is an issue that I feel is very, very important. Basically, we shouldn't be representing these structures as a single structure is one in the beginning. That is wrong. It is not what is in the crystal. The crystal has millions of structures in it. And some representation of what those structures are is where we have to eventually go. And so I totally agree with your point. So I would like to also consider the end point here. So we have the better process, the continual improvement. So we get to this better use of the data through the communal efforts. So now to pose the well-rehearsed question on behalf of Derwood Crookshank, who passed away a few years ago, who was first paper on understanding the precision of coordinates in crystal structures was 1949. So in looking at what we do, he was very concerned. It is perhaps more the matter of the graphics representation software that we all use of having a uniform view of the precision of non-bonded distances, two decimal places. And he said that just cannot be true. Just look at the B factors. If two atoms are potentially interacting in their 80 angstrom square, then looking at integer precision of their distances apart, let alone two decimal places. So we have our half a billion downloads per year of the fruits of our research. And these are not specialist in our techniques. And they go away with a false precision. So he thinks, and I agree with him, forced to agree with him, he had me on the ropes, this is a big issue for our field. And maybe it's solvable by education. And all that the PDB is doing, like we saw a precision on a wavelength to seven decimal places this morning. And it's just the over-enthusiastic computer scientist providing all of these columns for our experimental parameters and our results. But there is an issue here. I think he's right. Yes, it's an ensemble of structures because we have millions and millions in our sample. But that variation, let's perhaps avoid the word coordinate error, but coordinate variation tells us that we have to provide to non-expert users a much better feel for what the atom placement precision is. So I once again totally agree with this. So actually, let me make a little point here. We need to differentiate between two types of imprecision or uncertainty. One is in the actual crystal, there's millions and millions of different protein molecules or whatever they are. And everyone is a little bit different. And so that's describing what's actually in the crystal. And also, these are not necessarily just Gaussian distributions that are represented simply by nice little b-factors. They may be static disorder, many different things that can be going on. So what's in the crystal is fairly complicated and is not well represented by a single model. It would be better represented by a single model plus some numbers that say what the variation is. That would be a better step, which we aren't even doing. So that would be the first order improvement would be what's the variability in the crystal for those numbers. And then second order would be something about what are the different conformational ensembles that describe most of the variation or something like that. So that's where Mike's going to. So that's one thing. But separate issue is we're using our crystallographic tools to try to describe the structure. And we come up with models. And there's some, if we did the same experiment twice, we come up with different models. And how much do those vary? And so surely that second variation is a lower bound on our uncertainty of what is in the crystal itself. So the crystal varies, and we don't even know what's in the crystal. So we need to describe both of those things. And I think ultimately we're going to describe both of those by some way of talking about ensembles of structures. And it may be molecular dynamic simulations that kind of give us a handle on how to describe that. It may be we have a limited number of ensembles that describe most of the variability, and that's what we're going to report. But absolutely, the end user has to understand what they can know from what we're telling them and what they can't know. And right today, you give them a single structure, and they look at the distance between two atoms, and they think these things must be hydrogen bonded because they're the right distance apart. But maybe it's that way only 1,000th of the time. And most of the time there's some other completely different configuration. And we're not telling them that today. We're not even telling them. It's ridiculous. So I totally agree. I hope the Commission on Biological Micro Molecules will take this up on their leadership. Great. So again, it's just another comment. You said if you take two crystals and do the diffraction experiment, you get two different models. If you take the same crystal and the same data and process it in two different ways, you get two different models. Yes. So it's even worse than two different crystals. Yes, absolutely true. Absolutely true. Actually, the good thing is, so we studied this a little bit a few years ago with just macromolecular structures and just reinterpreting the exact same crystal data, just running it through our software a bunch of times with different random seeds. And the good thing is, in a macromolecule, the core of the molecule, they just superimpose extremely closely. You just can't do any, you can't interpret these structures any other way other than to basically put these atoms in pretty much the same places. But the stuff on the surface varies a huge amount. Yes. Okay. Thank you very, very much indeed, Tom. And I'm sure everybody wants to show the usual appreciation. Thank you very, very much. So Brian will reconfirm the logistics, which is seven o'clock minibus outside the front. And enjoy, I don't know, an hour and a half in the pool or whatever takes your fancy. Thank you.
 So it's my pleasure to introduce Tom Tawiliga, the final speaker of the day.  So Tom is a founder member of the fraction data deposition working group.  He was the agreed nominee of the Commission on Biological Molecules of the IUCR and I  believe he's now the chairman.  He's also the incoming president of the ACA and I think you will agree with me with the  vision for realizing the living PDB and how raw diffraction data and its metadata can help.  Tom has his finger on all the pulses.  Thanks John.  Yes, it's a delight to be here to talk to you about this.  You know, this talk is on continuous improvement of data, of interpretation of data in the  protein data bank and actually when I started off pushing this idea, there was a lot of  resistance that this just didn't seem like a practical thing and there's all these challenges  to doing it.  But a couple weeks ago I was at a workshop that the protein data bank organized was on  a validation of ligand structures in fact and this issue came up and it was obvious  that of course we should do this.  It was like, wow, it's really catching on.  So maybe I'm just going to be preaching to the choir here for the next minutes which  would be the easier thing to do than usual.  So this talk is basically coming from a paper that Gerard Braconia and I published last  year.  There was a series of papers that were published based on the diffraction data deposition working  group and this was just one of those.  And so let me give you a little background here.  The idea of reanalysis of deposited data goes back a long ways and here's one quotation  from Gerard Klevek and Alan Jones group.  After they developed a server, the electron density server, there's basically a tool to  allow anybody and it's there today.  It allows anybody to look at the electron density for any structure in the PDB for which structure  factors have been deposited.  So basically looking at those maps allows you, the user, to evaluate with your own eyes  as we heard earlier today whether or not a structure is likely to be right or whether  not a particular feature of that structure is right and you don't have to own any fancy  tools in order to do this.  You just go to the web and you can look right there or you can download the files and look  at them on your own computer.  So this is a very nice tool and they came up with the idea that yes, perhaps this whole  structure database should be dynamic, not just a fixed, here's a structure, take it  or leave it, but maybe we should be updating these things as things go on.  More recently, Robbie Justin, Braconia and Gert Vried and others developed a pipeline  that they call PDB Redu that was alluded to earlier today also in which the structures  in the PDB for which data or structure factors are available are continuously re-refined  and actually remodeled at a certain level as well.  So you can go to the PDB Redu site and you can download a version of any structure that's  in the PDB that is basically the best automated structure you could get using the tools that  they're using that starts off with the structures in the PDB.  And many of these are demonstrably less inaccurate than the structures that were originally in  the PDB.  So there's already significant merit to going to that site to get your structure rather  than one from the PDB.  I wouldn't necessarily take everyone from there yet, but certainly the direction is  the right one.  And they made some arguments why this is a good idea and amongst them the last one there,  it's good for the depositors.  So this is kind of one of the worries like well I deposit my structure that's my great  wonderful structure, how can anybody ruin it, but in fact these guys are going to make  your structure even better and you still get credit for it, so what's wrong with that?  So lots of ideas to do that.  So why are we even talking about this?  It's obvious, right?  I already told you people are feeling it's obvious.  It's because the current paradigm is not what I just described.  So the protein data bank is and for the foreseeable future will remain the definitive repository  of macromolecular structure.  It's really unique in the structural biology, in the biology world that there's really  only one place.  This is the place where if you want to get a structure it's going to be there.  And the crystallographic community sees the PDB as its archive of models and data and  the biological community which is much, much bigger uses the PDB as its essentially only  entry into structures.  They want a structure that's where they're going to get them.  And other repos- it's very important to recognize that other repositories like PDB Redo would  have to add major value to these structures to be widely viewed and they're not widely  viewed.  Anything like the PDB is at the moment.  And that's the important issue is that the protein data bank is totally special.  Now how does it work in terms of structure determination and analysis?  Well right now it's really one time thing.  A crystallographer or their team determines a structure.  They deposit the data now that goes with that structure and their model which is their interpretation  of the data.  And that's basically it.  And it is possible to update these structures.  It's easier for the depositor to update the structure than for anybody else.  Basically the original depositor can remove the original and replace it with another deposit.  And then that becomes the definitive new structure.  Anybody else can reanalyze and deposit another version of that structure.  But doing so requires a publication to go with it, be it's a lot of work.  And C, the new structure is for all intents and purposes not associated with the old structure.  It gets a new PDB identifier.  Of course there will be a link that says that this is highly related to a previous one.  But there's no link that says this is the exact same data except in text.  So it's basically like a new structure.  It's poorly linked.  That's today's version.  This will all be changing.  The PDB also is on board that it's obvious this has to change.  This is all very good.  Now another important aspect of this is the historical focus has been on the model, on  the interpretation, not on the data itself.  So you deposit a PDB structure.  It's the structure.  It's the coordinates that are the thing that everybody thinks about.  It's the coordinates that originally got the identifier.  Later as an afterthought with a lot of work, the structure factors started getting deposited  along with that.  Okay, great.  But those are like an annotation of the model rather than the other way around which you  would imagine.  It makes much more sense.  And there's only rare reference to the data itself.  It's really the reference to the structure of the coordinates.  In the community, the biological community doesn't see the data at all.  So why should this be different?  Why doesn't this make sense to the current state?  Well, to me, it doesn't make any sense at all.  And one is the reinterpretation idea is totally feasible.  So PDB redo is demonstrated that you can automatically reanalyze structures at a high  level and with high throughput.  So it's feasible to do that automatically even though it's not perfect yet.  But surely that'll get better.  And as software improves, that capability will be extended.  Third, validation procedures exist to evaluate model quality.  I'll mention this later.  And the protein data bank itself is spending a lot of effort on developing standards for  validation of protein structures and of the data.  And importantly for this meaning, the availability of raw images will extend the capability of  reinterpretation even further.  So an expert reanalyzing images can often obtain data that are far superior to the process  data that are obtained by routine application of even their own tools.  And just an example of that.  Recently, I was working on methods for solving structures by Sulfur SAD.  And we had some structures that were very challenging to solve.  We obtained the data from the original depositors and we tried to solve that.  And we could solve some of them but only with great difficulty by merging together.  And then we sent the same data, or the same data went to Spiesic Ostrinowski, who is Vladek's  companion who works on data processing and Spiesic reanalyzed all that data.  And then suddenly a lot of these structures could be solved, that basically couldn't be  solved before.  So an expert but not super expert is not the same as a super expert.  And these processes will always be getting better.  So it is surely true that down the road we will be able to take the exact same images  and obtain much better, more accurate structure factors and therefore more accurate phases  and we will get better models.  And this, you know, might say it doesn't really matter, you only can achieve small things,  but actually at lower resolution a little bit of difference can make a huge impact.  And so our structures may well be dramatically improved by doing this.  And reinterpretation is a good thing to happen.  It is not only feasible, but why would we do this?  Well, you can fix minor errors.  Many people, including some of the room, Vladek has published papers particularly on validation  of metal ions, for example, in structures.  Others here have also published structures about errors in structures.  Surely we can fix the little things.  And maybe we will fix the big things as well.  So why leave these errors in there if we know how to fix them?  Another thing is you might be able to get consistency amongst related structures by  using consistent methods.  So just imagine a situation where you are solving the structures of a whole bunch of  proteins that are more or less the same with different ligands or something like that.  And what you are really interested in is the differences between these.  Well if you use a different software packet for each one and you allow the side chains  to refine to different confirmations in each one, you get a whole bunch of difference that  are totally superfluous, that are irrelevant.  But if you did everything totally consistently and refined all those structures together  and only allowed differences amongst them that are required by the data, surely you would  get more accurate differences that way.  So that kind of thing is the kind of thing you might imagine people doing in a systematic  way if you are basically allowed to reinterpret all these structures.  Of course you can do that today, but you can't make it available to everybody in an easy  way.  You might have new formulations about how you even describe structures, multi-model descriptions  of structures.  And the other ones I really talked about, joint refinement of structures.  So why hasn't this happened already a long time ago?  So it's really a cultural thing.  And the interpretation of a structure is basically owned by the person who solved it.  So I solved my structure, I spent a whole lot of effort getting it there, I deposited  in the PDB and then everybody in the world knows that's Tom's structure.  And then if there's something wrong with it, it's my fault and everything.  But basically somebody else just goes and takes my data and reinterprets that there might  be conflict between me and them.  And we don't want that, right?  And that's just a cultural thing, there's no reason why that should be that way.  And why is it?  It's because I spent a lot of effort making that structure.  And if somebody else fixes it, I might look at that as criticism of me instead of an improvement  in my model.  So we need to fix that way of thinking in order for this whole thing to change.  So I need to feel that when you fix my structure, you have just made all this better and you've  made put more value into my structure and so I feel good too.  That's how we have to have it.  There's some practical issues too.  So if there's multiple, suppose there was some way to have lots of interpretations over  the same data.  Well you, the user, the biologist, well which one should I look at?  They're all kind of similar but everyone is different and I might get a different interpretation  of the thing I'm actually interested in depending on which one I look at.  So how do I decide?  So that's a big issue, not totally solved and that's a really important one.  And along with that, if I make a new structure, which one's better?  How do you decide what's improved?  The minor point is that the original positor, they might have access to some specialized  information that they didn't specify as metadata.  And if so, maybe an automated repeat or my own repeat later might not do as well.  So we need to keep that kind of thing in mind.  Another reason this doesn't happen is because deposition is a big job.  Okay so if we're going to continuously improve structures, what might it look like?  Well it could be systematic optimization of all the structures.  That's PDB redo, just continue that and make it better.  Great.  Redetermination of groups of related structures, I talked about that kind of idea.  Or your determination of certain groups of structures where you're focusing on specific  questions.  So look at just metal containing proteins and look at all the bond lengths that are  in those and really refine every structure in order to obtain that information most accurately.  So that would be a perfectly fine thing you might want to save and let other people have  down the road.  And so what do we have to do if we want to make this all happen?  So we really need a way to validate the data and have available to us validated process  data and metadata so that a person can start now starting off with the original structure  perhaps or maybe starting from scratch and redo everything and have all the information  that's necessary that was also available to the original depositor.  So we talked here a lot about collecting metadata and this is just totally critical, right?  Because you have to have a way to transfer the information that the original person had  to a person later so they can do the optimal job in developing a new model.  And I think it would really be ideal for the deposited data at deposition time basically  to one of the validation checks would be taking the deposited data, deposited metadata and  eventually just redoing the structure determination with those and validating that the answer you  get is pretty much the same thing as the depositor got.  So today it's not very feasible, it takes too much computer time, blah, blah, blah,  but that won't be a limiting factor down the road.  So no particular reason why you couldn't do that.  So that says yes, at least using today's technology would get pretty much the same thing they did.  They must have deposited the right amount of metadata.  If they didn't do that then we wouldn't be able to get the answer and they just send  it back to them and say give us enough data so we can reproduce what you did, right?  It seems pretty obvious, right?  You also need unmerged data.  So John Westbrook talked about this a little bit before.  It's really essential to have unmerged data or the images themselves in order to do this  validation process well.  So we already talked about it so much today, but basically the estimation of errors in  the intensities of reflections is not as difficult to do accurately.  And particularly for, say, measuring anomalous differences and estimating the uncertainties  and anomalous differences that go into phasing and so forth, it's very hard to get those  numbers right.  The most powerful way of getting that number pretty accurate is to have multiple measurements  of the same things and in order to validate whether or not somebody did that correctly,  you need all those multiple measurements of the same things or you need the raw images  so you can redo it from the beginning.  So these things are totally critical.  So the raw diffraction images that we're talking about today are by far the best way to do  this.  Okay, so just a couple more slides here on what do we need to do to make this all happen.  So if the original depositor's structure is not going to be the only structure that the  protein data bank or somebody else serves up, you have to have a way to decide which  one to look at.  And so I make a point here about the views, the protein data bank is the protein data  data bank, RCSV, and Europe and Japan, they're three different portals and they actually  give slightly different views of the same information.  The PDB, the www.pdb.org is by far the most hit, I think, and so whatever they put on  their website, that's what a lot of people are going to see.  So which structures they serve up and same thing for the other versions of the PDB make,  have a huge influence on what people are going to think or what they're going to do.  And the model that's appropriate for a particular use may depend on what that use is.  So I'll just give you some examples before.  If your use is what are the bond lengths for metal atoms to their ligands in proteins,  then you want a certain way of having handled that data and a certain pruning of all the  data in order to do that.  On the other hand, if you're just interested in what's the overall shape of my molecule,  then you don't care about those things nearly as much, right?  And so eventually it will be necessary to know what is in the user's mind, what am  I going to do with that structure, and then serve them up a structure that is most appropriate  for that use.  And there will presumably always be some default thing, which might be just, for example, the  highest resolution structure analyzed most recently with the best statistics in some  measure, right?  So you might serve that up.  And of course, the PDB would also serve up the very original one if people want the original  interpretation, which is always going to be there, right?  And so how are we going to go about doing this?  So as I say, we need a way to serve up the right model, and you have to have metrics  for making those decisions.  And so this is what many people are working on right now, is the idea of you have a list  of structures, a list of possible interpretations of the same data.  How do you rank these different interpretations for different uses, or just for validation,  is this which structure is most likely to be closest to the right answer, right?  And there's no firm answer to any of these things.  The PDB is working very hard on developing tools for this validation, though.  And so in this slide, I'm illustrating some of the things that are happening along these  lines.  So number one, there's a lot of discussion now about the idea of versioning of structures  in the PDB, instead of having a new PDB deposit for a new interpretation of the same thing,  there's just one data and one PDB identifier goes with the data, and then versions of that  same thing.  And those could be versions that would be the same depositor replacing it, might be  a version, or somebody outside making a new version that might be a new version, et cetera.  So it's just a conceptual thing.  Now everything is associated with data, and there are different interpretations of that  data.  So that makes more sense in terms of serving things up.  And then validation, as I've been talking about, in the protein data bank now, the PDB  has task forces for each overall class of data, like X-ray data, EM data, and so forth.  And for each one of these, the task forces have the responsibility of identifying criteria,  actual measures that can be used as criteria to identify are there problems or not problems  with this structure.  So the protein, or X-ray, validation task force went through that exercise a number of  years ago, came up with a long list of calculations to be made, things such as bond lengths, clashes,  agreement with the density, et cetera, a long list of these things.  And when you go to the PDB today, you can download, by clicking, a long sheet that lists  the results of all those things.  Now one might say these reports need a little bit more work so that they're easier to interpret.  That's just a minor technical aspect of it.  The reports themselves are very, very valuable.  And the PDB is continuing to upgrade and update these validation reports.  And the basic idea is these validation reports will be available to users of the PDB.  They'll be available, we hope, to reviewers always before accepting papers.  And then down the road, the same information can be used for serving up the right structure  for each purpose.  So that's where I think we're going to be going here, is we'll have metrics that say,  what are the numerical criteria for how the quality of the structure in many different  ways, and then use those as some mixture of those as a way to serve up the structure that's  most appropriate for particular users.  And we've been talking about images a lot, and I just want to mention that there's actually  several places right now where people can deposit or have their aura images made available.  And one is Blottic's site that he talked about earlier today, which is just super great,  and it's really wonderful to see that.  It's going to be a way to download images from any of a number of different places with  a lot of analysis that goes with them.  Also the SBGrid, which is an organization that supplies crystallographic software, many  different types of crystallographic software as a big package to many structural biology  groups, and which also has access to cloud computing.  They also have a system for depositing a raw diffraction data for their users.  So those users can also do that.  So in summary, I think everything I said is perfectly obvious, and I'm really glad that  today it's obvious, even if it wasn't a few years ago.  And I think there's a really great future for continually updating what's in the PDB,  making the structures more and more accurate, making them more and more comprehensive, and  serving up the right structures to the right users.  Thanks.  Thank you very much.  The idea of versioning of entries in the PDB is a good idea, but it's almost a paradigm  shift for the PDB because, I guess, up to now the basic unit of the PDB has been the  model, right?  It's been the coordinates.  And for the longest time, even structure factors were not required, were not recorded.  And so what you're proposing is basically making the structure factor the basic unit  of the PDB, which I think is a good idea and will be very helpful.  So can I respond to that one first?  So the PDB has been giving very positive indications that they really like this approach and would  like to do something along these lines.  So I'm anticipating this will happen soon.  Okay.  And so the second comment is something going back to the whole metadata idea.  You said that, and everyone knows that, I guess, with more modern software, more modern  approaches, you get better models, better structures from the same data.  And that, I guess, flies in the face of the idea of the user doing the time of the experiment,  making a decision of which metadata to record and which compression algorithm to use for  his project, because in the end, the whole long-term storage is not about this person's  project, but again, it's more about the data themselves, right?  So how do you make this decision of what you want to record, how you want to compress  for it to be useful down the road?  Yeah.  I think that's a really important question.  I don't know the answer to that, of course.  So the user at the time of collecting the data or at the time of deposition doesn't  know what 10 years from now, 20, 50 years from now, somebody will want to do with that  data.  So the more accurately, the more comprehensively we can save the information that we do have  about the data, the better off we'll be.  As I sort of maybe facetially said earlier, today the default is highly lossy compression,  which is basically interpretation of the data and saving merged structure factors.  That's the default.  And that's not so bad.  We're doing pretty well.  And it's way better than not having structure factors at all.  So every incremental thing that we add beyond that is just a plus.  And the more of those we have, the better off it would be.  And yeah, I can see somebody a long time from now having some idea about how to interpret  all these things.  And they'll go through the whole protein data banks of 10 million structures, and they'll  redo them all, analyzing something that we never thought about at all.  And I think that would just be fantastic.  Tom, I think that you made extremely good point that we really do not know what we should  do because we are working in fast prototyping mode.  How I know that it's 192 frame formats, at least 192, because I encounter 192, but somewhere  there may be more.  And the same we have with structures and metadata.  We do not, as long as we do not have even basic metadata set for all PDB, we really  do not know what we can do with that.  Yes, and to our images.  Agreed.  So this is a repeat of a comment I made earlier, but outside of the session.  Instead of worrying about the fact that there are so many different frame formats, why not  make it part of the archiving process that everything gets transformed into the same  frame format?  Then it's just one place that has the same frame formats, and you don't need to worry  about it.  Right?  Then the PDB has only CBF.  I know it doesn't have to be CBF, but the PDB only stores one type of frame, and then  everyone can reprocess it using whichever software they want.  So you essentially turn the frame format problem over to the archivers.  Yeah, actually, I am personally totally in favor of this idea.  And my interpretation of that is that the software developers like Volatic and Company  have a mechanism for taking 192 different whatever, and turning them into something internal  that is one version.  So yeah, looking for Volatic to answer this.  And then presumably that thing could be written out.  And presumably that thing could be standardized because there's only five people doing it  instead of 300 people doing it.  And I think if that would be possible, it would be really fantastic.  So I think that future proofs the problem at least a little bit.  And I had one second comment, which is really minor.  And that's if you're starting to talk about taking multiple versions of a file and then  giving data to choose which one is the best one, do you then need to introduce a new phraseology?  I mean, you've got data about data about data.  Maybe that's not metadata, it's hyper data.  Yeah, perhaps so.  And you know, better is the wrong word to be using for sure.  Yes, obviously, internally we have our own frame format.  Yes.  Otherwise we would have gone crazy.  Long time ago.  I mean, I am crazy.  I realize that.  But that's great.  But and we have a procedure to write in that format.  Unfortunately, it's not so easy, yes, to write because it's relatively easy to change the  Goniostad outside if you are learning after some time that this Goniostad is not accurate.  You would have to rewrite all this data.  Yes, you would have to rewrite all this.  I see.  So your interpretation, your internal data is actually interpreted data that depends  upon parameters, which may or may not be.  All parameters which are in Dev side.  So it's a little far from the original, which is a bit, you don't think it's risky.  It sounds risky to me, yes.  Yeah.  And you see that it's even more complicated because look, we are getting data from, let's  say, BIM line A. Yes, and we are doing, and they usually, they sent one data set, they  are saying it's extremely urgent because users are coming tomorrow.  We have to have something.  We are doing that.  We are giving them, three months later, we are learning that it's really not the one  frame format.  There are two frame formats.  Yes, and already some people use this other frame format.  Yes, I mean, you see that as an example of the company which does not exist as a valid  company is ADSC detector, extremely good detector, which had software billing and hardware  billing.  And companies, they do not know which one is better.  So they are saying, okay, we will leave that to users, but they are not saying anything  about that.  Yes, we are getting data and error density, so-called error density parameter is different  for this too.  It's different by the factor of five.  And overnight, you are learning that some people who use, let's say, software billing  when you got hardware billing, they have worse resolution than they obtained before.  But these are the problems which we have, I mean, we, the people who develop software,  processing software in real life.  So can I propose an alternative version?  So still, a user who takes your software and produces intensities out of it, you and the  user have gone through a process there with a little handshake that says you understand  their inputs and are able to interpret those inputs in terms of parameters that you need,  namely all the angles, all the geometry, whatever.  Could you not write out that information, which I think you are doing, that information  which is sufficient to reproduce the experiment, the analysis with your software, and also  with another, you are writing it out so that XDS or somebody else can also do that.  That is, that's halfway there and it seems like depositing that would be very, very beneficial.  Michael?  Right, Michael, I'll ask Alamos.  So the question I have has more to do with my perspective, viewing the protein structure  as an ensemble rather than a single structure.  So in a kind of Bayesian way, you can define a likelihood for different models depending  on the data and some models more likely than others.  So the approach that you described seems fine of reporting one structure to a user assuming  that this likelihood function is very sharp.  But if it is broader, then it might be good to actually force a user to pay attention to  multiple models instead of presenting one, especially if what they are going to do with  it will yield a different answer depending on which one they choose.  So what you described is like a huge advance over the way things are now.  What I am describing is something that I guess would go somehow beyond that.  So how far are we from doing this next step?  So Mike, I totally agree.  I didn't talk about that today, but actually that is an issue that I feel is very, very  important.  Basically, we shouldn't be representing these structures as a single structure is one in  the beginning.  That is wrong.  It is not what is in the crystal.  The crystal has millions of structures in it.  And some representation of what those structures are is where we have to eventually go.  And so I totally agree with your point.  So I would like to also consider the end point here.  So we have the better process, the continual improvement.  So we get to this better use of the data through the communal efforts.  So now to pose the well-rehearsed question on behalf of Derwood Crookshank, who passed  away a few years ago, who was first paper on understanding the precision of coordinates  in crystal structures was 1949.  So in looking at what we do, he was very concerned.  It is perhaps more the matter of the graphics representation software that we all use of  having a uniform view of the precision of non-bonded distances, two decimal places.  And he said that just cannot be true.  Just look at the B factors.  If two atoms are potentially interacting in their 80 angstrom square, then looking at  integer precision of their distances apart, let alone two decimal places.  So we have our half a billion downloads per year of the fruits of our research.  And these are not specialist in our techniques.  And they go away with a false precision.  So he thinks, and I agree with him, forced to agree with him, he had me on the ropes,  this is a big issue for our field.  And maybe it's solvable by education.  And all that the PDB is doing, like we saw a precision on a wavelength to seven decimal  places this morning.  And it's just the over-enthusiastic computer scientist providing all of these columns for  our experimental parameters and our results.  But there is an issue here.  I think he's right.  Yes, it's an ensemble of structures because we have millions and millions in our sample.  But that variation, let's perhaps avoid the word coordinate error, but coordinate variation  tells us that we have to provide to non-expert users a much better feel for what the atom  placement precision is.  So I once again totally agree with this.  So actually, let me make a little point here.  We need to differentiate between two types of imprecision or uncertainty.  One is in the actual crystal, there's millions and millions of different protein molecules  or whatever they are.  And everyone is a little bit different.  And so that's describing what's actually in the crystal.  And also, these are not necessarily just Gaussian distributions that are represented simply by  nice little b-factors.  They may be static disorder, many different things that can be going on.  So what's in the crystal is fairly complicated and is not well represented by a single model.  It would be better represented by a single model plus some numbers that say what the  variation is.  That would be a better step, which we aren't even doing.  So that would be the first order improvement would be what's the variability in the crystal  for those numbers.  And then second order would be something about what are the different conformational ensembles  that describe most of the variation or something like that.  So that's where Mike's going to.  So that's one thing.  But separate issue is we're using our crystallographic tools to try to describe the structure.  And we come up with models.  And there's some, if we did the same experiment twice, we come up with different models.  And how much do those vary?  And so surely that second variation is a lower bound on our uncertainty of what is in the  crystal itself.  So the crystal varies, and we don't even know what's in the crystal.  So we need to describe both of those things.  And I think ultimately we're going to describe both of those by some way of talking about  ensembles of structures.  And it may be molecular dynamic simulations that kind of give us a handle on how to describe  that.  It may be we have a limited number of ensembles that describe most of the variability, and  that's what we're going to report.  But absolutely, the end user has to understand what they can know from what we're telling  them and what they can't know.  And right today, you give them a single structure, and they look at the distance between two  atoms, and they think these things must be hydrogen bonded because they're the right  distance apart.  But maybe it's that way only 1,000th of the time.  And most of the time there's some other completely different configuration.  And we're not telling them that today.  We're not even telling them.  It's ridiculous.  So I totally agree.  I hope the Commission on Biological Micro Molecules will take this up on their leadership.  Great.  So again, it's just another comment.  You said if you take two crystals and do the diffraction experiment, you get two different  models.  If you take the same crystal and the same data and process it in two different ways, you  get two different models.  Yes.  So it's even worse than two different crystals.  Yes, absolutely true.  Absolutely true.  Actually, the good thing is, so we studied this a little bit a few years ago with just  macromolecular structures and just reinterpreting the exact same crystal data, just running  it through our software a bunch of times with different random seeds.  And the good thing is, in a macromolecule, the core of the molecule, they just superimpose  extremely closely.  You just can't do any, you can't interpret these structures any other way other than  to basically put these atoms in pretty much the same places.  But the stuff on the surface varies a huge amount.  Yes.  Okay.  Thank you very, very much indeed, Tom.  And I'm sure everybody wants to show the usual appreciation.  Thank you very, very much.  So Brian will reconfirm the logistics, which is seven o'clock minibus outside the front.  And enjoy, I don't know, an hour and a half in the pool or whatever takes your fancy.  Thank you.
RealisingtheLivingPDB_48_95.png RealisingtheLivingPDB_96_215.png RealisingtheLivingPDB_216_287.png RealisingtheLivingPDB_288_983.png RealisingtheLivingPDB_984_1031.png RealisingtheLivingPDB_1032_2543.png RealisingtheLivingPDB_2544_4271.png RealisingtheLivingPDB_4272_5975.png RealisingtheLivingPDB_5976_6443.png RealisingtheLivingPDB_6444_8195.png RealisingtheLivingPDB_8196_10331.png RealisingtheLivingPDB_10332_11567.png RealisingtheLivingPDB_11568_15239.png RealisingtheLivingPDB_15240_17579.png RealisingtheLivingPDB_17580_18599.png RealisingtheLivingPDB_18600_20687.png RealisingtheLivingPDB_20688_21311.png RealisingtheLivingPDB_21312_21635.png RealisingtheLivingPDB_21636_25607.png RealisingtheLivingPDB_25608_29639.png RealisingtheLivingPDB_29640_29759.png RealisingtheLivingPDB_29760_39311.png RealisingtheLivingPDB_39312_44903.png RealisingtheLivingPDB_44904_61907.png RealisingtheLivingPDB_61908_61991.png RealisingtheLivingPDB_61992_62039.png RealisingtheLivingPDB_62040_62135.png RealisingtheLivingPDB_62136_62183.png RealisingtheLivingPDB_62184_62243.png RealisingtheLivingPDB_62244_62291.png RealisingtheLivingPDB_62292_62339.png RealisingtheLivingPDB_62340_62458.png