Editing CASP (section)

==Evaluation==
The primary method of evaluation<ref>{{cite journal |vauthors=Cozzetto D, Kryshtafovych A, Fidelis K, Moult J, Rost B, Tramontano A |title=Evaluation of template-based models in CASP8 with standard measures |journal=Proteins |volume=77 Suppl 9 |issue=Suppl 9 |pages=18–28 |year=2009 |pmid=19731382 |pmc=4589151 |doi=10.1002/prot.22561}}</ref> is a comparison of the predicted model [[Alpha carbon|α-carbon]] positions with those in the target structure. The comparison is shown visually by cumulative plots of distances between pairs of equivalents [[Alpha carbon|α-carbon]] in the alignment of the model and the structure, such as shown in the figure (a perfect model would stay at zero all the way across), and is assigned a numerical score [[Global distance test|GDT-TS (Global Distance Test—Total Score)]] describing percentage of well-modeled residues in the model with respect to the target.<ref name="Zemla2003">{{cite journal |vauthors=Zemla A |title=LGA: A method for finding 3D similarities in protein structures |journal=Nucleic Acids Research |volume=31 |issue=13 |pages=3370–4 |date=July 2003 |pmid=12824330 |pmc=168977 |doi=10.1093/nar/gkg571}}</ref> Free modeling (template-free, or ''de novo'') is also evaluated visually by the assessors, since the numerical scores do not work as well for finding loose resemblances in the most difficult cases.<ref>{{cite journal |vauthors=Ben-David M, Noivirt-Brik O, Paz A, Prilusky J, Sussman JL, Levy Y |title=Assessment of CASP8 structure predictions for template free targets |journal=Proteins |volume=77 Suppl 9 |issue=Suppl 9 |pages=50–65 |year=2009 |pmid=19774550 |doi=10.1002/prot.22591 |s2cid=16517118}}</ref> High-accuracy template-based predictions were evaluated in CASP7 by whether they worked for molecular-replacement phasing of the target crystal structure<ref>{{cite journal |vauthors=Read RJ, Chavali G |title=Assessment of CASP7 predictions in the high accuracy template-based modeling category |journal=Proteins |volume=69 Suppl 8 |issue=Suppl 8 |pages=27–37 |year=2007 |pmid=17894351 |doi=10.1002/prot.21662 |s2cid=33172629 |doi-access=free}}</ref> with successes followed up later,<ref>{{cite journal |vauthors=Qian B, Raman S, Das R, Bradley P, McCoy AJ, Read RJ, Baker D |title=High-resolution structure prediction and the crystallographic phase problem |journal=Nature |volume=450 |issue=7167 |pages=259–64 |date=November 2007 |pmid=17934447 |pmc=2504711 |doi=10.1038/nature06249 |bibcode=2007Natur.450..259Q}}</ref> and by full-model (not just [[Alpha carbon|α-carbon]]) model quality and full-model match to the target in CASP8.<ref>{{cite journal |vauthors=Keedy DA, Williams CJ, Headd JJ, Arendall WB, Chen VB, Kapral GJ, Gillespie RA, Block JN, Zemla A, Richardson DC, Richardson JS |display-authors=6 |title=The other 90% of the protein: assessment beyond the Calphas for CASP8 template-based and high-accuracy models |journal=Proteins |volume=77 Suppl 9 |issue=Suppl 9 |pages=29–49 |year=2009 |pmid=19731372 |pmc=2877634 |doi=10.1002/prot.22551}}</ref>

Evaluation of the results is carried out in the following prediction categories:

* [[tertiary structure]] prediction (all CASPs)
* [[Protein structure prediction#Secondary structure|secondary structure prediction]] (dropped after CASP5)
* prediction of [[protein complex|structure complexes]] (CASP2 only; a separate experiment—[[Critical Assessment of Prediction of Interactions|CAPRI]]—carries on this subject)
* residue-residue contact prediction (starting CASP4)
* [[Intrinsically disordered proteins|disordered regions]] prediction (starting CASP5)
* [[protein domain|domain]] boundary prediction (CASP6–CASP8)
* [[Function (biology)|function]] prediction (starting CASP6)
* model quality assessment (starting CASP7)
* model refinement (starting CASP7)
* high-accuracy template-based prediction (starting CASP7)

Tertiary structure prediction category was further subdivided into:

* [[homology modeling]]
* fold recognition (also called [[protein threading]]; note that this naming is incorrect as threading is a method)
* ''de novo'' structure prediction, now referred to as 'New Fold' as many methods apply evaluation, or scoring, functions that are biased by knowledge of native protein structures, such as an [[artificial neural network]].

Starting with CASP7, categories have been redefined to reflect developments in methods. The 'Template based modeling' category includes all former comparative modeling, homologous fold based models and some analogous fold based models. The 'template free modeling (FM)' category includes models of proteins with previously unseen folds and hard analogous fold based models. Due to limited numbers of template free targets (they are quite rare), in 2011 so called CASP ROLL was introduced. This continuous (rolling) CASP experiment aims at more rigorous evaluation of template free prediction methods through assessment of a larger number of targets outside of the regular CASP prediction season. Unlike [[LiveBench]] and [[EVA (benchmark)|EVA]], this experiment is in the blind-prediction spirit of CASP, i.e. all the predictions are made on yet unknown structures.<ref>{{cite journal |vauthors=Kryshtafovych A, Monastyrskyy B, Fidelis K |title=CASP prediction center infrastructure and evaluation measures in CASP10 and CASP ROLL |journal=Proteins |volume=82 Suppl 2 |pages=7–13 |date=February 2014 |issue=2 |pmid=24038551 |pmc=4396618 |doi=10.1002/prot.24399}}</ref>

The CASP results are published in special supplement issues of the scientific journal ''Proteins'', all of which are accessible through the CASP website.<ref>{{cite web |title=CASP Proceedings |url=http://predictioncenter.org/index.cgi?page=proceedings}}</ref> A lead article in each of these supplements describes specifics of the experiment<ref>{{cite journal |vauthors=Moult J, Fidelis K, Kryshtafovych A, Rost B, Hubbard T, Tramontano A |title=Critical assessment of methods of protein structure prediction-Round VII |journal=Proteins |volume=69 Suppl 8 |issue=Suppl 8 |pages=3–9 |year=2007 |pmid=17918729 |pmc=2653632 |doi=10.1002/prot.21767}}</ref><ref>{{cite journal |vauthors=Moult J, Fidelis K, Kryshtafovych A, Rost B, Tramontano A |title=Critical assessment of methods of protein structure prediction - Round VIII |journal=Proteins |volume=77 Suppl 9 |issue=Suppl 9 |pages=1–4 |year=2009 |pmid=19774620 |doi=10.1002/prot.22589 |s2cid=9704851 |doi-access=free}}</ref>
while a closing article evaluates progress in the field.<ref>{{cite journal |vauthors=Kryshtafovych A, Fidelis K, Moult J |title=Progress from CASP6 to CASP7 |journal=Proteins |volume=69 Suppl 8 |issue=Suppl 8 |pages=194–207 |year=2007 |pmid=17918728 |doi=10.1002/prot.21769 |s2cid=40200832 |doi-access=free}}</ref><ref>{{cite journal |vauthors=Kryshtafovych A, Fidelis K, Moult J |title=CASP8 results in context of previous experiments |journal=Proteins |volume=77 Suppl 9 |issue=Suppl 9 |pages=217–28 |year=2009 |pmid=19722266 |pmc=5479686 |doi=10.1002/prot.22562}}</ref>