A sequence-based aggregation predictor reads a string of amino acids and returns a number per residue. The output has one axis, sequence position, and no axis for your buffer, your peptide concentration, your temperature or whether the vial was shaken. That asymmetry determines what the output is worth. A per-residue propensity profile is a statement about a sequence, and aggregation is an event in a solution, so the profile can rank candidates but cannot close a question.

What the score is actually computed from

The empirical footing is a correlation established on mutation series. Chiti and colleagues showed that the intrinsic effects of specific mutations on the rates of aggregation of unfolded polypeptide chains can be correlated to a remarkable extent with changes in simple physicochemical properties such as hydrophobicity, secondary structure propensity and charge, and presented this as enabling prediction of the effects of mutations on the aggregation propensity of any polypeptide chain [1]. Two qualifiers in that sentence run through the whole field: the chains were unfolded, and the quantity correlated was a change in rate on mutation rather than an absolute yes or no.

Two families of tool were built on that footing. TANGO is a statistical mechanics algorithm based on the physico-chemical principles of β-sheet formation, extended by the assumption that the core regions of an aggregate are fully buried; it evaluates competing conformational states rather than summing a residue scale. Its authors reported accurate prediction of aggregation for a data set of 179 peptides compiled from the literature and for a further set of 71 peptides derived from human proteins, and reported that the algorithm discriminates between β-sheet propensity and aggregation [2]. AGGRESCAN takes the other route: an aggregation-propensity scale for the natural amino acids derived from in vivo experiments, combined with the assumption that short and specific sequence stretches modulate protein aggregation [3]. One is a thermodynamic model carrying an explicit structural assumption; the other is a lookup table carrying an explicit locality assumption. They often agree, because both encode the same handful of residue properties — which makes agreement between them weaker evidence than it looks. Both are also built to be run comparatively rather than absolutely: the AGGRESCAN description covers analysis of the effect of mutations on aggregation propensities and comparison of the aggregation properties of different proteins or protein sets, and the authors report that it provided new insights into the differential aggregation properties displayed by globular proteins, natively unfolded polypeptides, amyloidogenic proteins and proteins found in bacterial inclusion bodies [3]. Comparison across a set is the mode these tools are strongest in, and the mode most often skipped.

The aggregation-prone region, and what a hit does not mean

Whatever the scoring function, the output is smoothed over a window and a contiguous run above threshold is called an aggregation-prone region. That construct is shared across the field, and so is its principal weakness: an APR is a property of the sequence, and the sequence is not alone in the tube.

Linding and colleagues used TANGO to compare a set of globular proteins derived from SCOP against 296 experimentally verified, non-redundant intrinsically disordered proteins, and found that globular proteins contain almost three times as many aggregation-nucleating regions as the disordered set. They also concluded that β-aggregation is not determined by hydrophobicity and β-sheet propensity alone, and that globular structures possess an appreciable amount of structural frustration [4]. Folding, on that reading, buys solubility by burying exactly the segments a sequence-only predictor flags.

Tartaglia and colleagues set out to close that gap with a method extending prediction to conditions under which the molecules concerned can be folded or contain a significant degree of persistent structure, estimating simultaneously the propensity for folding and for aggregation and the way the two compete. Their conclusion is the sentence to keep on the bench: regions of a protein with high intrinsic aggregation propensity can be identified in a robust manner, but the structural context of such regions in the monomeric form is crucial for determining their actual role in the aggregation process [5].

The structure-based literature hits the same wall from the other side. A genome-wide analysis by Goldschmidt and colleagues, examining segments able to form a steric zipper — two self-complementary β-sheets forming the spine of an amyloid fibril — found that self-complementary segments are present in almost all proteins, yet not all proteins form amyloids; the authors' implication was that chaperoning effects have evolved to constrain those segments from interacting with each other [6]. If nearly every sequence carries a candidate segment, then the bare presence of a predicted APR is close to uninformative. What the profile gives you is where, and how strongly relative to the rest of the same sequence.

The same study makes a second point that constrains how a composition-weighted scale can be read. Validating fibrillogenic predictions on RNase A as a model system, the authors demonstrated that a specific residue order is required for fibre formation, and identified conformational freedom of the self-complementary segment — its ability to interact with other molecules — as a separate enabling factor alongside the presence of the segment itself [6]. Order and accessibility are both outside what a bare amino acid composition can express, so two sequences with identical residue content can score alike and behave differently.

Structure-based scoring asks a narrower, sharper question

The 3D profile method begins from the crystal structure of the cross-β spine formed by the peptide NNQQNY. Each six-residue segment of the query is mapped onto an ensemble of templates generated from that structure by small displacements of one of the two intermeshed β-sheets relative to the other; the energy of each mapping is evaluated with ROSETTADESIGN, the lowest-energy match for a given peptide is taken as the putative prediction, and fibril formation is predicted when that energy falls below a threshold. The authors reported that the method can reach an accuracy of approximately 80% with a P value of approximately 10-12 when a conservative energy threshold is used to separate peptides that form fibrils from those that do not [7].

Read the question that accuracy figure answers. It is: can this hexapeptide be threaded onto this family of steric-zipper templates at low energy — a structural compatibility test on six residues, scored against a fibril-forming or non-fibril-forming label. It is not: will my 24-mer drop out of solution overnight at 2 mg/mL. The threshold is a knob, and the figure quoted above is tied to the conservative setting those authors chose; moving it trades sensitivity against specificity along the same curve, which is why a percentage detached from its threshold means nothing.

Amyloid-specific and general aggregation are different targets

Aggregation produces β-sheet-like assemblies that adopt either a variety of amorphous morphologies or ordered amyloid-like structures, and the difference is not cosmetic: amyloid and amorphous β-sheet aggregates have different chaperone affinities, accumulate in different cellular locations and are degraded by different mechanisms. Waltz was built by experimentally exploring the sequence space of amyloid hexapeptides and fitting a position-specific scoring matrix to the resulting data, explicitly so that users could better distinguish amyloid sequences from amorphous β-sheet aggregates [8].

Look at the labels behind such a matrix. WALTZ-DB 2.0 collects experimentally determined amyloid-forming hexapeptide sequences, including experimental validation of an in-house dataset of 229 hexapeptides using electron microscopy and thioflavin T binding assays, plus 98 amyloid-forming peptides manually curated from the literature [9]. A classifier fitted to that corpus has learned to answer one question: does a six-residue peptide, assayed under those conditions, give a thioflavin T signal and fibrils on the grid. It has learned nothing about amorphous precipitation, nothing about oligomers that never reach fibril, and nothing about a 30-residue analogue whose flanking residues are the reason it stays in solution.

The hexapeptide work also separates two axes that a single score cannot carry at once. Saturation mutagenesis of the de novo-designed amyloid peptide STVIIE revealed a position dependence on mutation of amyloid fibril formation, with both very tolerant and very restrictive positions found within one amyloid sequence, and — the result that should temper any leap from propensity score to observable — that in this system mutations which accelerate β-sheet polymerisation do not always lead to an increase of amyloid products, with abundant fibrils typically found for mutants that polymerise slowly [10]. A model trained on a binary fibril label carries no rate information, and a model tuned to rank kinetics is not a yield prediction.

A third question hides under the same phrase. CamSol performs a rapid computational screen of tens of thousands of mutations to identify those with the greatest impact on the solubility of a target protein while maintaining its native state and biological activity, and was demonstrated on a single-domain antibody, where the authors reported that the method predicts solubility changes upon mutation with great accuracy [11]. That is solubility of a folded domain, evidenced on an antibody rather than on a peptide — a legitimate target, and a different one from amyloid propensity. When a sequence is described as having been checked for aggregation, establish which of the three questions was asked.

What the published benchmarks establish

They establish that short-peptide classification is tractable, that residue-level annotation of whole proteins is much harder, and that any accuracy figure belongs to the dataset it was measured on.

AmyloGram trained random forest classifiers on n-grams, searching 524,284 reduced amino acid alphabets of three to six letters for the representation giving the best cross-validated performance, then benchmarked the result against the most popular tools for the detection of amyloid peptides using an external data set, obtaining AUC 0.90 and MCC 0.63 [12]. That is a strong figure, and it is a figure for discriminating amyloidogenic from non-amyloidogenic short peptides on a held-out set.

Now the other task. AMYLPRED2 combines eleven individual methods into a consensus prediction of aggregation-prone peptides in proteins from sequence alone, on the stated reasoning that combining tools allows comparison of the different algorithms and the production of more objective and accurate results; its authors report that the consensus functions better than the individual aggregation-prediction algorithms [13]. Tested against a set of 33 amyloidogenic proteins for which experimental data were available, that consensus reached sensitivity 39.27%, specificity 84.48%, a Q value (the mean of sensitivity and specificity) of 61.88% and MCC 0.22 — values from Table 1 of the paper, which is openly accessible [13]. Same tool family, same underlying physics, a far harder problem, because now the prediction must say which residues, inside a folded protein, against an annotation whose own boundaries are uncertain.

An independent comparison catalogues what the field is actually predicting: these tools aid prediction of aggregation propensity and of amyloidogenicity, as well as the identification of aggregation-prone regions — three outputs that get quoted interchangeably and are scored on different data. That comparison put nine prediction tools through a performance analysis on several non-redundant datasets ranging from hexapeptides to protein sequences, and from amyloidogenic antibody light chains to soluble protein sequences; the authors summarised the outcome as revealing both the robustness of the current prediction tools and the scope for improvement in their predictive performances [14]. Note the analyte classes in that list — antibody light chains are intact immunoglobulin domains, and a soluble-protein set asks about solubility rather than fibril formation. Two things follow for you. First, a headline number quoted without its dataset is not a number; a tool can be excellent at hexapeptides and unhelpful on light chains without any contradiction. Second, several of these predictors were fitted on the same public short-peptide corpora that short-peptide benchmarks are drawn from, which is precisely why the external-set discipline in the AmyloGram evaluation matters when you read a reported score.

The conditions the profile has no axis for

The clearest demonstration that solution conditions matter is a model that had to include them. DuBay and colleagues used knowledge of the amino acid sequence and of the experimental conditions to reproduce the in vitro aggregation rates of a wide range of unstructured peptides and proteins with a correlation coefficient of 0.92 over five orders of magnitude [15]. The conditions term is not decoration on that result: five orders of magnitude of rate is the range being spanned, and sequence alone is not what spans it.

Ionic strength by itself moves the rate steeply. Buell and colleagues probed the steady-state kinetics of fibrillar protein self-assembly and detected how it is modulated by changes in solution ionic strength, describing the reaction as an exquisitely sensitive probe of electrostatic effects in protein–protein interactions because of the exponential modulation of the reaction rate by electrostatic forces, and resolving a hierarchy of electrostatic effects that control protein aggregation. The material in those tubes was seeded elongation of fibrils of bovine insulin, the PI3K-SH3 domain, human glucagon and human β2-microglobulin, with the quantitative electrostatic analysis run over 0 to 20 mM added NaCl on a 10 mM HCl background and the salt series extended well beyond it [16]. None of those is the peptide on your bench; what transfers is the form of the dependence, not its magnitude.

A per-residue propensity profile has no axis for any of this. If a tool does not expose pH, ionic strength and peptide concentration as parameters, then two buffers that give you two different experimental outcomes return one identical profile. The corollary deserves stating plainly: a predictor cannot be wrong about your buffer, because it was never told about it. When a sequence with a clean profile crashes out at pH 5.5 in 150 mM NaCl, that is not a prediction failure worth reporting — it is a question the tool was never asked.

Using a prediction to rank, not to conclude

  • Fix the parameters before you look at the answer. Window length, smoothing and threshold all move APR boundaries, and as noted for the structure-based method above, a reported accuracy belongs to the threshold its authors chose. Record the tool, its version and every setting alongside the profile, or the profile is not reproducible.
  • Run tools from different families. A residue-scale method [3], a thermodynamic model [2] and an amyloid-trained matrix [8] fail in different ways. Treat their intersection as the shortlist and their union as the watch list — the logic that motivated the consensus approach [13], carried with the same caveat about how that consensus scored against protein-level annotation.
  • Compare within a congeneric series, not across chemistries. Ranking analogues that differ by a few substitutions is the defensible use, because whatever systematic error the scale carries is shared across the set. Ranking two unrelated sequences on absolute score is not.
  • Treat a clean profile as weak evidence. Given the sensitivity reported above for consensus prediction on whole proteins, an absence of flagged regions carries little information, and “predicted non-aggregating” does not belong in a results section.
  • Take every candidate to a measurement. The prediction generates the sample list; thioflavin T kinetics, light scattering and turbidity generate the result. The companion guides on aggregation and fibrillation in solution and on thioflavin T kinetics set out what those readouts license, including the buffer, concentration and agitation variables the predictor never saw.

Stated precisely, a sequence-based prediction licenses one claim: this segment scores higher on this function, under this parameter set, than that segment does. Everything downstream of that — whether it aggregates, how fast, into which morphology, above what concentration — is an experiment. Used this way the tools are cheap and genuinely useful: they turn a panel of twenty analogues into a ranked list of five worth putting on the plate first, and they nominate residues to change when the answer comes back badly, which is the use the mutation-effect correlation described at the outset was built for. Used the other way, as a substitute for the plate, they generate confident sentences with no measurement behind them.