Reproducibility in Peptide Research: Where Results Fail to Replicate

A result that does not replicate is not necessarily a wrong result. It may be a correct result about a different thing than the one named in the paper — a different cell line, a different molecule, a different number of independent observations than the statistics assumed. Most replication failures in peptide work are of that kind. Nobody falsified anything. The experiment simply was not the experiment the methods section described, and the discrepancy lived somewhere nobody was asked to report.

This guide is about where those discrepancies hide: in reagent identity, in cell identity, in how much peptide is actually in the vial, in what counted as a replicate, and in what the paper had to disclose. It is written for someone deciding whether to believe a figure, including a figure they produced themselves.

What replication attempts have actually found

The most systematic evidence comes from outside peptide chemistry. The Reproducibility Project: Cancer Biology repeated 50 experiments from 23 high-impact papers, generating data on 158 effects ^[1]. Of the original effects, 136 were positive and 22 were null, and 117 were reported as numerical values while 41 were reported only as representative images ^[1]. Comparing effect sizes for the positive effects, the median replication effect size was 85% smaller than the median original effect size, and 92% of replication effect sizes were smaller than their originals ^[1]. Using five binary success criteria, 40% of positive-effect replications (39/97) succeeded on three or more criteria; for null effects the figure was 80% (12/15); combined, 46% (51/112) ^[1].

Two cautions belong with those numbers. They describe preclinical cancer biology, not peptide assay work, and the project's own authors note that a failed replication does not disconfirm a finding — it indicates that further investigation is needed to establish its reliability ^[1]. The reason to cite them here is the pattern rather than the rate: original effects were mostly larger than what came back, and a substantial fraction of the original effects reached the replicators only as images rather than as numbers.

Reagent identity: the antibody problem, imported into peptide work

Antibodies are the best-documented reagent-identity failure, and the documentation generalizes. A 2024 review by 18 authors describes an "antibody characterization crisis": many antibodies used in biomedical and clinical research have not been adequately characterized, which the authors argue casts doubt on results reported in many papers, a problem compounded by a lack of suitable control experiments ^[2]. The review is explicitly a call for change addressed to researchers, journals, vendors, repositories and funders, not a measurement of how many results are wrong ^[2].

What the reporting looks like in practice has been measured. A study of 551 articles containing western blot images — 151 from neurosciences and 400 from cell biology journals, drawn from the top quartile of journals in each field — found that most published blots are cropped and that source data are usually not supplied in the supplement ^[3]. Blots published with visible molecular weight markers were rare. Methods sections frequently omitted the amount of protein loaded, the blocking step, and the antibody labeling protocol, and identifiers such as company, catalog number or RRID were omitted frequently for primary antibodies and regularly for secondary antibodies ^[3].

For a peptide experiment, the same logic applies to the peptide. A named sequence is not an identifier. A named sequence plus a supplier, a lot number and a certificate is.

Cell line misidentification

The canonical case is unusually well resolved. Melanoma line M14 and breast carcinoma line MDA-MB-435 were indicated by a variety of analytical approaches to originate from a common donor, meaning one of them had at some point ceased to correspond to its reported donor. Working from M14 samples cryopreserved in December 1975 — before MDA-MB-435 was established in culture — plus a lymphoblastoid line from the same donor and preserved donor serum, an 18-author study used short tandem repeat (STR) profiling and cytogenetics to demonstrate that M14 is the authentic line and MDA-MB-435 is the misidentified one ^[4].

STR profiling is the standard tool, and its thresholds are empirical rather than absolute. A 22-author analysis of 2,279 STR profiles from four cell banks, of which 1,157 fell into related sets (duplicate holdings, legitimately related samples, or misidentified lines), found that eight core STR loci plus amelogenin unequivocally authenticated 98% of those related sets ^[5]. Two simple match algorithms separated related samples at 80% match or above from unrelated samples below 50%, with a small overlap band at 50–79% match arising mostly from lines known to display variable STR profiles ^[5]. That overlap band is the honest part of the method: a match score is evidence, not a verdict.

Reporting appears to help. Text-mining of roughly two million papers in PubMed Central identified 305,161 unique cell-line names across 150,459 articles; an estimated 8.6% of those cell lines appeared on the ICLAC/Cellosaurus list of problematic lines, compared with 3.3% of the cell lines in the 634 papers that included RRIDs ^[6]. The authors describe this as suggesting that RRID use is associated with lower reported use of problematic cell lines — an association, in an observational text-mining sample, not a demonstrated causal effect ^[6].

Contamination is a separate axis from identity. Mycoplasmas remain a major cell-culture problem, are resistant to most antibiotics commonly used in culture, and can produce a very wide variety of effects on the cultures they infect ^[7]. A mycoplasma-positive culture is still the right cell line; it is simply no longer the same biological system that the assay was designed around.

Undisclosed peptide content

This is the failure mode most specific to peptide work, and the easiest one to make invisible. A lyophilized vial contains peptide, counter-ions, residual water, and whatever else the process left behind. The mass on the label is not the mass of peptide.

How large that gap can be is not hypothetical. Quantitative proton NMR analysis of two custom synthetic peptides sourced commercially, DR and DRVYI, uncovered mannitol as an undeclared constituent at 20% and 43% w/w ^[8]. Mannitol is highly polar and UV-transparent, which is precisely why a routine LC method with UV detection can be blind to it; the authors note that benchtop qHNMR at 60 MHz was sufficient to detect and quantify it ^[8]. Their conclusion is a methodological one: experimental verification supersedes trust, for research chemicals as much as for pharmaceutical ingredients ^[8].

What the arithmetic does to a stock

Suppose a 43% w/w non-peptide fraction, as in the higher of the two reported cases ^[8]. Weighing 5.00 mg of that solid yields at most 5.00 × 0.57 = 2.85 mg of peptide — at most, because assuming the remaining 57% is entirely peptide is itself an assumption. A stock prepared as though it were 1.00 mM is then no more than 0.57 mM. Stated the other way, with the true concentration as the denominator, the nominal figure overstates the real one by at least (1.00 − 0.57) / 0.57 = 0.43 / 0.57 = 75.4% — at least, because 0.57 mM is itself a ceiling on what is actually in the vial.

The same arithmetic runs on ordinary, fully disclosed lot-to-lot variation. If one lot is certified at 85.0% peptide content and the next at 72.0%, then weighing the same mass from each delivers (85.0 − 72.0) / 72.0 = 13.0 / 72.0 = 18.1% more peptide from the first. A vial labeled 10 mg gross at 78.5% content holds 7.85 mg of peptide; treating it as 10 mg inflates every derived concentration by (10 − 7.85) / 7.85 = 2.15 / 7.85 = 27.4%. In each case the true value is the denominator, and getting that backwards understates the error.

Two independent measurements address this. Amino acid analysis gives an absolute quantitation of a standard peptide: the material is hydrolyzed in 6 M HCl and the released amino acids are quantified, with the peptide concentration taken as the sum of the individual amino acid concentrations expected from the known sequence. One published implementation used certified isotopically labeled (carbon-13 and/or nitrogen-15) amino acids as internal standards with reversed-phase UPLC-MRM-MS, quantifying 17 amino acids including oxidized cysteine and methionine without derivatization or external calibration ^[9]. Separately, counter-ion content is itself measurable: a 2025 study validated FT-IR, fluorine-19 NMR and HPLC with evaporative light-scattering detection against ICH guidelines to quantify residual trifluoroacetate, and found that exchange to chloride by freeze-drying from 10 mM HCl was optimal among the conditions tested, with no observed impact on peptide purity ^[10].

Counter-ion identity is not a bookkeeping detail. Comparing acetate, hydrochloride and trifluoroacetate salts of five antimicrobial peptides, one study reported substantial differences between salt forms in measured antistaphylococcal activity, hemolytic activity against human red blood cells, and cytotoxicity in the HaCaT cell line — but explicitly noted that the pattern was not consistent across the peptides tested ^[11]. The trifluoroacetate study cited above also used a liposomal fluorescence assay to test the counter-ion's influence on passive membrane permeability, and found that effects on permeability coefficients depended on both peptide sequence and salt form ^[10]. Neither result licenses a rule about which salt is better; both license the narrower claim that the salt form belongs in the methods section, because two labs using the same sequence may not be using the same material.

Pseudoreplication: what counts as one experiment

Statistical inference rests on independent replication of the entity-intervention pair — the thing that was perturbed, paired with the perturbation ^[12]. When the wrong pair is replicated, the sample size is inflated without any new independent evidence being collected. In a survey of a random sample of animal experiments published between 2011 and 2016 in which interventions were applied to parents and effects examined in offspring, only 22% of studies (95% CI 17–29%) replicated the correct entity-intervention pair; 46% (95% CI 38–53%) showed pseudoreplication, and 32% (95% CI 26–39%) reported too little to judge ^[12]. The authors distinguish biological units, experimental units and observational units, and argue that keeping them separate is what tells you where replication has to occur ^[12].

In cell work the usual instance is counting measurements rather than independently treated containers. The framework in ^[12] puts the experimental unit at the entity randomly and independently assigned to a treatment condition — a well, a culture dish or another plastic container — rather than at the individual cell, because the treatment reaches all the cells in a well at once and cells in a well cannot reasonably be assumed to be free of mutual influence ^[12]. Three separately treated wells are therefore three experimental units, and repeated measurements taken within each well are observational units, not extra replicates ^[12].

The consequence is quantitative. Because the standard error of a mean scales as one over the square root of the sample size, reporting n = 9 (three wells, each measured three times) where n = 3 is the number of independently treated wells shrinks the reported standard error by a factor of √(3/9) = 0.577 — that is, it appears 42.3% smaller than it should, on the assumption that the underlying standard deviation is unchanged. The error bars get tighter and nothing was learned.

A second limit sits above that one. The authors note that in vitro results often depend on conditions that vary each time the protocol is run, so such experiments are usually repeated on separate days, with the wells, aliquots or dishes within a day treated as subsamples that do not contribute to N ^[12]. Wells from a single run can be used as the experimental unit and will give a larger sample size, but the result then cannot speak to how far the effect generalizes beyond that run ^[12].

Power, and why a small significant result is weak evidence

Low statistical power reduces the chance of detecting a true effect. Less intuitively, it also reduces the probability that a statistically significant result reflects a true effect, and it inflates the effect sizes that do reach significance; an analysis of the neuroscience literature concluded that average power in that field is very low ^[13].

The mechanism is arithmetic. Take, purely for illustration, a field where one hypothesis in six is true, a significance threshold of 0.05, and power of 0.20. Among 600 tested hypotheses, 100 are true; 0.20 × 100 = 20 true effects reach significance, while 0.05 × 500 = 25 false ones do. The proportion of significant findings that are real is then 20 / (20 + 25) = 20 / 45 = 44.4%. Raise power to 0.80 and the true positives become 80, giving 80 / (80 + 25) = 80 / 105 = 76.2%. The prior fraction and the power values here are chosen to illustrate the calculation, not measured for any field.

The practical reading is that a single significant result from a small, exploratory peptide experiment should move your beliefs less than the p-value suggests, and that the reported effect size is more likely to be an overestimate than an underestimate.

What actually helps, and what remains unknown

• Identify the material, not the molecule. Supplier, lot number, certificate, salt form, and the measured peptide content — not just the sequence.
• Authenticate cells and record the passage. STR profiling with a documented match score, interpreted against published thresholds ^[5], plus mycoplasma testing ^[7].
• Use persistent identifiers. RRIDs uniquely identify cell lines and antibodies. For cell lines, RRID use is associated with lower reported use of problematic lines ^[6]; for antibodies, RRIDs are among the identifiers most often missing from methods sections ^[3].
• State the experimental unit explicitly. Say what was independently treated, and let n follow from that ^[12].
• Show the unprocessed data. In the replication project, 41 of the 158 original effects were reported only as representative images rather than as numerical values, and several of the assessment methods the project used required numerical effect sizes ^[1].

What is not known deserves equal space. There is no published estimate we are aware of for how often undisclosed excipients appear in commercially sourced research peptides; the mannitol finding concerns two commercially sourced custom peptides and was reported because it was unanticipated ^[8]. There is no established general threshold for how large a counter-ion or content discrepancy has to be before a given assay result changes, and the two studies cited here both found sequence-dependent, non-uniform effects ^{[10, 11]}. And the replication rates in ^[1] come from cancer biology; whether peptide assay work replicates better or worse has not, to our knowledge, been measured at comparable scale. Treating those gaps as gaps — rather than as reassurance — is the honest position.