A result outside the limit is not a verdict. It is data whose referent is not yet known: the number may be describing the material in the vial, or it may be describing the measurement that produced it, and the investigation exists to decide which. Put metrologically, the task is to separate out-of-specification (OOS) results that indicate an actual change in the chemical composition of the analysed object from results caused by measurement problems while the object still meets its specification, and the route to that separation runs through the validation data of the measurement process, its metrological traceability chains, the measurement uncertainty, and the producer's and consumer's risks that follow from it [1]. Those two directions correspond to the two families of decision risk that dominate a testing laboratory: false decisions caused by human errors committed while performing the test, and false decisions arising from interpretation of a test result against a specification limit in the presence of measurement uncertainty [2].
Two phases, and the boundary between them
The conventional structure splits the work in two. An initial investigation concentrates on uncovering apparent laboratory-related or sampling errors; where that proves inconclusive, an expanded investigation looks for less conspicuous causes, which may be process-related or operator-dependent as well as analytical [3]. The load-bearing element of that description is not the two-phase shape but the condition attached to it: OOS results are best handled through procedures written in advance for evaluating their validity, and those same procedures specify how much retesting or resampling is permitted [3].
Writing the extent down beforehand matters because the alternative is an investigation whose scope is set by the number it is investigating. If the permitted amount of retesting is decided after the failure, the stopping rule becomes until a passing result appears, and every measurement taken under that rule is conditioned by it. Fixing the limit in the procedure removes the decision from the moment at which it cannot be made impartially.
The laboratory phase is also bounded in what it can conclude. It can identify a specific analytical event and show that the event, rather than the material, produced the number: a standard weighed against a wrong assumed content, a transposed dilution factor, a leaking fitting, an integration applied to the wrong peak. It cannot conclude that a result was analytical from the fact that a repeat passed. A passing repeat is one draw from a distribution whose position relative to the limit is precisely the thing in question, so it carries no information about the origin of the first draw.
An identified cause, not a plausible one
Invalidation turns on whether a cause has been identified and demonstrated, not on whether one can be imagined. Two published investigations show what that standard of evidence costs in practice.
In release testing of residual ethylene oxide in polyethylene glycol using the compendial gas chromatographic methods, an OOS occurrence prompted a root-cause investigation that found and identified a PEG degradant, methyl formate, which co-elutes with ethylene oxide under those methods; a static headspace GC-FID method was then developed that separated the pair, achieving baseline separation, chromatographic resolution above 1.5, for the related impurities in the PEG grades tested [4]. Nothing in that account is an analyst's slip. The method was not selective enough to mean what it was being read to mean, and the cause only became identified once the co-eluting species had been named and resolved.
A mass-balance failure encountered during release testing of a small-molecule active substance was worked in the same direction. The investigation moved stepwise from the most probable causes outward to progressively more obscure ones until the missing material was identified as API-related oligomer impurities formed by a side reaction, species that had evaded conventional small-molecule release testing; the characterisation gap was ultimately traced back to a deficiency in the LC results of the purity methods that had been developed for the substance [5]. Both investigations end in a change to the method. That is the usual shape of a genuine root cause: a mechanism specific enough to demonstrate and then design against, and demonstrating it almost always alters the procedure that produced the failing number.
Assignable causes worth ruling out early in peptide work
Losses to container surfaces come first, because they produce a low assay with no chromatographic anomaly at all to draw attention. Peptide binding to surfaces is unpredictable enough that containers have to be selected experimentally rather than assumed. When 125I-labelled endocrine peptides spanning a wide range of net charge, size, end group and modification were incubated for 48 h in glass and plastic tubes, either untreated or coated with siliconising fluid, the binding capacities of the different surfaces differed substantially; siliconisation reduced recovery, adding 1% bovine serum albumin improved it, and lyophilising peptide-plus-BSA solutions in the tube best suited to each individual peptide gave better than 89% recovery for every peptide tested [6]. A suspected adsorption loss is testable: spike and recover through the complete preparation, in the labware actually used for the failing sample. Until that has been done it remains a hypothesis, not a cause.
The assumed content of the reference material is the second candidate, and it is the one most likely to have been wrong all along rather than wrong on the day. Quantification of synthetic peptides by acid hydrolysis followed by reversed-phase LC with stable-isotope-dilution MS, using hydrolysis conditions optimised at 4-6 h in 6 mol/L HCl at 150 °C, returned peptide contents between 62.07% and 88.18% across the peptides examined, with relative standard deviations under 8% and relative errors under 5% [7]. Chromatographic purity and peptide content are not the same quantity, and in that study the two diverged widely for the same materials [7]. A standard prepared by mass as though the weighed solid were entirely peptide carries the shortfall into every result computed against it. The resulting bias is systematic, reproducible, and completely invisible to a repeat injection.
The third is the state of the separation itself, and it is where the system-suitability guide connects to this one. A suitability limit can be derived rather than adopted: a resolution limit constructed from an intermediate, time-different precision study across immobilised polysaccharide-based columns in supercritical fluid chromatography was used to decide whether a column had deteriorated and whether a transfer to a different column had succeeded [8]. Deterioration is gradual. Monitoring nineteen size-exclusion columns with an in-house monoclonal antibody, tailing factor and peak width had risen by more than 10%, and resolution and plate count fallen by more than 10%, from their starting values after 1000 injections [9]. That is an antibody on SEC rather than a peptide on reversed phase, so the numbers do not transfer; the shape of the trend is the transferable part. A system can be well into a drift long before it trips a suitability threshold, which is why a passed suitability test at the head of a sequence supports far less than it is usually asked to support.
Retesting and resampling interrogate different terms
A retest re-analyses the original laboratory sample; a resample obtains new material from the batch. They are not interchangeable, because they hold different things fixed. A retest holds the sampling step constant, so it can expose an error in preparation, injection or calculation while remaining blind to any error committed when the sample was taken. A resample varies the sampling step and the analysis together, so a difference it produces cannot be attributed to either one without a design that separates them.
This is why the sampling step deserves the same scrutiny as the analysis. Valid analytical data can only come from representative samples, and the theory of sampling exists to define material heterogeneity and the sampling errors it generates, its primary task being to eliminate bias-generating errors and to minimise sampling variability [10]. A resample drawn by the same non-representative route as the original adds nothing to what is known about the batch, however many times it is repeated.
Choosing between the two requires knowing what the method's own variability is, which is the point at which the method-validation guide becomes part of the investigation. Acceptable precision ranges obtained from validation and from related activities - transfer studies, control charts, routine batch and stability data - give a target variability, and having a reliable one is what makes atypical or out-of-specification behaviour detectable and investigable in routine use; holding a procedure in its validated state over long-term application depends on continuing to monitor performance parameters such as system suitability tests and control charts [11]. Without that baseline there is no basis for saying whether a retest that differs from the original differs by more than the method ordinarily does.
Averaging conceals a failure rather than resolving it
The commonest statistical error in an OOS investigation is to report the mean of the failing result and its passing repeats. Whether a mean is even the reportable quantity is a question the method must already have answered. If a single measured value is out of specification then so is the whole result, subject to three exceptions: a high number of measurements, a value shown to be an outlier, or a reportable result defined as something other than the single value, such as the mean [12]. The third of those exceptions is not available retrospectively. Either the method defined the mean as the reportable value before the run, or it did not.
The converse trap is quieter. A result is not automatically within specification because every individual measurement is: where all the measurements lie close to the limit and the measurement error is large, an OOS result is still possible [12]. The two statistics that settle such cases are the confidence interval and the prediction interval, and they answer different questions - where the mean lies, and where a future single measurement will fall [12]. Averaging narrows the first. It does not narrow the second, which is why adding replicates cannot buy compliance: for future measurements to stay within specification, the distance between the limit and the mean must not fall below 1.65 times the standard deviation, and that requirement holds even when very high numbers of measurements are provided [12].
Outlier tests and the narrow claim they support
Outlier tests have a legitimate but small role. The classical procedures decide statistically whether the highest observation, the lowest, the highest and lowest together, the two highest, the two lowest, or a larger group are statistical outliers, and they are tests of significance and nothing else [13]. Dixon's Q test, the one usually reached for in a small analytical data set of about three to twelve observations, is applied by comparing an experimental Q against tabulated critical values, so the analyst learns only the range within which the type I error probability lies rather than its value [14]. Significance testing of this kind is perhaps still the most popular approach to suspect results and is used in several standards despite possible drawbacks; the alternatives are median-based and robust methods, which respectively ignore and down-weight measurements at the extremes of a data set [15].
The deeper limitation is that extreme values are troublesome to evaluate statistically because the underlying probability distribution cannot be determined. Some values are identifiable as outliers beyond doubt, by Dixon's test or a box plot; in a number of borderline cases a value is suspected but cannot be proven to be an outlier by statistics [12]. A passed outlier test establishes that a value is improbable under an assumed model. It does not name a cause, and by itself it does not support invalidating a result.
What the documentation is actually for
An investigation and a search for a passing number generate the same raw material - chromatograms, weights, dilutions, injection logs - and are distinguished only by the record. Each test run during an investigation should carry its purpose in writing before it is executed, together with the hypothesis it discriminates and what each outcome would mean. A test whose stated purpose is to see whether the result passes this time yields a number that cannot be interpreted afterwards, because the interpretation depended on a decision that was never made.
The hypothesis set can be assembled before it is needed. Human-error scenarios for a technique can be enumerated and scored in advance: thirty-six scenarios of human error that may occur in elemental analysis of geological samples by ICP-MS were examined by expert elicitation, each scored for likelihood and severity and for the effectiveness of the laboratory quality system in preventing it [16]. A laboratory that has run that exercise on its own peptide methods opens an investigation with a ranked list rather than a blank page. The complementary discipline sits on the output side: a rigorous policy of investigating failures, limiting retest analyses, and documenting and reporting all results, passing and failing alike [3].
What an investigation finally licenses is narrow. Where a cause is identified and demonstrated, the original result is attributable to that cause, the correction is documented, and the method or the procedure normally changes as a consequence. Where no cause is identified, the failing result has not been shown to be caused by a measurement problem, so the other alternative the metrological framework asks you to weigh - an actual change in the chemical composition of the analysed object - remains unexcluded [1]. The two guides this one sits beside carry the other half of the work: method validation is where the target variability that makes an atypical result recognisable is established [11], and system suitability is where the state of the instrument at the time of the run is recorded [8]. Neither can be reconstructed after a failure, which is why both belong in the procedure rather than in the investigation.