Interlaboratory Proficiency Tests for the Detection of Cronobacter and Salmonella in Milk Powder

doi:10.21203/rs.3.rs-7814511/v1

Download PDF

Research Article

Interlaboratory Proficiency Tests for the Detection of Cronobacter and Salmonella in Milk Powder

https://doi.org/10.21203/rs.3.rs-7814511/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

The contamination of milk powder by pathogens such as Cronobacter and Salmonella represents a major public health risk, particularly as the primary consumers of milk powder, infants, have heightened vulnerability to bacterial infection. In the face of this danger, analytical laboratories must implement and practice methods for the detection of these microorganisms with high accuracy. For this reason, BIPEA (Bureau Interprofessionel d’Études Analytiques) launched a new proficiency testing program for the detection of Cronobacter and Salmonella in samples of milk powder in 2019.

This paper presents the design and implementation of the program, as well as a detailed analysis of laboratory results and systems for the evaluation of qualitative proficiency testing performances. Results from these tests are encouraging, as the majority of laboratories are able to correctly identify both contaminated and uncontaminated samples. These tests are an essential quality control tool for laboratories to assess and demonstrate their competence to carry out these microbiological analyses, which are critical for public health, and to comply with the requirements of the ISO/IEC 17025 standard.

Proficiency testing

Laboratory performance

Microbiology

Milk powder

Salmonella

Cronobacter

With a market size of over $35 billion in 2024, milk powder plays an essential role in the food sector, particularly in the manufacturing of infant formulas, because of its combination of high nutritional value and stability over time. The low water content in milk powder and derived products makes contamination by pathogenic microorganisms more difficult, but risks continue to exist from certain resistant pathogens [1]. One such genus is Cronobacter, of the Enterobacteriaceae family, a group of Gram-negative and oxidase-negative bacteria that are rod-shaped and facultatively anaerobic [2]. While Cronobacter rarely cause disease in adults, infants are extremely vulnerable; one eight-year study reported a 42% mortality rate for infants infected with Cronobacter spp. meningitis. Additional studies have documented a clear link between the numerous outbreaks that have been reported globally since 1960 and consumption of powdered infant formula [3], while a 2014 assessment of American milk powder facilities determined that Cronobacter was present in the manufacturing areas of 69% of the 55 facilities tested [4].

The other pathogen most frequently responsible for outbreaks associated with milk powder consumption is Salmonella, also of the Enterobacteriaceae family, and similarly capable of surviving for extended periods of time in foods with low moisture levels. While contamination of milk powder by Salmonella is less prevalent than by Cronobacter, there are significant risks as Salmonella is considered one of the pathogens inducing the highest rates of serious illness and hospitalization. Beginning in late 2017, France experienced a major Salmonella Agona outbreak that led to over 30 cases under six months old from consumption of infant milk products, including 18 hospitalizations, and similar outbreaks have been reported across the world with substantial economic and health consequences [5].

It is therefore essential for laboratories to detect bacterial contamination of milk powder by these two pathogens. The ISO 22964 and ISO 6579-1 standards describe, respectively, reference methods for the detection of Cronobacter spp. and Salmonella spp. in food products, and a number of alternative methods have been validated according to ISO 16140-2 [6–8]. Quality control of all of these methods is critical to ensure consumer safety and confidence, and the ISO 17025 standard [9] indicates that laboratories must demonstrate the validity of their results through external controls, including interlaboratory proficiency tests (PTs). To this end, BIPEA developed and implemented new PTs in February 2019 dedicated, initially, to the detection of Cronobacter in milk powder; Salmonella detection in the same samples was added to the tests in June 2023 to address laboratory demand.

Participation in these proficiency tests allows laboratories to detect and correct analytical problems, to demonstrate their performance for these analyses, and to compare results obtained by different protocols for the detection of these pathogens under operating conditions using real matrices.

Proficiency tests involve the analysis by different laboratories of the same analytical parameters on identical samples. The implementation of a PT can be summarized in three principal steps: preparation of homogeneous and stable samples, analysis by participating laboratories, and statistical treatment of the data, which includes determination of assigned values and evaluation of laboratory performances.

The first trial of the PT for Cronobacter detection in milk powder, in 2019, gathered nine participants, and participation has steadily increased in the intervening years (Fig. 1) to reflect increased interest in the detection of these pathogens in food products, particularly milk powder infant formulas. Regular commission meetings are organized by BIPEA to allow participants to discuss past results and potential technical evolutions for the program.

Figure 1 Evolution of participation since implementation of the proficiency testing program, divided by geographic region

Design

For each trial of this program, each laboratory receives both positive (contaminated) and negative (uncontaminated) samples, according to a contamination scheme unknown to them. The objective of the trial is to correctly detect or not detect the target pathogens in each of these samples.

Sample production and shipment

For proficiency tests to be effective, it is crucial that homogeneous and stable samples be produced. For this PT, a batch of milk powder is first analyzed to detect the possible presence of pathogens, before being contaminated with calibrated suspensions of Cronobacter sakazakii and/or Salmonella Enteritidis, with target final concentrations around 10² CFU (colony-forming units) per sample for each microorganism. The batch is then homogenized and divided into a series of samples.

To demonstrate stability, three samples were stored at room temperature and analyzed for the two target microorganisms after zero, four, eleven and fourteen days; detection of both Cronobacter and Salmonella in each of these analyses confirmed the stability of the samples for the course of the test period. Moreover, for positive samples, the homogeneity of the batch is additionally verified by an experimental study on ten samples, taken randomly across the batch and analyzed in random order, according to the requirements of ISO 13528 [10]. Detection analyses of Cronobacter and Salmonella are conducted using the reference methods ISO 22964 and ISO 6579-1 respectively, and the set of samples is considered homogeneous if the microorganisms are detected in all analyzed samples.

Three samples are then shipped to each participating laboratory at room temperature.

Analysis by laboratories

Laboratories can perform the analyses using either the reference methods or alternative methods, and are also asked to indicate the date of analysis. They then submit their results to BIPEA via online reply forms.

Considering the unstable nature of the samples, participants are requested to analyze the samples as soon as possible after reception, with a limit of three weeks from the shipment date.

Statistical treatment

The results from these tests are qualitative (detected/not detected), and are therefore evaluated as follows:

If the target microorganism is detected when the sample was contaminated with the strain, the result is satisfactory.
If the target microorganism is not detected when the sample was not contaminated with the strain, the result is satisfactory.
If a false negative or a false positive is obtained, the result is considered unacceptable and should be interpreted as an action signal.

In addition, BIPEA has chosen to present an overall assessment of each laboratory’s ability to correctly identify negative and positive samples by calculating relative specificity (r_SP), relative sensitivity (r_SE), and relative accuracy (r_AC), defined as follows:

r_SP (%): Number of true negatives divided by the total number of expected negative samples. Relative specificity measures a laboratory’s ability to correctly identify samples as being free of the target microorganism.
r_SE (%): Number of true positives divided by the total number of expected positive samples. Relative sensitivity measures a laboratory’s ability to detect the target microorganism when it is present.
r_AC (%): Number of true results divided by the total number of samples. Relative accuracy measures a laboratory’s overall ability to correctly conclude on the presence or absence of the target microorganism.

For this PT, each laboratory’s overall performance is considered acceptable if their relative specificity and relative sensitivity are 100%, and therefore if their relative accuracy is 100% as well.

Since January 2020, BIPEA has organized two regular proficiency tests per year for these analyses. The results of the four most recent tests, since Salmonella was added to the samples, are summarized in Table 1 (Cronobacter detection) and Table 2 (Salmonella detection).

Table 1
Summary of Cronobacter detection results for four trials. Unacceptable results are in indicated in italics.
		Sample 1	Sample 2	Sample 3
Trial 1	Contamination scheme	Not spiked	Spiked	Spiked
Trial 1	Laboratory results	Detected: 0 Not detected: 21	Detected: 21 Not detected: 0	Detected: 21 Not detected: 0
Trial 2	Contamination scheme	Spiked	Not spiked	Not spiked
Trial 2	Laboratory results	Detected: 19 Not detected: 0	Detected: 0 Not detected: 18	Detected: 0 Not detected: 19
Trial 3	Contamination scheme	Not spiked	Not spiked	Spiked
Trial 3	Laboratory results	Detected: 1 Not detected: 19	Detected: 2 Not detected: 18	Detected: 17 Not detected: 3
Trial 4	Contamination scheme	Spiked	Not spiked	Not spiked
Trial 4	Laboratory results	Detected: 20 Not detected: 0	Detected: 1 Not detected: 17	Detected: 0 Not detected: 19

Table 2
Summary of Salmonella detection results for four trials. Unacceptable results are in indicated in italics.
		Sample 1	Sample 2	Sample 3
Trial 1	Contamination scheme	Not spiked	Spiked	Not spiked
Trial 1	Laboratory results	Detected: 0 Not detected: 6	Detected: 6 Not detected: 0	Detected: 0 Not detected: 6
Trial 2	Contamination scheme	Spiked	Spiked	Not spiked
Trial 2	Laboratory results	Detected: 15 Not detected: 0	Detected: 14 Not detected: 0	Detected: 1 Not detected: 14
Trial 3	Contamination scheme	Not spiked	Not spiked	Spiked
Trial 3	Laboratory results	Detected: 0 Not detected: 15	Detected: 1 Not detected: 14	Detected: 14 Not detected: 1
Trial 4	Contamination scheme	Spiked	Spiked	Not spiked
Trial 4	Laboratory results	Detected: 16 Not detected: 1	Detected: 18 Not detected: 0	Detected: 0 Not detected: 18

For both pathogens, results are generally very satisfactory, regardless of the geographical origin of the laboratories. For 16 of the 24 sets of samples studied here, all laboratories concluded correctly, and for the remaining samples only between 7 and 15% of laboratories reported false negatives or false positives. Approximately 75% of laboratories used the reference method ISO 22964 [6] for Cronobacter detection and approximately 50% of laboratories used the reference method ISO 6579-1 [7] for Salmonella detection; no noticeable effect of the method utilized can be observed on the results. In addition, performance in these tests has remained relatively stable over time, demonstrating that most laboratories master these detection analyses. It is also important to note that the rates of false positives are either similar, in the case of Salmonella, or greater, in the case of Cronobacter, than the rates of false negatives. This is reassuring, as the consequences of false positives are primarily economic, such as unnecessary product recalls, while false negatives can lead to outbreaks and have serious impacts on public health, including the death of contaminated persons.

In recent years, several propositions have been published for numerical scoring systems, designed to allow for easy interpretation of participant performances, for qualitative proficiency testing data. The objective of these systems, which include the L-score [11], the a-score [12], and the S-score [13], is to mimic the widely accepted z-score used for quantitative data and give participants a simpler way to evaluate their results.

Each of these systems is a useful contribution to the assessment of qualitative PT data, making it easier to compare between tests and examine laboratory performance over time. However, each also has certain limitations. The L-score requires at least 10 participants, five different parameters where failure has been recorded, and specific statistical modeling software; in addition, it is fundamentally a relative evaluation rather than an absolute one, as the most satisfactory scores are impossible for a laboratory to achieve unless other laboratories perform poorly. For this reason, identical results can be judged differently on different tests, making continued assessment over time complicated. The a-score remedies several of these difficulties but needs a minimum of 20 participants to be implemented. The S-score removes this barrier but uses a more complex system that requires PT providers to define a priori the difficulty of each analysis, which leads to numerical scores with less transparent interpretations when compared with the simplicity of the z-score. Replicates are also necessary in some cases.

BIPEA applies the specificity and sensitivity indicated in ISO 22117 and considers the use of relative specificity, sensitivity, and accuracy to be the system best adapted to evaluating laboratory performance for its qualitative proficiency tests [14]. The relative accuracy is an easy-to-interpret assessment of the overall ability of the laboratory to complete the analyses studied, while relative specificity and sensitivity allow the essential distinction to be made between difficulty detecting positive samples and incorrectly identifying negative samples, which are errors necessitating significantly different corrective actions—just as large positive and large negative z-scores clearly indicate different kinds of analytical problems. In addition, if a laboratory participates in multiple rounds of such a PT, it is simple enough to monitor evolution in performance by graphing relative accuracy against time, as described by Chabirand et al. [15].

By examining in detail the results of the four trials previously presented for the detection of Cronobacter (Table 3) and Salmonella (Table 4) in milk powder, each laboratory’s global performance can be evaluated using these assessment parameters. For each pathogen, all but three laboratories achieved relative accuracy scores of 100%, and therefore 100% relative specificity and sensitivity as well. The overall performance on these tests can thus be considered highly satisfactory. Participants in this program are provided with their relative specificity, sensitivity, and accuracy for each trial for which they submit results, and can easily calculate these three scores over a period of multiple trials if desired, as demonstrated here.

Table 3
Detailed results of four trials for the detection of Cronobacter in milk powder, including evaluation and rate of participation (r_P) for each laboratory. The contamination scheme is displayed in the table header, 0 and 1 correspond to “Non detected” and “Detected” respectively, and unacceptable results are in indicated in italics.
	Trial 1			Trial 2			Trial 3			Trial 4			Evaluation
Lab	0	1	1	1	0	0	0	0	1	1	0	0	r_SP (%)	r_SE (%)	r_AC (%)	r_P (%)
1	0	1	1	1	0	0	0	0	1	1	0	0	100	100	100	100
2	0	1	1	1	0	0	0	1	0	1	0	0	86	80	83	100
3	0	1	1	1	0	0	0	0	1	1	0	0	100	100	100	100
4	0	1	1	1	0	0	0	0	1	1	0	0	100	100	100	100
5	0	1	1	1	0	0	0	0	1	1	0	0	100	100	100	100
6	0	1	1	1	0	0	1	1	0	1	0	0	86	80	83	100
7	0	1	1	1	0	0	0	0	1	1	0	0	100	100	100	100
8	0	1	1	1	0	0	0	0	1	1	0	0	100	100	100	100
9	0	1	1	1	0	0	0	0	1	1	0	0	100	100	100	100
10	0	1	1	1	0	0	0	0	1	1		0	100	100	100	92
11	0	1	1	1	0	0	0	0	1	1			100	100	100	83
12	0	1	1	1	0	0	0	0	1				100	100	100	75
13				1	0	0	0	0	1	1	0	0	100	100	100	75
14				1	0	0	0	0	1	1	0	0	100	100	100	75
15				1	0	0	0	0	1	1	0	0	100	100	100	75
16				1	0	0	0	0	1	1	0	0	100	100	100	75
17				1	0	0	0	1	0	1	1	0	67	67	67	75
18	0	1	1	1		0	0	0	1				100	100	100	67
19	0	1	1							1	0	0	100	100	100	50
20	0	1	1							1	0	0	100	100	100	50
21				1	0	0	0	0	1				100	100	100	50
22	0	1	1										100	100	100	25
23	0	1	1										100	100	100	25
24	0	1	1										100	100	100	25
25	0	1	1										100	100	100	25
26	0	1	1										100	100	100	25
27	0	1	1										100	100	100	25
28							0	0	1				100	100	100	25
29										1	0	0	100	100	100	25
30										1	0	0	100	100	100	25

Table 4
Detailed results of four trials for the detection of Salmonella in milk powder, including evaluation and rate of participation (r_P) for each laboratory. The contamination scheme is displayed in the table header, 0 and 1 correspond to “Non detected” and “Detected” respectively, and unacceptable results are in indicated in italics.
	Trial 1			Trial 2			Trial 3			Trial 4			Evaluation
Lab	0	1	0	1	1	0	0	0	1	1	1	0	r_SP (%)	r_SE (%)	r_AC (%)	r_P (%)
1	0	1	0	1	1	0	0	0	1	1	1	0	100	100	100	100
2	0	1	0	1	1	0	0	0	1		1	0	100	100	100	92
3				1	1	0	0	1	0	1	1	0	75	80	78	75
4				1	1	1	0	0	1	1	1	0	75	100	89	75
5	0	1	0	1	1	0				1	1	0	100	100	100	75
6				1	1	0	0	0	1	1	1	0	100	100	100	75
7				1	1	0	0	0	1	1	1	0	100	100	100	75
8				1	1	0	0	0	1	1	1	0	100	100	100	75
9				1	1	0	0	0	1	1	1	0	100	100	100	75
10				1	1	0	0	0	1	1	1	0	100	100	100	75
11				1	1	0	0	0	1	1	1	0	100	100	100	75
12				1	1	0	0	0	1	1	1	0	100	100	100	75
13	0	1	0	1		0	0	0	1				100	100	100	67
14				1	1	0				1	1	0	100	100	100	50
15	0	1	0							1	1	0	100	100	100	50
16				1	1	0	0	0	1				100	100	100	50
17										0	1	0	100	50	67	25
18	0	1	0										100	100	100	25
19							0	0	1				100	100	100	25
20							0	0	1				100	100	100	25
21										1	1	0	100	100	100	25
22										1	1	0	100	100	100	25
23										1	1	0	100	100	100	25

If one of the primary goals of proficiency testing is to enable laboratories to demonstrate their competence for given analyses, there is a final factor to be considered. While it is clear that a laboratory with 100% relative accuracy has demonstrated greater ability than one with 33% relative accuracy, and that a laboratory that consistently achieves scores of 100% masters the analyses to a greater extent than one that oscillates between scores of 100% and 50%, frequency of participation must also be taken into account. For example, by studying multiple trials collectively as in Tables 3 and 4, it is possible to observe for each participant not only the rates of relative specificity, sensitivity, and accuracy, but also a final rate, the rate of participation (r_P), which corresponds to the number of samples analyzed divided by the total number of samples proposed over a given time period. For an analytical laboratory, achieving a relative accuracy of 100% while participating in all four trials can be a way to signal greater expertise than obtaining the same rate while participating in a single test, and such performance can be extremely valuable for earning and maintaining consumer trust.

Outbreaks involving Cronobacter and Salmonella in milk powder or infant formulas have been reported in many countries and can have enormous consequences, including serious illness, hospitalization, and death, and the global nature of food supply chains means that contamination in one facility can lead to cases across the world. The health risks associated with non-detection of contaminated samples of milk powder are particularly high because the principal consumers are infants, who have underdeveloped immune systems. Furthermore, the rates of reported Cronobacter infections in infants have risen significantly in recent years.

The interlaboratory proficiency tests presented here have been developed to provide laboratories with a quality tool to assess their ability to detect these pathogens in milk powder. The PTs are offered regularly, with two rounds per year, and the performances of participating laboratories are highly satisfactory, with relative accuracy equal to 100% for both pathogens for most laboratories. By participating in proficiency testing, laboratories can verify the reliability and stability of their results, as well as obtain recognition of their analytical procedures by customers and accreditation bodies according to ISO 17025. These initial results are encouraging and reassuring for consumers and public organizations, as an indicator that laboratories master these essential detection analyses.

Acknowledgments

BIPEA acknowledges all laboratories participating in these proficiency tests.

Funding

No funding was received for conducting this study or to assist with the preparation of this manuscript.

Competing interests

All authors certify that they have no competing interests to declare that are relevant to the content of the article.

Data availability

Data will be made available upon request.

Paswan R, Park YW (2020) Survivability of Salmonella and Escherichia coli O157: H7 pathogens and food safety concerns on commercial powder milk products. Dairy 1.3:189-201. https://doi.org/10.3390/dairy1030014
Kalyantanda G, Shumyak L, Archibald LK (2015) Cronobacter species contamination of powdered infant formula and the implications for neonatal health. Front Pediatrics 3:56. https://doi.org/10.3389/fped.2015.00056
Song X, Teng H, Chen L, Kim M (2018) Cronobacter species in powdered infant formula and their detection methods. Korean J Food Sci An Resources 38.2: 376. https://doi.org/10.5851/kosfa.2018.38.2.376
Hayman MM, Edelson-Ammem SG, Carter PJ, Chen Y, Metz M, Sheehan JF, Tall BD, Thompson CJ, Smoot LA (2020) Prevalence of Cronobacter spp. and Salmonella in milk powder manufacturing facilities in the United States. J Food Prot 83.10:1685-1692. https://doi.org/https://doi.org/10.4315/JFP-20-047
Lang E, Sant’Ana AS (2021) Microbial contaminants in powdered infant formula: what is the impact of spray-drying on microbial inactivation? Curr Opin Food Sci 42:195-202. https://doi.org/10.1016/j.cofs.2021.06.007
ISO 22964:2017 - Microbiology of the food chain — Horizontal method for the detection of Cronobacter spp.
ISO 6579-1:2017 - Microbiology of the food chain — Horizontal method for the detection, enumeration and serotyping of Salmonella — Part 1: Detection of Salmonella spp.
ISO 16140-2:2016 - Microbiology of the food chain — Method validation — Part 2: Protocol for the validation of alternative (proprietary) methods against a reference method
ISO/IEC 17025:2017 - General requirements for the competence of testing and calibration laboratories
ISO 13528:2022 - Statistical methods for use in proficiency testing by interlaboratory comparison.
Uhlig S, Bläul C, Frost K, Sgorzaly S, Colson B, Simon K (2015) Qualitative PT data analysis with easy-to-interpret scores. Accred Qual Assur 20.5:347-353. https://doi.org/10.1007/s00769-015-1174-8
Beavis G, Wilson J, Sykes M (2019) Quantitative scores for binary qualitative proficiency testing. Accred Qual Assur 24.4:263-269. https://doi.org/10.1007/s00769-019-01386-8
Baudry C, Jadé G, Rayneau P, Lombard B, Feinberg M, Laurentie M (2024) S-score: a new score for binary qualitative proficiency testing schemes interpretable as the z-score. Accred Qual Assur 29.2:103-113. https://doi.org/10.1007/s00769-023-01561-y
ISO 22117:2019 - Microbiology of the food chain — Specific requirements and guidance for proficiency testing by interlaboratory comparison
Chabirand A, Ahtoine G, Pierson O, Hostachy B (2014) The organization of proficiency testing in plant pathology (qualitative methods of analysis) according to the ISO/IEC 17043: example of the French national reference laboratory. Accred Qual Assur 19.2:111-125. https://doi.org/10.1007/s00769-014-1034-y

No competing interests reported.

Download PDF

Editorial decision: Revision requested
24 Oct, 2025
Reviews received at journal
20 Oct, 2025
Reviews received at journal
14 Oct, 2025
Reviewers agreed at journal
13 Oct, 2025
Reviewers agreed at journal
12 Oct, 2025
Reviews received at journal
12 Oct, 2025
Reviewers agreed at journal
10 Oct, 2025
Reviewers agreed at journal
10 Oct, 2025
Reviewers invited by journal
10 Oct, 2025
Editor assigned by journal
10 Oct, 2025
Submission checks completed at journal
10 Oct, 2025
First submitted to journal
09 Oct, 2025

You are reading this latest preprint version

Interlaboratory Proficiency Tests for the Detection of Cronobacter and Salmonella in Milk Powder

Status:

Version 1

Abstract

Figures

Introduction

Methods

Design

Sample production and shipment

Analysis by laboratories

Statistical treatment

Results and Discussion

Conclusion

Statements and Declarations

References

Additional Declarations

Status:

Version 1