Clinical acceptance of a digital health clinical decision support algorithm for children in Tanzania and Rwanda: A mixed-method and before-after analysis from the DYNAMIC study

doi:10.21203/rs.3.rs-7753943/v1

Background:

Digital clinical decision support algorithms (CDSAs) can improve healthcare provider adherence to guidelines by streamlining clinical assessments and suggesting appropriate diagnoses and treatments. However, low uptake, acceptance, and adherence to CDSAs hinder their potential for improving quality of care. We conducted a mixed-methods study to evaluate healthcare provider acceptance of proposed diagnoses by ePOCT+, a digital CDSA used in Tanzania and Rwanda for the management of sick children age 1 day to 14 years in primary care health facilities, complemented by 13 semi-structured interviews with healthcare providers. A before–after analysis assessed changes in diagnosis acceptance following adaptations informed by the study findings.

Results:

Between December 2021 and October 2022, 27,593 new consultations using ePOCT + were completed at 36 Tanzanian and Rwandan health facilities. In Tanzania, 94.1% of diagnoses for children aged 2 months to 14 years of age were accepted, compared to 67.2% in Rwanda. In the ePOCT + algorithms for children < 2 months old, 61.5% of diagnoses were accepted in Tanzania, and 45.3% in Rwanda. Qualitative interviews revealed three major reasons for rejecting proposed diagnoses: 1) mismatch between clinical judgment and the proposed diagnosis based on clinical and anthropometric data (e.g. malnutrition diagnoses), 2) misunderstanding of diagnosis terms, criteria, or management recommendations, and 3) hesitancy to refer patients to the hospital (i.e., severe diagnoses). The algorithms were adapted based on these findings and expert input. A before–after analysis showed improved acceptance for some diagnoses following adaptations.

Conclusions:

Allowing healthcare providers to accept or reject diagnoses proposed by digital CDSAs, combined with qualitative feedback to explore reasons for rejection, provides useful insights into the acceptance of clinical content and should be considered by other CDSAs to inform approaches to address and improve acceptability. Differences in acceptance of diagnoses between Tanzania and Rwanda underscore contextual differences related to clinician autonomy, training, implementation, and acceptability. These findings can inform refinements and corrections to clinical algorithms, and help tailor strategies, such as targeted training, to enhance CDSA adoption.

Acceptance

digital health

child health

global health

quality of care

health services

Digital clinical decision support systems (CDSSs) provide clinical guidance at the point of care, ranging from alerts within an electronic medical record system, to comprehensive support throughout every step of the consultation process [1]. The latter, often described as clinical decision support algorithms (CDSAs), are developed to help health providers integrate and adhere to clinical guidelines [2, 3]. Knowledge-based CDSAs prompt health providers to collect the necessary clinical information and propose the appropriate diagnoses and treatments based on guideline-derived algorithms, thereby supporting more rational and evidence-based prescribing practices. While the implementation of some CDSAs in primary care has been associated with increased adherence to clinical guidelines [4–6], improved antimicrobial stewardship [7, 8], and better clinical outcomes [8, 9], others have shown little to no impact [10, 11].

Several factors can help explain these inconsistent findings. First, difficulties associated with the use and understanding of new or unfamiliar technologies can result in low uptake and adherence to CDSAs [10, 12–14]. Second, some CDSAs have limited demographic and clinical scope: they only address specific patients subgroups, limiting their usefulness in routine practice [12, 15, 16]. Finally, the implementation of CDSAs has the potential to interfere with existing workflows, demanding additional focus and time, which may inadvertently divert attention from direct patient care and reduce the uptake of CDSAs [17, 18].

Improving CDSAs based on user feedback and data collected through the tool aligns with the Principles for Digital Development [19], and the WHO SMART guidelines [20]. This approach ensures the adaptability and relevance of the CDSA to a specific setting, thereby improving the usability, acceptance, and effectiveness of digital health tools [21].

Within the DYNAMIC project, we developed, implemented, and evaluated ePOCT+, a CDSA for the care of sick children under 15 years of age in primary care level health facilities in Tanzania and Rwanda [22, 23]. ePOCT + integrates a feedback mechanism that allows healthcare providers to accept or reject proposed diagnoses and treatments. This mixed-method study aimed to describe the acceptance of ePOCT + clinical content and the impact on diagnosis acceptance rates following data-driven changes made to the algorithms.

Setting

The study was part of the DYNAMIC project [24], an initiative aimed at improving pediatric healthcare delivery at the primary care level in Rwanda and Tanzania by equipping healthcare providers with ePOCT+, a CDSA for managing acute conditions in children under 15 years. ePOCT + proposes diagnoses and treatment strategies based on the symptoms, clinical signs, and test results entered by the healthcare provider in the tool (Fig. 1). The clinical algorithm underpinning ePOCT + is based on the World Health Organization’s (WHO) Integrated Management of Childhood Illness (IMCI) chartbook [25], previous generations of CDSAs developed and implemented by our team [8, 26], and input from experts [22]. Notably, the scope of ePOCT + extends beyond the standard IMCI content to include a broader range of age groups and syndromes, introducing clinical content that may be less familiar to healthcare providers. Building on a common core, the algorithm was adapted to Rwanda and Tanzania specific contexts through integration of national guidelines and feedback from local experts. A detailed description of ePOCT + and its digital platform, medAL-suite, is provided elsewhere [22, 23, 27].

ePOCT + was implemented in Rwandan and Tanzanian primary care facilities along with essential IT infrastructure, mentorship, and point-of-care tests for measuring C-reactive protein (CRP), hemoglobin levels, and pulse oximetry [22, 23]. A key objective was to improve antimicrobial stewardship by reducing unnecessary antibiotic prescriptions [7]. Effectiveness was assessed via a cluster-randomized controlled trial in Tanzania and a non-randomized controlled trial in Rwanda; results are reported elsewhere [4, 28]. The tool was then deployed under routine care conditions (Figure S1).

In Tanzania, healthcare services, including medications for acute illnesses, are provided free of charge to children under the age of 5 years in government or government-designated primary health facilities. In Rwanda, while healthcare is subsidized and community health insurance is available at a low cost, there is still a modest fee for healthcare services, with full coverage provided only for individuals in the most vulnerable category, for whom the government covers all healthcare costs [29]. The included primary care facilities were dispensaries/health posts and health centers, with the latter representing a higher level of care than the former.

Study Design

Throughout implementation, we conducted a mixed-method study to gain a comprehensive understanding of the acceptability, adherence, and challenges associated with ePOCT + clinical content, thereby leading to iterative improvements.

First, we performed a quantitative descriptive analysis of diagnosis acceptance in Tanzania and Rwanda. These results informed a qualitative study among Tanzanian healthcare workers, aimed at exploring the reasons for diagnosis rejection and identifying areas of improvement. In Rwanda, we conducted a more detailed quantitative analysis to examine clinical characteristics associated with diagnoses rejection and the alternative diagnoses selected. Based on these findings, targeted modifications were implemented in ePOCT + for both countries. Finally, a before-after analysis was carried out to assess trends in diagnosis acceptance following these modifications.

Quantitative descriptive analysis of diagnoses acceptance and rejection

This analysis focused on the intervention arm of the cluster randomized controlled trial in Tanzania and the cluster non-randomized controlled trial in Rwanda, both evaluating the impact of ePOCT+. Data were collected from outpatient consultations conducted between December 1, 2021, and October 31, 2022 (Supplementary Fig. 1). In Tanzania, the intervention arm included 20 government primary care facilities (dispensaries or health centers), 12 in the Morogoro region and 8 in the Mbeya region, spanning semiurban and rural districts [7]. In Rwanda, 16 government health centers were included across two semiurban and rural districts of the Western Province, with 6 facilities in Rusizi district and 10 in Nyamasheke district.

The quantitative analysis was limited to consultations in which ePOCT + was used throughout the entire clinical encounter. This ensured that all diagnoses and treatments proposed by the tool were either accepted, rejected or manually added by the healthcare worker. Consultations affected by identified IT issues that could have compromised data validity or completeness were excluded from the analysis. Only initial consultations were included, excluding all reattendance consultations or referrals.

Descriptive statistics were used to present the most commonly proposed, rejected and manually added diagnoses. Given the substantial differences in clinical algorithms between infants under 2 months and children over 2 months, as well as variations across countries, separate analyses were conducted by age group and country.

In Rwanda, an additional quantitative analysis was performed, to characterize the clinical profiles (symptoms, signs, and test results) associated with frequently rejected or manually added diagnoses, as well as alternative diagnoses selected.

Findings were reported in alignment with the STROBE statement checklist [30].

Qualitative interviews

Semi-structured interviews were conducted with healthcare providers in Tanzania from health facilities participating in the ePOCT + trial. The interview guide was developed based on a preliminary analysis of routinely collected data, focusing on the most frequently rejected diagnoses and treatments between December 1, 2021, and August 17, 2022. Input from the study implementation team informed its design. The guide was further refined and adapted following insights from the initial interviews. The final version of the questions and prompts is provided in Supplementary material Note 1.

Healthcare providers in Tanzania were purposively selected to include a minimum of 4 providers from health facilities that had low uptake of ePOCT+ (below average) and 4 from high uptake (above average) health facilities. High uptake of ePOCT + was defined as the proportion of eligible consultations conducted using the tool higher than the mean uptake at the time (77%). A minimum of eight interviews were planned, with additional interviews carried out until data saturation was reached, which occurred after 13 interviews.

During the interviews, we collected information on the general characteristics of participating healthcare providers, their overall satisfaction and frustrations with the digital health tool, perceived missing functionalities, and the tool’s impact on consultation duration. Additionally, the interviews included targeted questions exploring providers’ perspectives and attitudes toward specific items that had been pre-identified by the study implementation team and investigators (based on informal feedback) as potentially unclear or frequently linked to rejected diagnoses.

Interviews took place between August 24 and October 11, 2022, at the workplace of the healthcare providers. Each interview lasted approximately 45 minutes and included 1 to 3 researchers, led by a Tanzanian researcher (GK or MJ) and assisted by Swiss researchers during the first 3 interviews (AP, RT). The four researchers were RT, a Swiss male medical doctor with prior experience conducting research in Tanzania; AP, a Swiss male medical student with no previous field or research experience; MJ, a Tanzanian female clinical officer with strong familiarity with the local context; and GK, a Tanzanian male medical doctor with research and field experience. Both MJ and GK were well-acquainted with most interviewees, having led training sessions with them at the beginning of the broader project. All the interviewees were informed about the purpose of the research prior to their participation. The interviews were conducted in Swahili and sometimes in English, with Swahili responses immediately translated to English by the interviewer. The interviews were conducted with PowerPoint slides to show specific clinical algorithms, clinical images, and screenshots of ePOCT + that were the subject of questioning.

The interviews were recorded via the smartphone application OtterPilot, which provides automated transcriptions in English, and thereafter reviewed and refined via the Trint application. Intelligent verbatim sequences were used for the final transcription performed by AP, who takes responsibility for ensuring the accuracy and quality of the transcripts. The transcripts were not returned to participants for comments or corrections.

Our qualitative analysis employed a hybrid approach, combining narrative description with the framework method, whereby data were organized into thematic matrices to facilitate comparison across interviews [31]. This hybrid approach was chosen to accommodate the exploratory nature of the study, which required both structures to compare key topics across interviews, and flexibility to identify unanticipated themes. While guided by principles of grounded theory, the analysis remained primarily inductive, allowing themes to emerge organically from the data. No formal coding method was utilized during this process. The study design and findings are presented in accordance with the Consolidated Criteria for Reporting Qualitative Research (COREQ) checklist [32].

ePOCT + content adaptation and pre-post comparison of acceptance

Based on these quantitative and qualitative findings, and with input from the study team, we specified targeted modifications to the ePOCT + clinical content. In Tanzania, a clinical expert committee reviewed and endorsed these changes prior to implementation. Post-update quantitative data were collected from the tool in the same facilities (Tanzania: October 6, 2023–September 30, 2024; Rwanda: October 09, 2023–November 30, 2024; Figure S1). We identified a priori the modifications expected to directly impact diagnoses acceptance and compared pre- versus post-update acceptance rates for the affected diagnoses using Fisher’s exact test.

Data analysis was performed using STATA version 17.0 and R software (version 4.2.1).

Baseline characteristics

A total of 23,593 consultations were recorded in the parent project trial in Tanzania between December 1, 2021, and October 31, 2022, and 21,052 in Rwanda between December 1, 2021, and October 31, 2022. Overall, the uptake of ePOCT + was 76.2% (17,985/23,593) in Tanzania and 76.8% (16,173/21,052) in Rwanda among all consultations. A total of 15,744 consultations from Tanzania and 11,849 consultations from Rwanda met inclusion criteria for this analysis (Fig. 2). The majority of consultations were among children aged 2 months to 14 years, with 95.8% in Tanzania and 94.6% in Rwanda (Table 1). Regarding the healthcare facility level, most Tanzanian consultations were from dispensaries, while Rwandan consultations were exclusively from health centers. The distributions of patients by sex, health facility type, and region are detailed in Table 1, and presenting complaints by age group are provided in Supplementary Tables 1–2. Respiratory complaints, fever, and gastrointestinal complaints were the most common in both countries and age groups (children under and over 2 months of age).

Table 1

Baseline characteristics of patients (before ePOCT + modifications)
	Tanzania	Rwanda
Initial consultations, n	15,744	11,849
Sex: - Female, n (%) - Male, n (%) - Unknown, n (%)	8,064 (51.2%) 7,646 (48.6%) 34 (0.2%)	6,092 (51.4%) 5,757 (48.6%) 0 (0.0%)
Age: - < 2months, n (%) - 2–5 months, n (%) - 6 months-4 years, n (%) - 5–11 years, n (%) - 12–14 years, n (%)	666 (4.2%) 1,835 (11.6%) 11,238 (71.4%) 1,717 (10.9%) 288 (1.8%)	636 (5.4%) 804 (6.8%) 6,650 (56.1%) 3,161 (26.7%) 598 (5.0%)
Health facility type: - Dispensary, n (%) - Health Center, n (%)	12,662 (80.4%) 3,082 (19.6%)	0 (0.0%) 11,849 (100.0%)
Region (Tanzania) or district (Rwanda): - Nyamasheke, n (%) - Rusizi, n (%) - Morogoro, n (%) - Mbeya, n (%)	- - 11,361 (72,2%) 4,383 (27,8%)	6,217 (52.5%) 5,632 (47.5%) - -

Overall acceptance of diagnoses

Among initial consultations, an average of 3 diagnoses per consultation were proposed, equivalent in Tanzania and Rwanda (Table 2). Among children aged 2 months or older, 94.1% of diagnoses were accepted in Tanzania, compared to 67.2% in Rwanda. Among infants < 2 months old, 61.5% of diagnoses were accepted in Tanzania, compared to 45.3% in Rwanda. The proportion of diagnoses accepted by providers was higher among patients aged 2 months and older, and higher in Tanzania than in Rwanda (Table 2). Among patients under 2 months of age, at least one diagnosis was rejected in 60.8% of consultations in Tanzania and 86.0% in Rwanda. In comparison, for children aged 2 months and older, at least one diagnosis was rejected in 11.7% of consultations in Tanzania and 62.2% in Rwanda. A significant portion of this disparity in Rwanda was attributed to the higher rejection of «Prevention and screening» diagnosis (Supplementary Table 5). Manually added diagnoses were rare overall, but more frequent in Rwanda than in Tanzania. The diagnoses most frequently rejected (both in absolute and relative terms as well as the most frequently added, are listed in Supplementary Tables 3–6.

Table 2

Overall acceptance of diagnoses (before ePOCT + modifications) for ePOCT + in children < 2 months and > = 2 months old
	Tanzania (< 2 months)	Rwanda (< 2 months)	Tanzania (≥ 2 months)	Rwanda (≥ 2 months)
Initial consultations, n	666	636	15,078	11,213
Total diagnoses, n	2,153	1,853	38,501	35,423
Manually added diagnoses, n	8	27	245	642
Diagnoses proposed by the algorithm, n	2,145	1,826	38,256	34,781
Average number of diagnoses proposed by the algorithm/consultation	3.2	2.9	2.5	3.1
Acceptance proportion, % (n/N)	61.5% (1,320/2,145)	45.3% (827/1826)	94.1% (36,001/38,256)	67.2% (23,369/34,781)
Rejection proportion, % (n/N)	38.5% (825/2,145)	54.7% (999/1826)	5.9% (2,255/38,256)	32.8% (11,412/34,781).
Consultations with at least one diagnosis rejected	60.8% (405/666)	86.0% (547/636)	11.7% (1,770/15,078)	62.2% (6,977/11,213)

Qualitative interviews and quantitative analysis of acceptance and rejection of diagnoses

Between August 24 and October 11, 2022, 13 semi-structured interviews with health care providers took place at 11 health facilities. The characteristics of the participants included in the semi-structured interviews are described in Supplementary Table 7. The majority of participants were clinical officers (10/13), aged 30–39 years (8/13), with more male participants interviewed than female (8 vs. 5). All invited healthcare providers agreed to participate in the study.

Mismatch between clinical judgment and proposed diagnosis

One theme that recurred with 9 different healthcare providers related to the rejection of diagnoses related to malnutrition, findings that are also visible in rejection rates of diagnoses (Table 3). Healthcare providers note that they may rely more on the clinical appearance of the patient rather than the calculated anthropometric values (weight-for-age z-score, mid-upper arm circumference, or weight-for-height z-score) or caregiver-provided responses.

“I would mostly rely on the clinical appearance of the child. If he looks malnourished or not. So, then the other measurements will come next to agree to a given diagnosis… In the end I would disagree because clinically, the child might not look like having low weight” (SSI 7, male, clinical officer with 10 years of experience, CDSA uptake below average)

Table 3

Proportion of malnutrition and feeding diagnoses rejected
Diagnoses	Number rejected/Total proposed (%) in Tanzania	Number rejected/Total proposed (%) in Rwanda
Feeding problem (insufficient feed) (YI)	141/287 (49.1%)	104/133 (78.2%)
Feeding problem (lack of exclusive breastfeeding) (YI)	17/34 (50%)	13/18 (72.2%)
Feeding problem (lactation) (YI)	16/36 (44.4%)	18/26 (69.2%)
Low weight for age (YI)	27/62 (43.5%)	34/47 (72.3%)
Very low weight for age (YI)	2/18 (11.1%)	16/23 (69.6%)
Moderate malnutrition	287/1060 (27.1%)	767/1171 (65.5%)
Uncomplicated severe acute malnutrition	48/137 (35%)	165/248 (66.5%)
Complicated severe acute malnutrition	14/27 (51.9%)	133/155 (85.8%)

*Unless otherwise noted as YI (young infant), all diagnoses refer to consultations with children aged 2 months to 14 years.

The emphasis on clinical judgment extended to the context of feeding problem diagnoses, with 6 interviewees expressing similar reliance on visual clinical cues over quantitative measurements or caregiver-provided responses. This preference was evident among healthcare workers in both patient age groups; further highlighting the significance of clinical judgment.

“Sometimes she [the mother] may tell you that the child is not feeding well. Then when you try to observe, the child is in fact feeding well.” (SSI 7, male, clinical officer with 10 years of experience, CDSA uptake below average)

Moreover, healthcare providers described a mismatch between clinical presentation and the proposed diagnosis as a rationale for the rejection of some severe diagnoses with three specifically describing the rejection of “critical illness” and “severe clinical infection” diagnoses among young infants.

“Upon assessment the algorithm brought up the diagnosis of severe clinical infection. Now, just looking at the child, he doesn't look that critically ill…” (SSI 8, male, clinical officer with 0–4 years of experience, CDSA uptake above average)

Incorrect data entry leading to inappropriate diagnoses

While some responses implied disagreement with diagnoses based on clinical judgment, 5 healthcare providers suggested that incorrect data entries could be a potential cause for the rejection of malnutrition diagnoses. Participants reported that, in some cases, measurements were not taken during the consultation but were instead based on previously recorded values, caregiver reports, or estimates. This practice may have contributed to the over-classification of weight-related issues when weight-for-age or height-for-age calculations were inaccurate.

“Sometimes a clinician may enter values for MUAC and weight measurement that are not valid. They were not measured that day. Like he just enters them so that he can move to the next page…, then the clinician, by looking at the child, might decide to disagree because he knows the measurements were not correct.” (SSI 8, male, clinical officer with 0–4 years of experience, CDSA uptake above average)

Incorrect data entries were described beyond diagnoses related to malnutrition and provided different reasons for incorrect inputs. They range from submitting arbitrary responses to questions with unclear wording, to items that were skipped or misunderstood. It was often alluded that such arbitrary or false responses were sometimes inputted to speed up the consultation process.

Someone may not understand how to ask the question appropriately, or you may not even ask the question at all and just fill in, in order to move to the next page… I think there are some clinicians who do clinical assessment by looking at the palms or the conjunctiva to see if there is anemia. If they haven't measured [hemoglobin], they would put just a given range of measurement, a given value so that they can go past that stage to go to the diagnosis part...

“Rejection of mild croup could happen because of [mis]understanding. One may not be able to clearly understand and ask a question about stridor and respond to that question.” (SSI 7, male, clinical officer with 10–14 years of experience, CDSA uptake below average)

Disagreement with diagnostic label, criteria, or management recommendations

Disagreement with diagnostic label, diagnostic criteria or management pathways contributed to some of the observed disagreement. One respondent reported rejecting the diagnosis “suspicion of tuberculosis” believing that only a positive response to the question about close contact with a known tuberculosis case could lead to this diagnosis. However, according to the IMCI guidelines, other clinical conditions, such as cough lasting more than 14 days or signs of malnutrition (moderate or severe, uncomplicated) are also considered. So while the management is in line with IMCI, the diagnosis label was new, which may also explain the high proportion of diagnoses rejected (69.1% in Tanzania, and 89.8% in Rwanda among children 2 months to 15 years).

Semi-structured interviews further revealed that some providers dismissed diagnoses based not on disagreement with the clinical conclusion but rather on assumptions about the management that would follow. For example, the diagnosis "abscess” for young infants was sometimes rejected because providers believed the algorithm would only recommend treatment, whereas they preferred to refer the patient for further care (diagnosis rejected in 37.5% in Tanzanian infants under 2 months, and 66.7% in Rwanda).

“They reject the diagnosis because they think it will give them options for management and not referral…” (SSI 4, female, nurse-midwife with 7 years of experience, CDSA uptake below average)

Sometimes, the ePOCT + algorithm slightly differed from IMCI or other national guidelines, which unsettled healthcare providers. This was observed, for instance, with the diagnosis “suspicion of malaria,” which was almost systematically rejected (95.7% rejected in Tanzanian children aged 2 months to 14 years, and 83.1% in Rwanda). Multiple healthcare workers were unaware that this diagnosis was generated specifically when a child presented with fever and a malaria test was indicated but unavailable. In these situations, the algorithm recommended either referral for testing or, if referral was not feasible, presumptive treatment. However, the referral option only became visible once the diagnosis was accepted. This contrasts with the IMCI recommendations, which support presumptive antimalarial treatment in all febrile children in high-risk areas, and in low-risk areas when no obvious alternative diagnosis is found and no test is available, and is not labelled as “suspicion of malaria” [25].

“She would reject the diagnosis because if she agrees to that the tool proposes to give an antimalarial which I think is not appropriate to prescribe without a test result.” (SSI 4, female, nurse-midwife with 7 years of experience, CDSA uptake below average)

Some healthcare providers also reported rejecting diagnoses because the recommended medications (often two possible lines of treatment) were not available at their facility. This suggests that resource constraints also played a role in diagnostic disagreements.

Misunderstandings related to diagnostic labels contributed to some of the observed disagreement. Quantitative data suggests that overly specific diagnostic labels may have caused confusion. For example, in Rwanda, the diagnosis “constipation” was frequently rejected in children 2 months and older (41%), with “non-severe abdominal condition” selected as an alternative in two-thirds of those cases. A similar trend was noted for “corneal abrasion,” which was rejected in 68% of cases, with “severe eye disease” chosen as the alternative in 70% of them. Notably, both diagnoses required referral, suggesting that the disagreement likely pertained not to the management decision itself but rather to the diagnostic label, which is less detailed in national guidelines [33].

Rejecting diagnoses to avoid referring a patient or to report to higher authorities

A number of healthcare providers described rejecting severe diagnoses because they did not agree with the subsequent proposal of referring the patient if they were to accept such diagnoses. This was also found in the quantitative data where many severe diagnoses were rejected (Table 4). In such situations, milder diagnoses were often selected instead despite the presence of symptoms and signs of severe disease. For example, the diagnosis of “severe abdominal condition” was often rejected despite the presence of bilious vomiting, with “non-severe abdominal condition” chosen as an alternative. Many interviewees also described rejecting certain diagnoses, often severe diagnoses, for fear of alerting officials.

“Initially I thought the diagnosis of low weight for age would recommend a referral which I would say, if that is the case, maybe clinicians reject to avoid giving referrals and to be seen that they are referring so many patients.” (SSI 5, male, nurse with 1 year of experience, CDSA uptake above average)

Table 4

Examples of severe diagnoses frequently rejected
Diagnoses	Number rejected/Total proposed (%) in Tanzania	Number rejected/Total proposed (%) in Rwanda
Complicated severe acute malnutrition	14/27 (51.9%)	133/155 (85.8%)
Omphalitis complicated severe (YI)	0/3 (0.0%)	6/9 (66.7%)
Severe abdominal condition	28/45 (62.2%)	62/103 (60.2%)
Severe abdominal problem (YI)	8/45 (17.8%)	4/8 (50.0%)
Severe clinical infection (YI)	96/291 (33.0%)	92/150 (61.3%)
Severe croup	2/10 (20.0%)	16/32 (50.0%)
Severe dehydration	23/51 (45.1%)	49/82 (59.8%)
Severe eye disease	26/57 (45.6%)	286/536 (53.4%)
Severe skin infection (YI)	7/30 (23.3%)	62/103 (60.2%)
Suspected severe malaria	3/4 (75.0%)	6/6 (100%)
Uncomplicated severe acute malnutrition	48/137 (35.0%)	165/248 (66.5%)

*Diagnoses identified based on semi-structured interviews and qualitative evaluation of quantitative data; Unless otherwise noted as YI (young infant), all diagnoses refer to consultations with children aged 2 months to 14 years.

Error in the algorithm

The analysis of diagnosis rejections also revealed an error in the algorithm, which led to a prompt correction. High rejection of hyperglycemia (205/246; 83.3%) diagnoses in Rwanda was partly attributed to confusion around the units used in the algorithm. While healthcare providers in Rwanda typically use mg/dL, the algorithm displayed values in mmol/dL, a unit they were unfamiliar with and which is incorrect, as the standard unit should be mmol/L.

Algorithm modification and before-after analysis

Based on the quantitative analysis of acceptance and rejection rates, insights from qualitative interviews, and input from clinical and technical experts involved in algorithm development and field implementation, 21 proposed modifications were submitted to the Tanzanian clinical expert committee. These changes aimed to achieve the following objectives: 1. Reduce the number of questions, 2. Align more closely with the IMCI guidelines to avoid confusion among healthcare providers, 3. Enhance guidance for challenging diagnoses, 4. Improve clarity in phrasing, and 5. Restrict anthropometric measurements to only essential parameters. Of these proposed changes, 18 were validated by the clinical expert committee and subsequently implemented (Supplementary Table 10). Several questions and diagnoses were rephrased to avoid negative wording (“Vaccinations incomplete for age?” was replaced by “Vaccinations complete for age?”, and the diagnosis “HIV unlikely” by “General counseling”) and to improve clarity (“Suspected malaria” was replaced by “Malaria test not available”). The diagnoses “Fever without source” and “Severe dehydration” were revised to align more accurately with IMCI guidelines. Additional guidance was introduced for skin diagnoses, with diagnoses now conditioned based on accompanying symptoms such as fever, pruritus, or pain. Finally, some complex diagnoses were simplified, for example, “corneal abrasion” was consolidated under “severe eye disease”. The identification of potential issues with the algorithm in one country informed adaptations in the other, when similar challenges were anticipated or the solutions were deemed potentially beneficial across settings. In Rwanda, 8 of these changes were deemed relevant and implemented (Supplementary Table 10).

After the ePOCT + modifications, 10,595 consultations were recorded in Tanzania between October 2023 and September 2024, and 21,888 in Rwanda between October 2023 and November 2024. Of these, 9,630 consultations from Tanzania and 17,697 from Rwanda met inclusion criteria for this analysis (Figure S2). Relative to the pre-modification period, the post-modification cohorts included fewer young infants in Rwanda (5.4% before vs 1.6% after) and fewer children older than 5 years in Tanzania (12.8% before versus 5.4% after) (Supplementary Table 8). The mean number of algorithm-proposed diagnoses per consultation was similar to the pre-modification period (Supplementary Table 9). Overall diagnosis acceptance was also similar before and after, except among young infants in Tanzania, where acceptance increased from 61.5% to 84.1% and the proportion of consultations with ≥ 1 rejected diagnosis fell from 60.8% to 26.6%.

In our granular analysis of diagnosis rejections, we prespecified that the modifications would increase acceptance proportions for six diagnoses in Tanzania and five in Rwanda (Supplementary Table 10). Four of them did show increased acceptance in Tanzania (Table 5). Notably, clearer phrasing – for instance, “Vaccinations complete for age” “HIV unlikely,” and “Suspected malaria”- was associated with increased acceptance, suggesting that diagnostic wording might play a role in health-worker interpretation and understanding. Conversely, consolidating “critical illness,” “severe clinical infection,” and “severe pneumonia” into “possible serious bacterial infection” for young infants did not result in higher acceptance rates. This suggests that other factors, such as challenges in identifying severity indicators or hesitancy to refer may have contributed to the rejections, as noted earlier.

In Rwanda, “Vaccinations complete for age” and “Suspected malaria” modifications were not associated with increased acceptance (Table 6). Three other revised diagnoses, including “severe eye disease” and “non-severe abdominal condition” showed an increased acceptance.

Table 5

Acceptance rates of diagnoses before and after changes to the algorithm in Tanzania
Diagnosis	Modifications	Number Accepted/Total proposed (%) before change	Number Accepted/Total proposed (%) after change	p-Value (Fisher’s exact test)
Incomplete vaccination (YI)	"History of Vaccinations Incomplete for Age" was renamed "Vaccinations Complete for Age" to avoid negative wording	147/305 (48.2%)	77/93 (82.8%)	< 0.001
Suspected malaria	Renamed "Malaria Test Not Available" to clarify its intended use.	6/140 (4.3%)	6/10 (60%)	< 0.001
HIV unlikely	Renamed "General counselling " to clarify its intended use.	324/652 (50.4%)	93/96 (96.9%)	< 0.001
Severe dehydration	Revised to align more accurately with IMCI guidelines	28/51 (54.9%)	8/9 (88.9%)	0.7
Severe eye disease	The « Corneal abrasion » diagnosis was merged into «Severe eye disease», incorporating specific management for «corneal abrasion».	31/57 (54.4%)	20/36 (55.6%)	1.0
Corneal abrasion		4/5 (80.0%)	20/36 (55.6%)	1.0
Critical illness	The « Critical illness», «Severe clinical infection”, and “Severe pneumonia” diagnoses were merged into «Possible serious bacterial infection».	9/15 (60.0%)	25/41 (61.0%)	0.5
Severe clinical infection		195/291 (67.0%)
Severe pneumonia		0/2 (0.0%)

*For the cases where the merging of diagnoses was the modification (such as Severe eye disease with Corneal abrasion, and Critical illness with Severe clinical infection and Severe pneumonia), the pre-modification acceptance rates were summed to allow comparison with the merged post-modification diagnoses using Fisher’s exact test. Unless otherwise noted as YI (young infant), all diagnoses refer to consultations for children aged 2 months to 14 years.

Table 6

Acceptance rates of diagnoses before and after changes to the algorithm in Rwanda
Diagnosis	Modifications	Number Accepted/Total proposed (%) before change	Number Accepted/Total proposed (%) after change	p-Value (Fisher’s exact test)
Incomplete vaccination (YI)	"History of Vaccinations Incomplete for Age" was renamed "Vaccinations Complete for Age" to avoid negative wording	62/177 (35.0%)	14/72 (19.4%)	0.02
Suspected malaria	Renamed "Malaria Test Not Available" to clarify its intended use.	11/65 (16.9%)	4/37 (10.8%)	0.6
Uncomplicated suspicion of poisoning	An info button was added to clarify the definition of "Accidental Ingestion of a Potentially Harmful Entity," specifying that it refers to the intake of a potentially poisonous substance.	0/7 (0.0%)	3/7 (42.9%)	0.2
Severe eye disease	The « Corneal abrasion » diagnosis was merged into «Severe eye disease», incorporating specific management for «corneal abrasion».	250/536 (46.6%)	445/707 (62.9%)	< 0.001
Corneal abrasion		7/20 (35.0%)	445/707 (62.9%)	< 0.001
Non-severe abdominal condition	The « Constipation » diagnosis was merged into «Non-severe abdominal condition», incorporating specific management for «corneal abrasion».	1,712/2,028 (84.4%)	3,091/3,537 (87.4%)	< 0.001
Constipation		18/39 (46.1%)	3,091/3,537 (87.4%)	< 0.001

*For the cases where the merging of diagnoses was the modification (such as Severe eye disease with Corneal abrasion, and Non-severe abdominal condition with Constipation), the pre-modification acceptance rates were summed to allow comparison with the merged post-modification diagnoses using Fisher’s exact test. Unless otherwise noted as YI (young infant), all diagnoses refer to consultations for children aged 2 months to 14 years.

This study employed a mixed-methods approach to assess healthcare workers’ acceptance in the clinical content of the ePOCT + CDSA, after approximately one year of routine clinical use within a trial setting. The analysis utilized data from over 25,000 consultations from 36 health facilities across Tanzania and Rwanda, as well as semi-structured interviews conducted in Tanzania. Based on these insights, we proposed and implemented targeted modifications to the algorithms and explored their subsequent impact on user acceptance.

To the best of our knowledge, this is the first study to investigate potential enhancements to a CDSA by analyzing diagnosis acceptance proportions. This approach, unique to the ePOCT + CDSA, enables precise identification of diagnostic algorithms that may require modification. Additionally, the qualitative component of this study provided insights into attitudes toward specific algorithm components and broader barriers, aligning with findings reported in similar contexts [12, 15, 34].

An analysis of the overall acceptance proportion of ePOCT + diagnoses highlighted notable differences between the two countries, with an overall acceptance of 64.8% in Rwanda compared with 92.4% in Tanzania. Rwandan providers demonstrated a higher rejection for diagnoses among both young infants and older children, as well as a greater frequency of manual additions. This may reflect considerations specific to the Rwandan ePOCT + algorithm (such as the blood glucose unit discrepancy) but might also indicate a more proactive stance among Rwandan providers, potentially driven by higher confidence in their clinical skills, differences in training, CDSA implementation, healthcare practices, or the organizational structure and functioning of health facilities between the countries [35].

However, further examination of the rejected diagnoses revealed that approximately 40% were common across both countries, suggesting that some issues might be related to the algorithm itself, rather than solely due to contextual factors. Indeed, both quantitative and qualitative results, echoing findings from the literature, revealed a complex interplay of software-related, guideline-related, and healthcare provider-related factors influencing the acceptance and usage of the tool [36, 37].

Guideline-Related Barriers: Reflections with the implementation team, informed by quantitative findings, suggested that ePOCT + may at times be overly specific, distinguishing between similar conditions with narrowly defined diagnoses or insufficiently supportive of clinical reasoning, for example by omitting key symptoms such as fever or pruritus in the assessment of skin conditions. This underscores the need for a careful balance between simplicity and specificity in algorithm design. Additionally, certain terminologies and labels appeared overly complex or confusing for healthcare workers. For example, diagnoses such as "suspicion of malaria" versus “malaria test unavailable” or the use of negative phrasing created ambiguity, potentially impairing diagnostic clarity [38]. Moreover, discrepancies between ePOCT + and established national guidelines led to cognitive dissonance among providers, reducing their confidence and trust in the tool, the CDSA recommendations conflicting with their expertise and beliefs [39].

The pattern of rejected hypoglycemia and hyperglycemia diagnoses in Rwanda revealed an algorithmic error, specifically a blood-glucose unit mismatch.

These findings underscore the critical role of end-user engagement in developing and refining clinical algorithms, while also demonstrating how real-world provider feedback can serve as an essential safety mechanism. The before-after analysis showed a trend toward increased diagnosis acceptance, highlighting the value of an iterative design process that incorporates end-user feedback, helping to increase its quality and relevance to the local context [21, 40–42].

Healthcare provider-related barriers: we observed instances where diagnoses were not confirmed by healthcare workers despite the presence of established diagnostic criteria, particularly for severe conditions. Both qualitative and quantitative findings offer some possible explanations: (1) limited familiarity with certain diagnostic criteria and severity indicators; (2) hesitancy among healthcare workers or caregivers to accept severe diagnoses because of downstream implications (e.g., referrals), a challenge noted in prior studies on CDSS impact [43] or fear of scrutiny from higher authorities; and (3) potential inaccuracies in data entry. For example, in Tanzania, some providers prioritized their clinical judgment over algorithm recommendations, especially in complex cases such as malnutrition, where visual assessment and experience take precedence over anthropometric measures, leading to the rejection of valid diagnoses. Such an approach can result in worse clinical outcomes as observed in a randomized trial subgroup analysis comparing visual assessment compared to anthropometric values to identify and manage children with malnutrition [44].

Together, these findings underscore the need for targeted training programs that not only enhance clinical knowledge and skills but also build trust in the algorithm. By bridging the gap between algorithm recommendations and providers’ clinical judgment, such programs can address barriers to optimal CDSS use, fostering better adherence to guidelines and supporting more confident, guideline-aligned decision-making in complex cases [45, 46].

Context-Related Barriers: Structural limitations within the Tanzanian and Rwandan health systems,particularly workforce shortages and inconsistent availability of medications, likely undermine the applicability of certain algorithmic recommendations, as found in other settings [47, 48]. These constraints may contribute to provider reluctance to refer patients [49], highlighting the importance of aligning clinical decision support tools with the operational realities of low-resource settings [21].

Limitations

The study focused on the acceptability of the clinical content, without extensively addressing other critical factors influencing CDSA uptake, such as technical aspects (e.g., user-friendliness, processing speed, and integration with existing electronic medical records) and organizational factors (e.g., disruptions to routine workflow and healthcare provider workload) [39]. While the overall number of consultations was substantial, the number of cases for some specific diagnoses was small, limiting the strength and generalizability of certain findings. Additionally, the pre-post analysis was exploratory. In the absence of a formal comparison with a control group, causal inference cannot be made. Other factors - such as increased familiarity with the tool over time, or selective use by healthcare workers trusting the tool - may have influenced diagnostic acceptance following the intervention.

Given that the interviewers were part of the implementation team (Swiss and Tanzanian), healthcare providers may have been reluctant to openly criticize the tool, potentially introducing social desirability bias. This may have influenced the responses, leading to an underreporting of challenges. Furthermore, the qualitative component used a question format designed to explore reasoning by asking, “Why do you think health providers…?”, a phrasing that may have encouraged speculative responses, given that participants might lack direct insight into their colleagues’ thought processes. Moreover, the study employed a customized analytical framework instead of formal qualitative coding methods, introducing some subjectivity. Finally, the use of “intelligent verbatim” transcriptions, which clean up transcripts rather than providing word-for-word accuracy, may have led to the omission of nuanced details from the original interviews.

No qualitative studies were conducted among healthcare providers in Rwanda; the analysis was limited to quantitative findings. However, additional contextual factors, such as clinical practices and the integration of CDSA algorithms, may influence outcomes within the Rwandan health system.

The analysis of quantitative routine CDSA data, complemented by qualitative feedback, provided important insights that informed modifications to the ePOCT + CDSA that may have increased acceptance of some diagnoses. In addition to refining the clinical content by enhancing its relevance, clarity, and level of detail, this approach supports the identification of additional facilitators for CDSA adoption, such as targeted training on specific aspects of the tool [39]. The adaptability of this methodology offers a promising framework for optimizing other clinical decision support systems in diverse healthcare settings, aligning with the Principles for Digital Development [19] and the WHO SMART guidelines [20].

CDSA

Clinical Decision Support Algorithm

CRP

C-Reactive Protein

ePOCT+

electronic Point Of Care Test +

HIV

Human Immunodeficiency Virus

IMCI

Integrated Management of Childhood Illness

IT

Information Technology

SSI

Semi-structured interview

WHO

World Health Organization

Ethics approval and consent to participate

Written informed consent was obtained from all parents or guardians of the participants when they attended the participating health facility. Written informed consent from all healthcare providers who participated in the semi-structured interviews was also obtained. Ethical approval was granted in Tanzania from the Ifakara Health Institute (IHI/IRB/No: 11-2020), the Mbeya Medical Research Ethics Committee (SZEC-2439/R.A/V.1/65), the National Institute for Medical Research Ethics Committee (NIMR/HQ/R.8a/Vol. IX/3486 and NIMR/HQ/R.8a/Vol. IX/3583) in Tanzania. Ethics approval was granted in Rwanda from the National Ethics Committee (original protocol: 752/RNEC/2020; extensions and amendments: 975/RNEC/2021, 431/RNEC/2022, 246/RNEC/2023, 269/RNEC/2023), the National Health Research Committee (NHRC/2020/Prot/031), and the National Institute of Statistics (0654/2020/10/NISR). Ethics approval was granted in Switzerland from the cantonal ethics review board of Vaud (CER-VD 2020–02800 and CER-VD 2020–02799).

Consent for publication

Not applicable. This manuscript does not contain any individual person’s data in any form (including individual details, images, or videos).

Competing interests

The authors declare no competing interests.

Author’s information

Aymeric Poitiers: [email protected]; ORCID id: 0009-0002-7732-1817

Haykel Karoui: [email protected]; ORCID id: 0009-0007-9444-5818

Alix Miauton: [email protected]; ORCID id: 0000-0003-1849-8874

Rainer Tan: [email protected]; ORCID id: 0000-0002-9273-9632

Funding

The study sponsor, Centre for Primary Care and Public Health (Unisanté), University of Lausanne, was responsible for the study design, preparation of the manuscript, and the decision to submit it for publication. This work was supported by grants from the Fondation Botnar, Switzerland (grant number 6278) and from the Swiss Development Cooperation (project number 7F-10361.01.01). The funders of the study had no role in the study design, data collection, data analysis, interpretation of data, or writing of the report.

Author Contribution

AP, HK, AM, and RT designed the study, analyzed the data, and drafted the first version of the manuscript. AP, HK, AM, AK, VVK, GAL, and RT contributed to manuscript revisions. AP, RT, MJ, and GK collected the qualitative data. AP, HK, GK, AK, VR, VVK, MN, GA, CE, CM, LL, TD, LC, GAL, FB, VDA, AM, RT and the ePOCT+ collaboration group contributed to the development and modification of the algorithm. IEM, PA, AM, AK, and RT coordinated data collection. HK, MJ, GK, AK, VR, GA, CE, CM, LL, IEM, PA, TD, LC, AM and RT implemented the study in Tanzania and Rwanda. VDA and AK were cross-site coordinator of the DYNAMIC trial. NN, HM, and VDA acquired funding and supervised the study. All authors read and approved the final version of the manuscript.

Acknowledgement

We gratefully acknowledge the contributions of the research assistants from the Ifakara Health Institute, the Mbeya Medical Research Centre—National Institute for Medical Research, and the Swiss Tropical and Public Health Institute in Kigali, Rwanda, for their support in data collection. We also thank the Tanzanian expert panel for their valuable input in adapting the algorithm to the local context.We extend our heartfelt thanks to all the patients and caregivers who participated in the study.We would also like to acknowledge the important contributions of Godfrey Kavishe, who sadly passed away before the study was completed.

Data Availability

The datasets used and analysed during the current study are available from the corresponding author on reasonable request.The code for the medAL-reader application used to collect data entered by healthcare workers (including demographic, clinical, diagnosis, prescription and referral data of the consultations) can be found at https://github.com/Unisante/medal-reader.

Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI (2020) An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digit Med. ;3:17. Epub 20200206. 10.1038/s41746-020-0221-y. PubMed PMID: 32047862. PubMed Central PMCID: PMC7005290
Keitel K, D’Acremont V (2018) Electronic clinical decision algorithms for the integrated primary care management of febrile children in low-resource settings: review of existing tools. Clin Microbiol Infect 24:845–855 Epub 20180801. 10.1016/j.cmi.2018.04.014
Beynon F, Guérin F, Lampariello R, Schmitz T, Tan R, Ratanaprayul N et al (2023) Digitalizing Clinical Guidelines: Experiences in the Development of Clinical Decision Support Algorithms for Management of Childhood Illness in Resource-Constrained Settings. Glob Health Sci Pract 11 Epub 20230828. 10.9745/GHSP-D-22-00439
Sarrassat S, Lewis JJ, Some AS, Somda S, Cousens S, Blanchet K (2021) An Integrated eDiagnosis Approach (IeDA) versus standard IMCI for assessing and managing childhood illness in Burkina Faso: a stepped-wedge cluster randomised trial. BMC Health Serv Res 21:354 Epub 20210416. 10.1186/s12913-021-06317-3
Mitchell M, Hedt-Gauthier BL, Msellemu D, Nkaka M, Lesh N (2013) Using electronic technology to improve clinical care – results from a before-after cluster trial to evaluate assessment and classification of sick children according to Integrated Management of Childhood Illness (IMCI) protocol in Tanzania. BMC Med Inf Decis Mak 13:95 Epub 20130827. 10.1186/1472-6947-13-95
Tan R, Kavishe G, Kulinkina AV, Renggli S, Luwanda LB, Mangu C et al (2024) A cluster randomized trial assessing the effect of a digital health algorithm on quality of care in Tanzania (DYNAMIC study). PLOS Digit Health. ;3:e0000694. Epub 2024déc. 10.1371/journal.pdig.0000694
Tan R, Kavishe G, Luwanda LB, Kulinkina AV, Renggli S, Mangu C et al (2024) A digital health algorithm to guide antibiotic prescription in pediatric outpatient care: a cluster randomized controlled trial. Nat Med 30:76–84 Epub 202401. 10.1038/s41591-023-02633-9
Keitel K, Kagoro F, Samaka J, Masimba J, Said Z, Temba H et al (2017) A novel electronic algorithm using host biomarker point-of-care tests for the management of febrile illnesses in Tanzanian children (e-POCT): A randomized, controlled non-inferiority trial. PLOS Med 14:e1002411 Epub 20171023. 10.1371/journal.pmed.1002411
Schmitz T, Beynon F, Musard C, Kwiatkowski M, Landi M, Ishaya D et al (2022) Effectiveness of an electronic clinical decision support system in improving the management of childhood illness in primary care in rural Nigeria: an observational study. BMJ Open 12:e055315 Epub 20220721. 10.1136/bmjopen-2021-055315
Horwood C, Haskins L, Mapumulo S, Connolly C, Luthuli S, Jensen C et al (2024) Electronic Integrated Management of Childhood Illness (eIMCI): a randomized controlled trial to evaluate an electronic clinical decision-making support system for management of sick children in primary health care facilities in South Africa. BMC Health Serv Res 24:177 Epub 20240208. 10.1186/s12913-024-10547-6
Beynon F, Mhalu G, Kumar D, Cicconi S, Langet H, Levine GA et al (2024) Effectiveness of Introducing Pulse Oximetry and Clinical Decision Support Algorithms for the Management of Sick Children in Primary Care in India and Tanzania on Hospitalisation and Mortality: The TIMCI Pragmatic Cluster Randomised Controlled Trial. Epub 20241105 10.2139/ssrn.5008738
Shao AF, Rambaud-Althaus C, Swai N, Kahama-Maro J, Genton B, D’Acremont V et al (2015) Can smartphones and tablets improve the management of childhood illness in Tanzania? A qualitative study from a primary health care worker’s perspective. BMC Health Serv Res 15:135 Epub 20150402. 10.1186/s12913-015-0805-4
Jensen C, McKerrow NH (2022) The feasibility and ongoing use of electronic decision support to strengthen the implementation of IMCI in KwaZulu-Natal, South Africa. BMC Pediatr 22:80 Epub 20220207. 10.1186/s12887-022-03147-y
Chirambo GB, Muula AS, Thompson M, Hardy VE, Heavin C, Connor YO et al (2021) End-user perspectives of two mHealth decision support tools: Electronic Community Case Management in Northern Malawi. Int J Med Inf 145:104323 Epub 202101. 10.1016/j.ijmedinf.2020.104323
Bessat C, Zonon NA, D’Acremont V (2019) Large-scale implementation of electronic Integrated Management of Childhood Illness (eIMCI) at the primary care level in Burkina Faso: a qualitative study on health worker perception of its medical content, usability and impact on antibiotic prescription and resistance. BMC Public Health 19:449 Epub 20190429. 10.1186/s12889-019-6692-6
Mitchell M, Getchell M, Nkaka M, Msellemu D, Van Esch J, Hedt-Gauthier B (2012) Perceived improvement in integrated management of childhood illness implementation through use of mobile technology: qualitative evidence from a pilot study in Tanzania. J Health Commun 17(Suppl 1):118–127 PubMed PMID: 22548605
Kilsdonk E, Peute LW, Jaspers MWM (2017) Factors influencing implementation success of guideline-based clinical decision support systems: A systematic review and gaps analysis. Int J Med Inf 98:56–64 Epub 201702. 10.1016/j.ijmedinf.2016.12.001
Bates DW, Kuperman GJ, Wang S, Gandhi T, Kittler A, Volk L et al (2003) Ten Commandments for Effective Clinical Decision Support: Making the Practice of Evidence-based Medicine a Reality. J Am Med Inf Assoc 10:523–530 Epub 200311. 10.1197/jamia.M1370
Waugaman A (2016) From principle to practice: Implementing the Principles for Digital Development. Epub 201602
Mehl G, Tunçalp Ö, Ratanaprayul N, Tamrat T, Barreix M, Lowrance D et al (2021) WHO SMART guidelines: optimising country-level use of guideline recommendations in the digital age. Lancet Digit Health 3:e213–e216 Epub 20210401. 10.1016/S2589-7500(21)00038-8
Gilbert S, Pimenta A, Stratton-Powell A, Welzel C, Melvin T (2023) Continuous Improvement of Digital Health Applications Linked to Real-World Performance Monitoring: Safe Moving Targets? Mayo Clin Proc Digit Health 1:276–287 Epub 20230901. 10.1016/j.mcpdig.2023.05.010
Tan R, Cobuccio L, Beynon F, Levine GA, Vaezipour N, Luwanda LB et al (2023) ePOCT + and the medAL-suite: Development of an electronic clinical decision support algorithm and digital platform for pediatric outpatients in low- and middle-income countries. PLOS Digit Health 2:e0000170 Epub 20230119. 10.1371/journal.pdig.0000170
von Kalckreuth V, Rwandarwacu VP, Cobuccio L, Dusengumuremyi T, Levine GA, Norris M et al (2025) ePOCT + Rwanda: A Clinical Decision Support Algorithm For Managing Sick Children Below 15 Years of Age in Primary Healthcare Settings. Rwanda J Med Health Sci 8:148–162 Epub 20250408
The Dynamic Project A digital tool to improve the management of sick children (Tanzania & Rwanda) [Internet]. Lausanne: Unisanté; c2024 [cited 2025 Sep 18]. Available from: https://dynamic-study.com/
World Health Organization (2014) Integrated Management of Childhood Illness: Chart Booklet. Geneva: WHO. ISBN: 9789241506823
Rambaud-Althaus C, Shao AF, Kahama-Maro J, Genton B, d’Acremont V (2015) Managing the Sick Child in the Era of Declining Malaria Transmission: Development of ALMANACH, an Electronic Algorithm for Appropriate Use of Antimicrobials. PLoS ONE 10:e0127674 Epub 20150710. 10.1371/journal.pone.0127674
Cobuccio LG, Faivre V, Tan R, Vonlanthen A, Beynon F, Barchichat E et al (2025) medAL-suite: A software solution for creating and deploying complex clinical decision support algorithms. BMC Med Inf Decis Mak 25:249 Epub 20250704. 10.1186/s12911-025-03077-6
Kulinkina AV, Rwandarwacu VP, Habakurama J, Cobuccio L, Norris M, Kalisa E et al Effectiveness of a digital clinical decision support algorithm for guiding antibiotic prescribing in pediatric outpatient care in Rwanda: A pragmatic cluster non-randomized controlled trial. medRxiv. 2025;:2025.08.01.25332743. Epub 20250101. 10.1101/2025.08.01.25332743
National Institute of Statistics of Rwanda (2023) Fifth Rwanda population and housing census, 2022: main indicators report. National Institute of Statistics of Rwanda, Kigali
von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP (2007) The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: Guidelines for Reporting Observational Studies. PLoS Med 4 Epub 200710. 10.1371/journal.pmed.0040296. :e296
Gale NK, Heath G, Cameron E, Rashid S, Redwood S (2013) Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med Res Methodol 13:117 Epub 20130918. 10.1186/1471-2288-13-117
Tong A, Sainsbury P, Craig J (2007) Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care 19:349–357 Epub 20071201. 10.1093/intqhc/mzm042
Ministry of Health, Republic of Rwanda Primary eye care handbook for first level health workers. Kigali: Ministry of Health
Savai S, Kamano J, Misoi L, Wakholi P, Hasan MK, Were MC (2022) Leveraging mHealth usage logs to inform health worker performance in a Resource-Limited setting: Case example of mUzima use for a chronic disease program in Western Kenya. PLOS Digit Health 1:e0000096 Epub 20220901. 10.1371/journal.pdig.0000096
Esmaeilzadeh P, Sambasivan M, Kumar N, Nezakati H (2015) Adoption of clinical decision support systems in a developing country: Antecedents and outcomes of physician’s threat to perceived professional autonomy. Int J Med Inf 84:548–560 Epub 20150801. 10.1016/j.ijmedinf.2015.03.007
Odhus CO, Kapanga RR, Oele E (2024) Barriers to and enablers of quality improvement in primary health care in low- and middle-income countries: A systematic review. PLOS Glob Public Health 4 Epub 20240118. 10.1371/journal.pgph.0002756. :e0002756
Castillo RS, Kelemen A (2013) Considerations for a successful clinical decision support system. Comput Inf Nurs CIN 31:319–326 quiz 327–328. Epub 201307. 10.1097/NXN.0b013e3182997a9c
Colosi R (2005) Negatively worded questions cause respondent confusion. Proceedings of the Survey Research Methods Section, American Statistical Association. :2896–903
Meunier P-Y, Raynaud C, Guimaraes E, Gueyffier F, Letrilliart L (2023) Barriers and Facilitators to the Use of Clinical Decision Support Systems in Primary Care: A Mixed-Methods Systematic Review. Ann Fam Med 21:57–69 Epub 2023. 10.1370/afm.2908
Khairat S, Marc D, Crosby W, Al Sanousi A, Reasons For Physicians Not Adopting Clinical Decision Support Systems (2018) Critical Analysis. JMIR Med Inf 6:e24 Epub 20180418. 10.2196/medinform.8912
Thum F, Kim MS, Genes N, Rivera L, Beato R, Soriano J et al (eds) (2014) Usability improvement of a clinical decision support system. Design, User Experience, and Usability User Experience Design for Everyday Life Applications and Services: Third International Conference, DUXU 2014, Held as Part of HCI International 2014, Heraklion, Crete, Greece, June 22–27, Proceedings, Part III 3; 2014: Springer
Trafton J, Martins S, Michel M, Lewis E, Wang D, Combs A et al (2010) Evaluation of the acceptability and usability of a decision support system to encourage safe and effective use of opioid therapy for chronic, noncancer pain by primary care providers. Pain Med Malden Mass. ;11:575–585. Epub 201004. 10.1111/j.1526-4637.2010.00818.x. PubMed PMID: 20202142
Muhiyaddin R, Abd-Alrazaq AA, Househ M, Alam T, Shah Z (2020) The Impact of Clinical Decision Support Systems (CDSS) on Physicians: A Scoping Review. Stud Health Technol Inf 272:470–473 Epub 20200626. doi: 10.3233/SHTI200597. PubMed PMID: 32604704
Tan R, Kagoro F, Levine GA, Masimba J, Samaka J, Sangu W et al (2020) Clinical Outcome of Febrile Tanzanian Children with Severe Malnutrition Using Anthropometry in Comparison to Clinical Signs. Am J Trop Med Hyg 102:427–435 Epub 202002. 10.4269/ajtmh.19-0553
Borges do Nascimento IJ, Abdulazeem H, Vasanthan LT, Martinez EZ, Zucoloto ML, Østengaard L et al (2023) Barriers and facilitators to utilizing digital health technologies by healthcare professionals. NPJ Digit Med 6:161 Epub 20230918. 10.1038/s41746-023-00899-4
Fenneman A, Sickmann J, Pitz T, Sanfey AG (2021) Two distinct and separable processes underlie individual differences in algorithm adherence: Differences in predictions and differences in trust thresholds. PLoS ONE 16 Epub 20210225. 10.1371/journal.pone.0247084. :e0247084
Titaley CR, Jusril H, Ariawan I, Soeharno N, Setiawan T, Weber MW (2014) Challenges to the implementation of the integrated management of childhood illness (IMCI) at community health centres in West Java province, Indonesia. WHO South-East. Asia J Public Health 3:161–170 Epub 2014. 10.4103/2224-3151.206732
Kruse C, Betancourt J, Ortiz S, Valdes Luna SM, Bamrah IK, Segovia N (2019) Barriers to the Use of Mobile Health in Improving Health Outcomes in Developing Countries: Systematic Review. J Med Internet Res 21:e13263 Epub 20191009. doi: 10.2196/13263. PubMed PMID: 31593543. PubMed Central PMCID: PMC6811771
Nakayuki M, Basaza A, Namatovu H (2021) Challenges Affecting Health Referral Systems in Low-And Middle-Income Countries: A Systematic Literature Review. Eur J Health Sci 6(3):33–44. https://doi.org/10.47672/ejhs.809

No competing interests reported.

ImprovingePOCTSupplementaryMaterial20250928.docx

Clinical acceptance of a digital health clinical decision support algorithm for children in Tanzania and Rwanda: A mixed-method and before-after analysis from the DYNAMIC study

Status:

Version 1

Abstract

Background:

Results:

Conclusions:

Figures

INTRODUCTION

METHODS