Study Design
This study evaluates the outcomes and costs of the DHI named TESTporuch ("TEST nearby"), designed to promote HIV testing in wartime Ukraine between August 2022 and January 2025. The World Bank's Framework for the Economic Evaluation of Digital Health Interventions informed the study design [47].
DHI Description
The DHI was developed by the State Enterprise Center of Public Health of the Ministry of Health of Ukraine (the CPH of the MoH of Ukraine) with the Community Action for HIV Control Project support [48]. Following digital marketing strategy principles, the DHI integrated three synergistic elements, each serving a distinct role in the HIV testing promotion funnel (Figure 1): Communication campaigns to provide outreach through targeted multi-channel digital marketing; Multi-landing website to engage visitors with dynamic, population-specific interfaces tailored to different key populations; and Telegram-based chatbot to facilitate linkage to services through personalized HIV risk assessment and connections to HIV testing service delivery points. This comprehensive approach created a seamless user journey specifically designed to promote HIV testing uptake among hard-to-reach populations in Ukraine during wartime conditions.
Communication Campaign
A communication campaign was implemented as a component of the intervention. In addition, information about the chatbot and multi-landing launch was disseminated through communication materials through official government and community stakeholder channels [49, 50].
Two sequential information campaigns were implemented during 2023-2024 within the framework of a national initiative to enhance demand and access to HIV services across Ukraine (excluding temporarily occupied territories), coordinated by the CPH of the MoH of Ukraine. Both campaigns utilized a combination of digital (≥70%) and offline (≤30%) channels. The present study analyzes only the digital component of these campaigns designed to facilitate user engagement with the multi-landing website. Campaign messaging consistently emphasized accessibility, privacy, and cost-free aspects of HIV testing services through communication strategies tailored to specific audience segments (see Supplementary Materials 1 and 2 for campaign messages and creative examples).
The first campaign, conducted from April to July 2023, targeted "infantile" (the general population who do not test for HIV despite risky behaviors). This campaign aimed to increase awareness about community-based HIV testing services provided by NGOs and to drive demand for these services. Digital channels included dating sites, adult websites, gossip and lifestyle news portals, and marketplaces. Promotion tactics employed Facebook and Instagram advertising, YouTube video campaigns, Google Display Network banner advertising, Vpoint programmatic advertising, and contextual targeting.
The second campaign, implemented from March to May 2024, focused on targeting specific key populations: female sex workers (FSW), men who have sex with men (MSM), and persons who inject drugs (PWID). Digital channels included platforms similar to the first campaign, with refined targeting. Promotion technologies utilized included Google Display Network, Facebook and Instagram advertising, Performance Max automated campaigns, DemandGen (demand generation) tactics, and Vpoint programmatic targeting. Additional promotion methods included blogger engagement, mobile application advertising, and contextual advertising on popular regional websites. Geotargeting strategies were employed to reach PWID in specific territories.
Multi-landing Website
A multi-landing website was launched in April 2023 following the chatbot deployment. The website was designed for specific key populations, including PWID, MSM, and FSW, as well as the general population at risk of HIV infection (see Appendix 3 for the multi-landing website design and content).
The multi-landing platform featured four distinct user interfaces, each offering population-specific content and messaging while maintaining a unified technical infrastructure. Each interface presented targeted information about HIV, testing procedures, an open interactive map with HIV testing service delivery points, and direct integration with the chatbot. The website architecture employed responsive design principles to ensure functionality across various devices, with particular optimization for mobile access.
The website's launch was postponed due to a government prohibition on providing open information about the locations of health facilities, citing the threat of an attack on them.
Chatbot
Launched in August 2022, the TESTporuch chatbot, beginning with privacy assurances, offers personalized HIV risk evaluation through a structured algorithm of initial screening and in-depth risk assessment questions with privacy assurances. It also provides location-based referrals to HIV testing service delivery points across Ukraine with personalized recommendations based on user profiles (see Appendix 4 for algorithm specifications). Additional functionalities include connections to the National HIV/AIDS Hotline, information on pre- and post-exposure prophylaxis, free oral test ordering for HIV self-testing, a personal user account with stored search history, and reminders for HIV testing.
The system was adopted to operate within Ukraine's wartime constraints, incorporating security features to protect the location information of HIV services delivery points from potential targeting. Development occurred between June and August 2022, using a low-code platform to minimize costs while ensuring compliance with the national security requirements.
The chatbot’s messenger selection criteria included widespread adoption among target populations and perceived privacy features, making it an accessible channel for reaching key populations in Ukraine. The chatbot development platform was selected based on formal registration in Ukraine and compliance with the General Data Protection Regulation (GDPR).
DHI Classification
According to the WHO Classification of Digital Interventions, Services and Applications in Health (CDISAH) [46], the TESTporuch DHI addresses multiple health system challenges: communication roadblocks (1.4), lack of access to information (1.5), poor experience of persons (3.1), low demand for services (5.1), loss to follow-up (5.4), insufficient person engagement (8.1), and inadequate representation (9.2). The targeted primary users for this intervention are persons at risk of HIV infection, for whom it transmits targeted health information based on health status or demographics (1.1.2), targeted alerts and reminders (1.1.3), look-up of information on health and health services (1.6.1), and simulated human-like conversations (1.6.2). These capabilities are delivered through communication systems (A1), decision support systems (A3), and telehealth systems (A9).
Data Collection and Preparation
The study utilized multiple data sources for a comprehensive evaluation.
First, anonymized user interaction data were extracted from two interconnected Telegram-based chatbots: TESTporuch, which focused on HIV testing promotion, and ARTporuch, which aims to reconnect war-affected Ukrainian PLHIV to antiretroviral therapy, as well as other medical and social services in Ukraine and abroad. A left join operation was performed between the TESTporuch and ARTporuch users’ data tables in databases, utilizing unique Telegram identifiers (UID) as the primary key for database integration and ensuring the availability of registration timestamps within the study period (Figure 2).
As a next step, specific inclusion criteria were established to identify TESTporuch users who subsequently registered in ARTporuch as a proxy for HIV positive cases. Users were included in the further statistical analysis if they: (1) registered in both TESTporuch and ARTporuch platforms; (2) completed TESTporuch initial screening; and (3) had a TESTporuch registration timestamp preceding or coinciding with ARTporuch registration. Data preprocessing and integration were conducted using Microsoft Power BI with standardized validation procedures to ensure data integrity and compatibility between the chatbot’s databases. Data cleaning procedures included removing duplicate entries, managing missing data, and verifying data completeness. Timestamps were converted to a uniform date format to enable proper chronological analysis of user registration and activity patterns. The resulting integrated dataset, with all personal identifiers removed, is available in Appendix 5.
Second, project documentation, including technical specifications, implementation reports, and financial records, was systematically reviewed to characterize the TESTporuch DHI features and determine cost structures. Implementation reports from two sequential digital communication campaigns (2023-2024) were analyzed to present the target population's reach.
Also, Google Analytics data for the multi-landing website provided supplementary insights on user engagement patterns between April 2023 and January 2025.
Statistical Analysis
Data were analyzed using the R statistical software package (version 4.1.2). Descriptive statistics were applied to analyze user data across all three intervention components (chatbot, multi-landing website, and communication campaign), characterizing the user population through frequencies and proportions for categorical variables.
For the TESTporuch chatbot, user interaction data were analyzed based on predefined categories, with detailed descriptions of all metrics provided in Table 1. For the analysis of conversion from TESTporuch to ARTporuch, we calculated the conversion rate defined as the percentage of potential HIV-positive cases relative to the total number of users who completed initial screening. Pearson's chi-square test with Yates' continuity correction was used to assess the statistical significance of differences in conversion rates between user groups. Fisher's exact test was used to calculate odds ratios (OR) with 95% confidence intervals. Relative risk (RR) was calculated using the epitools package in R. A p-value < 0.05 was considered statistically significant for all analyses.
Table 1. Key Metrics and Definitions for TESTporuch Chatbot Analysis
|
Metric/User Category
|
Description
|
|
Total users
|
Number of unique users who accessed the chatbot
|
|
Initial screening
|
Users who completed the preliminary sorting stage of the chatbot algorithm
|
|
Referred for HIV testing
|
Users who utilized the chatbot's HIV testing service delivery point search function
|
|
Identified HIV risk
|
Number of users with any identified HIV risk factors based on responses to screening questions
|
|
Risky behavior
|
Users who reported behaviors associated with increased HIV transmission risk
|
|
Recent potential HIV exposure
|
Users who reported recent (within 48 hours) potential HIV exposure requiring urgent intervention
|
|
Key population member
|
Users who identified themselves as belonging to at least one key population group at higher risk for HIV
|
|
Unprotected sex
|
Users who reported sexual contact with a partner whose HIV status was unknown
|
|
Chemsex experience
|
Users who reported using psychoactive substances during sexual activity
|
|
STI history
|
Users who reported history of sexually transmitted infections (STIs)
|
|
PWID partner
|
Users who reported sexual contact with PWID
|
|
OST (Opioid Substitution Therapy) experience
|
Users who reported participation in OST
|
|
MSM
|
Users who self-identified as MSM
|
|
PLHIV partner
|
Users who self-identified as sexual partners of PLHIV
|
|
FSW
|
Users who self-identified as FSW
|
|
PWID
|
Users who self-identified as PWID
|
|
Transgender people
|
Users who self-identified as transgender
|
|
Former prisoners
|
Users who self-identified as former prisoners
|
|
Conversion to ARTporuch
|
Users who met criteria for potential HIV positive case
|
Cost Analysis Methodology
Cost analysis was conducted employing the author's DHI lifecycle framework that categorized expenditures across two main stages: Creation and Maintenance. The Creation stage encompassed three phases: Preparation, Development, and Launch. The Maintenance stage included the Operation and Update phases. This model allowed for a structured assessment of both capital expenditures (CAPEX) and operational expenses (OPEX) throughout the DHI lifecycle (see Figure 3). For each phase, costs were categorized by type (Human Resources, Software, Services) and unit of measurement (number of hours or items). All costs were documented in US dollars based on the exchange rate at the time of the transaction.
Effectiveness Evaluation Methodology
A structured approach was established to evaluate the DHI's two key effectiveness metrics, HIV positivity rates and cost-effectiveness, compared to national estimates that represent the overall effectiveness of HIV testing interventions nationwide. The HIV positivity rate was defined as the conversion rate derived from the statistical analysis. Comparative data for national estimates were obtained from official statistical reports published by the CPH of the MoH of Ukraine for 2022-2024. Averages from years’ reports spanning this timeframe were calculated to ensure appropriate comparison with our intervention period (August 2022 through January 2025). For the cost-effectiveness evaluation, we calculated the cost per HIV case identified by dividing the total DHI cost by the number of potential HIV positive cases. These figures were then compared with modeled national estimates for 2022-2024, projected based on publicly available data from 2021. Since the DHI does not include the physical testing component, the cost of HIV testing services is added to ensure a fair comparison with the national estimates.