Process evaluation for the delivery of a water, sanitation and
hygiene mobile health program: findings from the randomised
controlled trial of the CHoBI7 mobile health program
Md. Sazzadul Islam Bhuyian1
, Ronald Saxton2
, Khaled Hasan2
, Jahed Masud1
, Fatema Zohura1
,
Shirajum Monira1
, Shwapon Kumar Biswas1
, M. Tasdik Hasan1
, Tahmina Parvin1
, Ismat Minhaj1
,
Kazi Md. Zillur Rahman3
, Nowshin Papri1
, Mahamud-ur Rashid4
, Lubaba Sharin1
, Alana Teman2
,
Elizabeth D. Thomas2
, Kelsey Alland2
, Alain Labrique2
, David A. Sack2
, Jamie Perin2
, Munirul Alam1 and
Christine Marie George2
1 International Centre for Diarrhoeal Disease Research, Bangladesh(icddr,b), Dhaka, Bangladesh
2 Department of International Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA
3 University of New South Wales, Sydney, NSW, Australia
4 University of Manitoba, Winnipeg, MB, Canada
Abstract objective The Cholera-Hospital-Based Intervention for 7-days (CHoBI7) mobile health (mHealth)
program delivers mobile messages to diarrhoea patient households promoting water treatment and
handwashing with soap. The randomised controlled trial (RCT) of the CHoBI7 mHealth program
demonstrated this intervention was effective in significantly reducing diarrhoea and stunting amoung
young children. The objective of this study was to assess the implementation of the CHoBI7 mHealth
program in delivering mHealth messages during this RCT.
methods 517 diarrhoea patient households with 1777 participants received weekly text, voice and
interactive voice response (IVR) messages from the CHoBI7 mHealth program over the 12-month
program period. The program process evaluation indicators were the following: the percentage of
CHoBI7 mHealth messages received and fully listened to by program households (program fidelity
and dose), and household members reporting receiving and sharing an mHealth message from the
program in the past two weeks (program reach).
results Ninety two percent of text messages were received by program households. Eighty three
percent of voice and 86% of IVR messages sent were fully listened to by at least one household member.
Eighty one percent of IVR quiz responses from households were answered correctly. Program households
reported receiving a CHoBI7 mHealth message in the past two weeks at 79% of monthly household visits
during the 12-month program. Seventy seven percent of participants reported sharing a program message
with a spouse, 55% with a neighbour and 49% with a child during the program period.
conclusion There was high fidelity, dose and reach of mobile messages delivered for the CHoBI7
mHealth program. This study presents an approach for process evaluation that can be implemented
to evaluate future mHealth programs.
keywords mobile health, diarrhoea, randomised controlled trial, water, sanitation and hygiene
Sustainable Development Goals (SDGs): SDG 3 (good health and well-being), SDG 6 (clean water and sanitation),
SDG 9 (industry, innovation and infrastructure), SDG 17 (partnerships for the goals)
Introduction
Diarrhoeal diseases result in an estimated 500,000 deaths
amoung young children annually in low- and middle-in￾come countries (LMICs) [1]. Previous studies have found
that lack of handwashing with soap and water treatment
are important risk factors for paediatric diarrhoeal
disease [2-4]. Water, sanitation and hygiene (WASH)
interventions have the potential to reduce diarrhoeal dis￾eases globally [5]. However, encouraging households to
sustain WASH behaviours over time remains a significant
public health challenge [6]. Effective and scalable WASH
interventions are needed to improve child health in
LMICs.
© 2020 John Wiley & Sons Ltd 985
Tropical Medicine and International Health doi:10.1111/tmi.13414
volume 25 no 8 pp 985–995 august 2020
Phone-based reminders of public health information
have been shown to reduce disease morbidity and
increase health-protective behaviours [7-9]. A recent sys￾tematic review found that 15 randomised controlled trials
(RCTs) of mobile health (mHealth) interventions resulted
in significant improvements in health-protective beha￾viours, and 16 RCTs resulted in significant improvements
in clinical outcomes [10]. In the past 10 years, mobile
phone access and ownership have doubled globally
[11,12]. The total number of mobile phone subscribers in
Bangladesh reached over 160 million registered users in
2019, according to the Bangladesh Telecommunication
Regulatory Commission [13]. This presents an ideal
opportunity to use mobile phones to deliver public health
information in Bangladesh. Mobile technology can be used
to create automated systems to deliver voice and text mes￾sages to a large number of individuals at a low cost [14].
However, there are only two published studies, to our
knowledge, that have evaluated the use of mHealth for
delivery of WASH programs to households and no ran￾domised controlled trials of WASH mHealth programs
[15,16]. Henry et al. was conducted in Tanzania, and this
study targeted 60 youth 18 to 25 years old [15] and
focused on text messages on hygiene promotion. This study
found that 92% of participants received and responded to
program text messages. Tidwell et al. was conducted in
India and focused on mobile phone messaging for mothers
of young children; it found a modest (~5%) significant
improvement in handwashing with soap behaviour [16].
There is an urgent need for rigorous methods to evaluate
delivery of WASH mHealth programs.
In an effort to reduce diarrhoeal diseases in Bangla￾desh, our research group developed the Cholera-Hospital￾Based Intervention for 7 days (CHoBI7) mHealth pro￾gram. This program delivers automated text, voice and
interactive voice response messages promoting handwash￾ing with soap and water treatment behaviours. Messages
are sent by a character named Dr Chobi, a doctor at a
hospital who calls and texts participants about the pro￾moted WASH behaviours [17]. All diarrhoea patient
households receive weekly interactive voice response
(IVR), voice and text messages from the CHoBI7
mHealth program for a 12-month period. Mobile mes￾sages are sent using the web-based VIAMO platform
[18]. The recent RCT of the CHoBI7 mHealth program
demonstrated this program was effective in significantly
increasing handwashing with soap and stored drinking
water quality, and reducing diarrhoea and stunting in
young children [19]. In this study, we assessed the fide￾lity, dose and reach of the CHoBI7 mHealth program
through process evaluation conducted during the RCT of
this intervention. We used data obtained from the
VIAMO platform and participant reports.
Methods
The RCT of the CHoBI7 mHealth program enrolled
769 diarrhoea patients and 1857 household members
from December 2016 to April 2018. The RCT com￾pared the standard recommendation given in Bangladesh
to diarrhoea patients (of any aetiology) at discharge on
oral rehydration solution use for dehydration (standard
message arm) to the CHoBI7 mHealth program with
either a single in-person visit for health facility delivery
of the program (mHealth with no home visits arm) or
health facility delivery of the program plus two home
visits (mHealth with two home visits arm). The two
home visits were delivered during the one-week high
risk period after the diarrhoea patient was discharged
from the health facility. Diarrhoea patient households in
the mHealth arms were sent weekly IVR, voice and text
messages from the CHoBI7 mHealth program over a
12-month period. A summary text message was sent to
households after voice and IVR messages, in addition to
standalone text messages. Twenty six voice, 14 IVR
quiz, 2 IVR (questions about the condition of interven￾tion hardware) and 48 text messages (12 standalone
text and 36 summary text messages) were sent to
households during the study period. A detailed descrip￾tion of the CHoBI7 mHealth program is published
elsewhere [17].
Implementation findings are presented using the Medi￾cal Research Council framework for process evaluation
[20]. The adaption and the theory of change for delivery
of the CHoBI7 mHealth program is described in detail in
our recent publication George et al. 2019 [17]. The
CHoBI7 mHealth program was developed through a the￾ory-driven approach informed by the Integrated Behav￾ioral Model for Water, Sanitation and Hygiene (IBM￾WASH) and the Risks, Attitudes, Norms, Abilities and
Self-regulation (RANAS) Model [21,22]. Program adap￾tion was conducted based on mobile health workshops,
semi-structured interviews and group discussions fol￾lowed by a pilot study. Our goal was to deliver at least
80% of program text, voice and IVR messages to benefi￾ciary households. Therefore, program fidelity was defined
by at least 80% of program text, voice and IVR messages
being answered/received by beneficiary households. Pro￾gram dose was defined as the percentage of program
voice, text and IVR messages answered/received and fully
listened to by beneficiary households. Program reach was
defined as whether household members (target audience)
986 © 2020 John Wiley & Sons Ltd
Tropical Medicine and International Health volume 25 no 8 pp 985–995 august 2020
Md. S. Islam Bhuyian et al. Evaluation of the delivery of a water, sanitation, and hygiene mobile health program
reported receiving a CHoBI7 program mHealth message
in the past two weeks.
VIAMO platform process evaluation indicators
The following process evaluation indicators were used
to assess the CHoBI7 mHealth program based on data
from the VIAMO platform: (i) the percentage of
CHoBI7 text, voice and IVR messages received (for text
messages) or answered (for voice and IVR) by benefi￾ciary households (program fidelity and dose) (received
by at least one phone in the household); (ii) the percent￾age of CHoBI7 voice and text messages fully listened to
by beneficiary households (program fidelity and dose)
(by at least one phone in the household); and (iii) the
percentage of households replying correctly to IVR quiz
responses (by at least one phone in the household). The
VIAMO platform has two classifications for text mes￾sages, either ‘Finished (received)’ or ‘Network Failed’,
and ‘Failed’ or ‘Finished (answered)’ for voice and IVR
messages. The VIAMO platform can identify whether a
text message is received and whether a voice call is
answered. The percentage of CHoBI7 text messages
received by beneficiary households was calculated by
dividing the total number of unique text messages at
each time point received by the household by the total
number of unique messages sent to the household. For
voice and IVR messages, the duration of the recorded
message listened to (call duration) can be assessed using
the VIAMO platform. If the health promotion content
of the voice or IVR message was completely listened to
by a household, then it was classified as ‘fully listened
to’ (fully completed) otherwise this was classified as
‘partially listen to’. The percentage of CHoBI7 voice
and IVR messages fully listened to by beneficiary house￾holds were calculated by dividing the number of unique
messages at each time point ‘fully listened to’ by the
household by the total number of unique messages sent
to the household. In addition, IVR quiz messages
prompted beneficiaries to select one of two possible
options on questions related to handwashing with soap
and water treatment behaviours, and transmission of
diarrhoeal diseases. If a listener selected a response, this
was classified as ‘replied’ and then assigned to be either
‘correct’ or ‘incorrect’.
Participant reports
Participant reports were used to assess the percentage of
beneficiaries receiving and sharing an mHealth message
from the CHoBI7 mHealth program in the past two
weeks (program reach). Information was also collected
on the percentage of beneficiaries reporting challenges
with the mHealth platform during the program period.
Participant reports were collected from a mHealth
surveillance form administered monthly to enrolled
household members 12 years of age or older. To assess
whether CHoBI7 mHealth messages sent to households
were being received by the primary caregiver, we calcu￾lated the percentage of primary caregivers that reported
receiving a CHoBI7 mHealth message in the past two
weeks at each time point.
Statistical analysis
Logistic and linear regression analyses were performed
with study arm as the predictor and the process evalua￾tion indicator as the outcome to determine whether there
were significant differences in indicators between study
arms. All analyses described above were performed using
STATA software version 13.
Results
Seventeen hundred seventy seven participants from 517
households were enrolled in the CHoBI7 mHealth pro￾gram (264 households in the mHealth with no home vis￾its arm and 253 households in the mHealth with two
home visits arm) (Table 1). The mean age for program
participants was 18 years, and 54% (966/1777) were
female. Ninety four percent (487/517) of households had
at least one person who could read and write. Forty four
percent (230/517) of households reported refrigerator
ownership, and 28% (146/517) reported having a con￾crete roof. Twenty three percent of households (121/517)
were lost to follow-up at the 12-month time point.
Program households received 91% (5425/5987) of
standalone text messages and 92% (16451/17831) of
summary text messages. Sixty percent of households
(311/517) received all standalone text messages sent, and
85% (440/517) of households received over 80% of stan￾dalone text messages sent. Forty nine percent (254/517)
of households received all summary text messages sent,
and 88% (454/517) of households received over 80% of
weekly summary text messages sent. Eighty three percent
(9448/11345) of voice messages sent were fully listened
to. Six percent of households (29/517) answered all voice
messages sent, and 88% (456/517) of households
answered over 80% of voice messages sent. Eighty six
percent (6618/7669) of IVR messages sent to program
households were fully listened to. Fifty nine percent of
households (358/517) answered all IVR messages sent,
and 93% (482/517) of households answered over 80% of
IVR messages sent.
© 2020 John Wiley & Sons Ltd 987
Tropical Medicine and International Health volume 25 no 8 pp 985–995 august 2020
Md. S. Islam Bhuyian et al. Evaluation of the delivery of a water, sanitation, and hygiene mobile health program
There was no significant difference by study arm in the
proportion of program messages received for standalone
or summary text messages or a significant difference by
study arm for IVR messages being fully listened to. The
percentage of voice messages fully listened to was signifi￾cantly higher in the mHealth with two home visits arm
compared to the mHealth with no home visits arm
(P = 0.01, 82% vs. 84%) (Table 2).
Program households replied to 68% (4004/5906) of
unique IVR quiz messages sent, and 81% (3258/4004) of
these quiz responses were correct (Table 3). At Week 3,
when asked whether hands should be washed with soap
after or before eating, 91% (311/340) of households
responded ‘before eating’. At Month 11.75, 83% (219/
264) of participants responded correctly that contami￾nated water and hands spread germs. The only difference
by study arm in the proportion of households that
responded correctly to quiz questions was for the Month
4 program message, where more respondents in the
mHealth with two home visits arm knew that hands and
water could spread germs than the mHealth with no
home visits arm (P = 0.04, 87% vs. 78%).
For the mHealth with no home visits arm, 73% (132/
182) of program households (household-level) reported
receiving a mobile message from the CHoBI7 mHealth
program in the past two weeks at the 6-month time
point, and 78% (150/192) at the 12-month time point
(Figure 1). For the mHealth with two home visits arm,
77% (151/197) of program households reported receiving
a mobile message from the CHoBI7 mHealth program in
the past two weeks at the 6-month time point, and 78%
(157/201) at the 12-month time point. Seventy one per￾cent (350/492) of program participants (participant-level)
at the 6-month time point reported receiving a message
from the CHoBI7 mHealth program in the past two
weeks, and 69% (414/604) at the 12-month time point
(Figure 2). This did not significantly differ by study arm.
Seventy percent (249/357) of caregivers reported receiving
a message from the CHoBI7 mHealth program in the
past two weeks at the 6-month time point, and 67%
(240/357) at the 12-month time point. This also did not
significantly differ by study arm.
Eighty six percent of participants (768/896) reported
sharing CHoBI7 mHealth program mobile messages with
another individual during the 12 month program period.
Seventy seven percent (694/896) of participants reported
sharing CHoBI7 mHealth program messages with
spouses, 55% (494/896) with neighbours, 49% (443/896)
with their children, 24% (215/896) with parents, 21%
(190/896) with siblings, 18% (158/896) with in-laws,
18% (161/896) with friends, and 13% (117/896) with
co-workers over the 12-month program period. There
were no significant differences between study arms for
sharing a message with others (Table 4). Eighty one per￾cent (733/896) of participants reported that someone in
their household shared a CHoBI7 program mobile mes￾sage with them during the 12 month program period.
Seventy four percent of participants (663/896) reported
that a spouse shared a CHoBI7 mHealth message with
them during the program period, 35% (318/896)
Table 1 Baseline population characteristics by study arm
mHealth with
no home visits
arm
mHealth with
two home visits
arm
Study households 264 253
All study participants 886 891
Household members of
index diarrhoea patients
622 638
Number of primary
caregivers
264 253
Number of individuals
12 years of age or older
496 513
Mean study participants
per household
Mean 
 SD
(Min–Max)
3.4 
 0.85
(2–7)
3.5 
 0.93
(2–6)
Mean study participants per household (≥ 12 years)
Mean 
 SD
(Min–Max)
1.89 
 0.65
(1–5)
2.03 
 0.66
(1–5)
Baseline household member age
Mean 
 SD
(Min–Max) (years)
18 
 15
(0.08–80)
18 
 15
(0.08–75)
0–5 years 34% 32%
5–12 years 10% 10%
12–18 years 1% 2%
18 years or greater 55% 56%
Gender
% female 55% 54%
Household roof type
Concrete 28% 28%
Tin 71% 72%
Other <1% 0%
Household wall type
Concrete 71% 71%
Mud 3% 4%
Tin 26% 25%
Other 0% <1%
Household floor type
Concrete 97% 96%
Other 3% 4%
Electricity 93% 92%
Refrigerator ownership 45% 44%
At least one household
member can read and
write
95% 94%
SD, standard deviation.
988 © 2020 John Wiley & Sons Ltd
Tropical Medicine and International Health volume 25 no 8 pp 985–995 august 2020
Md. S. Islam Bhuyian et al. Evaluation of the delivery of a water, sanitation, and hygiene mobile health program
reported a child in their household, 12% (105/896)
reported parents, 7% (60/896) reported siblings, and 7%
(65/896) reported in-laws (Table 5). There were no sig￾nificant differences in having a message shared between
study arms.
Forty nine percent (438/896) of participants reported
at least once during the study period that they were too
busy to receive program messages, 38% (337/896)
reported that they were not able to read a text message,
37% (327/896) reported that mobile messages were not
being shared by the phone’s owner, 34%(307/896)
reported that their phone was damaged, and 17% (148/
896) had no one available to read text messages to them.
There were no significant differences between study arms
for reported challenges (Table 6).
Discussion
The process evaluation of the CHoBI7 mHealth program
demonstrated high fidelity, dose and reach of mobile mes￾sage delivery in both mHealth study arms. Over 80% of
voice and IVR quiz messages were fully listened to, and
90% of text messages were received by program house￾holds. The majority of households responded to IVR quiz
questions, and 81% of their responses were correct. Fur￾thermore, the majority of households (78%) reported
receiving an mHealth message from our program in the
past two weeks at our 12-month follow-up. We also
observed frequent message sharing within households and
with those outside of program households. These findings
demonstrate that the CHoBI7 mHealth program presents
an effective approach to deliver voice, IVR and text mes￾sages to households in slum areas of Dhaka, Bangladesh.
These results complement our recent RCT of the CHoBI7
mHealth program that demonstrated this program was
effective in significantly increasing handwashing with
soap and stored drinking water quality, and reducing
diarrhoea and stunting in young children [19].
A major success of the CHoBI7 mHealth program was
the ability to successfully deliver mHealth messages with
high reach to the primary caregivers in the household.
Phone access for the primary caregiver in the home
emerged as an important challenge during our formative
research [17]. This is consistent with findings from the
Aponjon mHealth program [23]. This mHealth program
delivers text messages to women during pregnancy on
antenatal care in Bangladesh. Female Aponjon subscribers
reported that sometimes program calls were missed
because someone else in the household had the phone,
and that program messages were often not shared with
them. Through tailoring our CHoBI7 mobile messages to
encourage message sharing with all household members
and through emphasising the importance of primary care￾givers receiving our mobile messages during health facil￾ity visits, we were able to overcome this challenge. It is
estimated that 82% of adult males are mobile phone
Table 2 Summary of CHoBI7 mHealth program mobile messages sent to households
Message type Indicator
mHealth with
no Home visits
arm
mHealth with
two home visits
arm
Overall results
for both arms
n % n % n % p-value
Standalone text Sent 3087 -- 2900 -- 5987 --
Failed 307 10% 255 9% 562 9% 0.38
Received 2780 90% 2645 91% 5425 91% 0.71
Summary text Sent 8986 -- 8845 -- 17831 --
Failed 693 8% 687 8% 1380 8% 0.83
Received 8293 92% 8158 92% 16451 92% 0.17
Voice Sent 5684 -- 5661 -- 11345 --
Failed 466 8% 440 8% 906 8% 0.92
Partially Completed 541 10% 450 8% 991 9% 0.06
Fully Completed 4677 82% 4771 84% 9448 83% 0.01
Interactive voice response Sent 3928 -- 3741 -- 7669 --
Failed 418 11% 335 9% 753 10% 0.19
Partially Completed 166 4% 132 4% 298 4% 0.32
Fully Completed 3344 85% 3274 88% 6618 86% 0.32
n is based on the number of unique mobile messages sent to a household. ‘Fully Completed’ is the same as ‘Fully Listen to’ in the
manuscript text. One hundred twenty one households were lost to follow-up at the 12 month time point.
© 2020 John Wiley & Sons Ltd 989
Tropical Medicine and International Health volume 25 no 8 pp 985–995 august 2020
Md. S. Islam Bhuyian et al. Evaluation of the delivery of a water, sanitation, and hygiene mobile health program
Table 3 Summary of CHoBI7 mHealth program interactive voice response quiz questions
IVR Quiz question
mHealth with no home visits arm mHealth with two home visits arm
P-value†
Households
sent messages
(N)
Received
%(n)
Replied
%(n)
Correct
%(n)
Households
sent messages
(N)
Received
%(n)
Replied
%(n)
Correct
%(n)
IVR Quiz Day 3: How long after adding the
chlorine tablet should you wait to drink this
water?
Correct Answer: 30 min
Wrong Answer : 15 min
258 73% (189) 71% (135) 80% (108) 247 81% (201) 73% (147) 84% (123) 0.42
IVR Quiz Week 3: Should you wash your
hands with soapy water after or before
eating?
Correct Answer: Before
Wrong Answer : After
255 91% (231) 73% (169) 91% (153) 242 95% (231) 73% (171) 92% (158) 0.54
IVR Quiz Week 5: How many key times are
there for hand washing with soapy water?
Correct Answer: Four times
Wrong Answer : Three times
258 93% (240) 63% (150) 72% (108) 244 95% (231) 68% (158) 72% (113) 0.93
IVR Quiz Week 7: After heating your water
until a rolling boil, how will you store this
water?
Correct Answer: Keep the water covered
with a lid
Wrong Answer: Keep the water open without
any cover
260 90% (234) 67% (156) 76% (118) 242 96% (232) 66% (154) 82% (127) 0.14
IVR Quiz Month 3.75: How many times do
you need to wash your hands with soapy
water and use safe drinking water in a day?
Correct Answer: Every time in every day
Wrong Answer: At least once in a day
216 94% (202) 60% (122) 88% (107) 212 91% (192) 71% ( 137) 91% (125) 0.36
IVR Quiz Month 4: How can diarrhea germs
spread?
Correct Answer: Germs spread by hands and
through water
Wrong Answer: Germs spread by air
260 92% (239) 74% (178) 78% (139) 245 91% (222) 74% (165) 87% (143) 0.04
IVR Quiz Month 5: How often do I need to
clean my buckets?
Correct Answer: Every week
Wrong Answer : Every day
253 92% (232) 61% (141) 48% (67) 241 92% (221) 58% (128) 48% (62) 0.88
990 © 2020 John Wiley & Sons Ltd
Tropical Medicine and International Health volume 25 no 8 pp 985–995 august 2020
Md. S. Islam Bhuyian et al. Evaluation of the delivery of a water, sanitation, and hygiene mobile health program
Table 3 (Continued)
IVR Quiz question
mHealth with no home visits arm mHealth with two home visits arm
P-value†
Households
sent messages
(N)
Received
%(n)
Replied
%(n)
Correct
%(n)
Households
sent messages
(N)
Received
%(n)
Replied
%(n)
Correct
%(n)
IVR Quiz Month 6 : How many days is the
high risk period for becoming sick after
someone in your home has severe diarrhea?
Correct Answer: 7 days
Wrong Answer : 14 days
258 88% (228) 70% (160) 80% (128) 240 90% (217) 74% (161) 74% (119) 0.20
IVR Quiz Month 6.75: When do you have to
wash your hands with soap, Before or after
cutting cucumbers?
Correct Answer: Before cutting cucumbers
Wrong Answer: After cutting cucumbers
246 89% (216) 63% (136) 90% (123) 220 91% (200) 67% (134) 93% (124) 0.54
IVR Quiz Month 7.75: What do you need to
do after washing both hands with soapy
water?
Correct Answer: You should use a clean
cloth
Wrong Answer: You should dry your hands
on your clothing
255 87% (222) 68% (151) 85% (120) 244 90% (219) 63% (139) 88% (122) 0.46
IVR Quiz Month 8.75: How do you dispense
water from your blue bucket for drinking?
Correct Answer: By using the tap on the blue
bucket
Wrong Answer: By dipping a mug or glass in
the blue bucket
255 84% (215) 66% (142) 85% (120) 248 88% (218) 71% (154) 90% (138) 0.19
IVR Quiz Month 10.75: How should Aklima
boil her water to keep her family safe from
severe diarrhea?
Correct Answer: She should heat her water
until it reaches a rolling boil
Wrong Answer: She should heat her water
until it becomes hot
254 85% (216) 67% (145) 83% (121) 248 85% (212) 70% (148) 84% (125) 0.81
IVR Quiz Month 11.75: How are the germs
that cause severe diarrhea spread?
Correct Answer: Germs are spread by
contaminated water and hands
Wrong Answer: Germs are spread from bad
air
251 84% (211) 65% (138) 85% (117) 232 85% (198) 64% (126) 81% (102) 0.41
N and n values are based on the number of households. †
P-value for percent of correct IVR quiz responses compared by study arm.
© 2020 John Wiley & Sons Ltd 991
Tropical Medicine and International Health volume 25 no 8 pp 985–995 august 2020
Md. S. Islam Bhuyian et al. Evaluation of the delivery of a water, sanitation, and hygiene mobile health program
owners vs. only 55% of adult females in Bangladesh
[24]. Therefore, it is important that future mHealth pro￾grams targeting child health ensure that mobile messages
are reaching female caregivers in the household.
Another successful aspect of the CHoBI7 mHealth pro￾gram was the high number of beneficiaries that fully lis￾tened to IVR quiz messages and responded correctly.
Furthermore, this did not significantly differ by mHealth
arm. This finding suggests that home visits are not needed
to ensure households correctly answer program quiz
questions. The IVR quiz messages present a novel
approach to assess awareness on key program behaviours
and to tailor program messages accordingly. For exam￾ple, we learned through our IVR quiz messages during
the pilot study that households were unclear on the four
key times for handwashing with soap. This led us to fur￾ther emphasise these key times during our RCT. We also
learned during the pilot that beneficiaries often took
quizzes together as a family in the evenings [17]. This
finding suggested that IVR quizzes could serve as an
approach to engage several household members at once
in mobile message content. This is the first study to our
87%
77% 74% 74%
81%
73% 72% 71%
78% 77% 78% 78%
91%
85%
79% 81% 78% 77%
71%
83%
79% 82% 79% 78%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Month 1 Month 2 Month 3 Month 4 Month 5 Month 6 Month 7 Month 8 Month 9 Month 10 Month 11 Month 12
mHealth with no home visits arm mHealth with two home visits arm
Households reporting receiving a 
CHoBI7 mHealth message in the past 2 weeks
Study month
Nn = 194
n = 200
n = 179
n = 187
n = 174
n = 171
n = 182
n = 169
n = 170
n = 181
n = 159
n = 169
n = 192
n = 171
n = 189
n = 175
n = 176
n = 197
n = 175
n = 172
n = 186
n = 178
n = 188
n = 201
Figure 1 Households reporting receiving a CHoBI7 mHealth message in the past 2 weeks over the study period.
85%
77% 74% 75% 75% 71%
67% 70% 73% 73% 71% 69%
88%
79%
74% 74% 74%
70%
64% 68% 71% 71% 68% 67%
0%
10%
20%
30%
40%
50%
70%
60%
80%
90%
100%
Month 1 Month 2 Month 3 Month 4 Month 5 Month 6 Month 7 Month 8 Month 9 Month 10 Month 11 Month 12
Participant report Caregiver report
Participants reporting receiving CHoBI7 
a mHealth message in the past 2 weeks
Study month
Nn = 483
n = 370
n = 426
n = 442
n = 476
n = 492
n = 513
n = 484
n = 546
n = 481
n = 604
n = 317
n = 357
n = 419
n = 330
n = 448
n = 349
n = 325
n = 322
n = 357
n = 315
n = 306
n = 323
n = 298
Figure 2 Participants and caregivers reporting receiving a CHoBI7 mHealth messages in the past 2 weeks over the study period.
992 © 2020 John Wiley & Sons Ltd
Tropical Medicine and International Health volume 25 no 8 pp 985–995 august 2020
Md. S. Islam Bhuyian et al. Evaluation of the delivery of a water, sanitation, and hygiene mobile health program
knowledge to use IVR quizzes to engage beneficiaries in a
WASH program. Two-way text messaging has been asso￾ciated with improved medication adherence practices
compared to one-way texting; however, the literature on
the impact of voice IVR messages is limited [25]. Our
finding that 86% of IVR messages were fully listened to
by beneficiaries is consistent with a previous study in
Senegal that used IVR educational questions as part of a
mobile learning program on management of contracep￾tive side effects and misconceptions among mid-wives
and nurses. This study found that 80% of these IVR mes￾sages were received by beneficiaries [26]. Future studies
should evaluate the effectiveness of IVR quiz messages in
encouraging other WASH behaviours.
Through our process evaluation, we identified key chal￾lenges to CHoBI7 mHealth program implementation. A
common challenge was not being able to read text mes￾sages. However, it is important to note that only 17% of
beneficiaries reported that no one in the household could
read a text message for them. In addition, over 94% of
program households report at least one person in the
household that could read and write. Therefore, most
beneficiaries were able to receive the content of text mes￾sages, even if they could not read the messages them￾selves. The literacy rate in Bangladesh for females over
15 years of age is 70% and 76% for males [27]. This
finding highlights the importance of including voice mes￾sages in mHealth programs. Most mHealth programs in
LMICs employ either text or voice messages; rarely, are
both used [28]. Another challenge was beneficiaries being
too busy to receive program messages at 49%. We tried
to overcome this challenge by sending messages in the
Table 4 Participants reporting sharing a CHoBI7 mHealth mes￾sage with others over the study period (Months 1–12) (N = 896)
Shared with
mHealth with no
home visits arm
%(n)
mHealth with two
home visits arm
%(n) P-value*
Any sharing 85% (367) 87% (401) 0.43
Spouse 76% (328) 79% (366) 0.24
Neighbour 54% (232) 57% (262) 0.37
Children 51% (220) 48% (223) 0.43
Parents 23% (100) 25% (115) 0.54
Sibling 20% (88) 22% (102) 0.54
In-Law 19% (81) 17% (77) 0.42
Friends 19% (83) 17% (78) 0.37
Co-Worker 12% (54) 14% (63) 0.61
*P-value is a comparison of proportions by study arm.
Table 5 Participants reporting someone shared a CHoBI7
mHealth message with them over the study period (Months 1–
12) (N = 896)
Someone
shared
mHealth with no
home visits arm
%(n)
mHealth with two
home visits arm
%(n) P-value*
Any sharing 79% (344) 84% (389) 0.08
Spouse 73% (317) 75% (346) 0.60
Children 36% (156) 35% (162) 0.75
Parents 11% (46) 13% (59) 0.33
Sibling 6% (25) 8% (35) 0.29
In-Law 7% (30) 8% (35) 0.72
*P-value is a comparison of proportions by study arm.
Table 6 Reported challenges with CHoBI7 mHealth program
over the study period (Months 1–12) (N=896)
Challenge
mHealth with
no home visits
arm %(n)
mHealth with
two home
visits arm %(n) P-value*
Participants were
too busy to
receive mobile
messages
50% (216) 48% (222) 0.56
Not being able to
read text messages
37% (162) 38% (175) 0.91
Mobile messages
not being shared
by phone owner
38% (163) 35% (164) 0.49
Damage to
household phone
35% (153) 33% (154) 0.51
No one available in
the household to
read text messages
for them
18% (78) 15% (70) 0.24
Poor cell phone
reception made
voice calls hard to
hear
14% (61) 10% (46) 0.06
Accidentally
deleted text
messages before
reading it
3% (13) 3% (13) 0.86
Full inbox blocked
incoming messages
3% (14) 2% (10) 0.32
Accidentally hung
up on voice calls
before message
finished
5% (21) 3% (14) 0.16
Distracting/noisy
background made
it hard to hear the
voice calls
6% (25) 6% (29) 0.76
*P-value is a comparison of proportions by study arm.
© 2020 John Wiley & Sons Ltd 993
Tropical Medicine and International Health volume 25 no 8 pp 985–995 august 2020
Md. S. Islam Bhuyian et al. Evaluation of the delivery of a water, sanitation, and hygiene mobile health program
evening when most household members were home [17].
Our approach is consistent with an mHealth program in
Senegal that found that users accessed the program the
most in the evening [29]. Future studies may consider
evaluating the effectiveness of sending text and voice
messages based on the time window requested for each
individual household.
Mobile message sharing was high among beneficiaries
with the majority reporting sharing CHoBI7 mHealth
messages with spouses, neighbours, children, parents, sib￾lings, in-laws, friends or co-workers. This finding is con￾sistent with our formative research which found that
participants valued the content of program messages and
thought that it was important to share the content of
these messages with others to help to improve their
health [17]. Additional work is needed to identify the
spillover effects of the CHoBI7 mHealth program to
other households.
This study has some limitations. First, the study was
conducted only in Dhaka and therefore cannot be gener￾alised to rural areas in Bangladesh. Second, we focused
our study on diarrhoea patient households, so these find￾ings cannot be generalised to other populations. Third,
we focused on households that reported phone owner￾ship. Future studies should include those households that
have shared access to phones.
This study has several strengths. First, the monthly
mHealth surveillance allowed us to determine whether
participants were receiving program mHealth messages
and the challenges they were encountering. Second, the
use of the process evaluation indicators developed from
the VIAMO platform allowed us to determine whether
program mobile messages were being received, answered
and fully listened to, and whether quiz IVR messages
were answered correctly. Third, the 12-month duration
of the study allowed us to observe trends in fidelity indi￾cators over time.
Conclusion
The CHoBI7 mHealth program was implemented with
high fidelity, dose and reach. Through process evaluation,
we identified robust indicators that can be used to track
the progress of mHealth programs. This included the per￾centage of voice and IVR messages fully listened to and
the percentage of voice, text and IVR messages received
and answered as a measure of program dose and fidelity,
and the number of household members that received and
shared a program message as a measure of program
reach. This study presents an approach for process evalu￾ation that can be implemented to evaluate future
mHealth programs globally.
Acknowledgements
This research was supported by a USAID grant awarded
to Johns Hopkins School of Public Health. We thank
USAID for their support. We thank the study participants
and the following individuals for their support with the
implementation of this study: Professor Abul Khair
Mohammad Shamsuzzaman, Professor Be-Nazir Ahmed,
Fosiul Alam Nizame, Khobair Hossain, Jahed Masud,
Ismat Minhaj Uddin, Rafiqul Islam, Maynul Hasan, SM.
Arifur Rahman, Abdullah Al Morshed, Zakir Hossain,
Kabir Hossain, Amal Sarker, Abul Bashar Sikder, Abdul
Matin, Sadia Afrin Ananya, Lubna Tani, Farhana Ahmed,
Tahera Taznen, Marufa Akter, Akhi Sultana, Nasrin
Akter, Laki Das, Abdul Karim, Shirin Akter, Khan Ali
Afsar and Wasim Ahmed Asif. We also thank hospital
staff for their support. icddr,b acknowledges the govern￾ments of Bangladesh, Canada, Sweden, and United King￾dom, for providing core/unrestricted support.
References
1. Collaborators GDD. Estimates of global, regional, and
national morbidity, mortality, and aetiologies of diarrhoeal
diseases: a systematic analysis for the Global Burden of Dis￾ease Study 2015 (vol 17, pg 909, 2017). Lancet Infect Dis
2017; 17: 897.
2. Esrey SA. Water, waste, and well-being: a multicountry
study. Am J Epidemiol 1996: 143: 608–623.
3. Saha D. Acute diarrhoea in children in rural Gambia:
Knowledge, attitude and practice, aetiology, risk factors and
consequences among children less than five years of age:
University of Otago; 2013.
4. George CM, Perin J, Neiswender de Calani KJ et al. Risk
factors for diarrhea in children under five years of age resid￾ing in peri-urban communities in Cochabamba, Bolivia. Am
J Trop Med Hyg 2014: 91: 1190–1196.
5. Ejemot RI, Ehiri JE, Meremikwu MM, Critchley JA.
Cochrane review: Hand washing for preventing diarrhoea.
Evid Based Child Health Cochrane Rev J 2009: 4: 893–939.
6. Luby SP, Agboatwalla M, Bowen A, Kenah E, Sharker Y,
Hoekstra RM. Difficulties in maintaining improved hand￾washing behavior, Karachi, Pakistan. Am J Trop Med Hyg
2009: 81: 140–145.
7. Yang QH, Van Stee SK. The comparative effectiveness of
mobile phone interventions in improving health outcomes:
meta-analytic review. JMIR Mhealth Uhealth 2019: 7:
e11244.
8. Shin JC, Kim J, Grigsby-Toussaint D. Mobile phone inter￾ventions for sleep disorders and sleep quality: systematic
review. JMIR Mhealth Uhealth 2017: 5: e131.
9. Bassi A, John O, Praveen D, Maulik PK, Panda R, Jha V.
Current status and future directions of mHealth Interven￾tions for health system strengthening in India: systematic
review. JMIR Mhealth Uhealth 2018: 6: e11440.
994 © 2020 John Wiley & Sons Ltd
Tropical Medicine and International Health volume 25 no 8 pp 985–995 august 2020
Md. S. Islam Bhuyian et al. Evaluation of the delivery of a water, sanitation, and hygiene mobile health program
10. Free C, Phillips G, Galli L et al. The effectiveness of
mobile-health technology-based health behaviour change or
disease management interventions for health care con￾sumers: a systematic review.PLoS Medicine 2013: 10:
e1001362.
11. ICT. Key ICT indicators for developed and developing coun￾tries and the world (totals and penetration rates) 2017
(Available from: https://www.itu.int/en/ITU-D/Statistics/Page
s/stat/default.aspx).
12. Gibson DG, Pariyo GW, Wosu AC et al. Evaluation of
mechanisms to improve performance of mobile phone sur￾veys in low- and middle-income countries: research protocol.
JMIR Res Protoc 2017: 6: e81.
13. Available from: http://www.btrc.gov.bd/content/mobile￾phone-subscribers-bangladesh-april-2019.
14. LeFevre A, Cabrera-Escobar MA, Mohan D et al. Forecast￾ing the value for money of mobile maternal health informa￾tion messages on improving utilization of maternal and
child health services in Gauteng, South Africa: cost￾effectiveness analysis. JMIR Mhealth Uhealth 2018: 6: e152.
15. Henry CA. Using Mhealth to promote hand washing with
soap: how do Tanzanian youth perceive text message interven￾tions for hand hygiene? Am J Trop Med Hyg 2017: 97: 280.
16. Tidwell JB, Gopalakrishnan A, Lovelady S et al. Effect of
two complementary mass-scale media interventions on hand￾washing with soap among mothers. J Health Commun.
2019: 24: 203–215.
17. George CM, Zohura F, Teman A et al. Formative research
for the design of a scalable water, sanitation, and hygiene
mobile health program: CHoBI7 mobile health program.
BMC Public Health 2019: 19: 1028.
18. Available from: www.viamo.io
19. George CM, Monira S, Zohura F et al. Effects of a water,
sanitation and hygiene mobile health program on diarrhea
and child growth in Bangladesh: A cluster-randomized con￾trolled trial of the CHoBI7 mobile health program. Clinical
Infectious Diseases 2020: in press.
20. Moore GF, Audrey S, Barker M et al. Process evaluation of
complex interventions: Medical Research Council guidance.
BMJ 2015: 350: h1258.
21. Dreibelbis R, Winch PJ, Leontsini E et al. The integrated
behavioural model for water, sanitation, and hygiene: a sys￾tematic review of behavioural models and a framework for
designing and evaluating behaviour change interventions in
infrastructure-restricted settings. BMC Public Health 2013:
13: 1015.
22. Mosler H-J. A systematic approach to behavior change
interventions for the water and sanitation sector in develop￾ing countries: a conceptual model, a review, and a guideline.
Int J Environ Health Res 2012: 22: 431–449.
23. Rajan R, Raihan A, Alam Met al. MAMA ‘Apon￾jon’formative research report. 2013.
24. Rowntree O. The Mobile Gender Gap Report 2018. GSMA,
(Available from https://www.gsma.com/mobilefordevelopme
nt/programmes/connected-women/the-mobile-gender-gap￾report-2018.) 2018.
25. Wald DS, Butt S, Bestwick JP. One-way versus two-way text
messaging on improving medication adherence: meta-analy￾sis of randomized trials. Am J Med 2015: 128: 1139.
26. Diedhiou A, Gilroy KE, Cox CM et al. Successful mlearning
pilot in senegal: delivering family planning refresher training
using interactive voice response and SMS. Glob Health Sci
Pract 2015: 3: 305–321.
27. Bangladesh UNESCO.
28. Higgs ES, Goldberg AB, Labrique AB et al. Understanding
the role of mHealth and other media interventions for
behavior change to enhance child survival and development
in low- and middle-income countries: an evidence review. J
Health Commun 2014: 19(Suppl 1): 164–189.
29. Diedhiou A, Gilroy KE, Cox CM et al. Successful mLearn￾ing pilot in senegal: delivering family planning refresher
training using interactive voice response and SMS. Glob
Health 2015: 3: 305–321.
Corresponding Author Christine Marie George, Associate Professor, Associate Professor, Department of International Health,
Program in Global Disease Epidemiology and Control, Johns Hopkins Bloomberg School of Public Health, 615 N. Wolfe Street,
Room E5535, Baltimore, MD 21205-2103, USA. Tel.: +1 410 955 2485; E-mail: cmgeorge@jhu.edu
© 2020 John Wiley & Sons Ltd 995
Tropical Medicine and International Health volume 25 no 8 pp 985–995 august 2020
Md. S. Islam Bhuyian et al. Evaluation of the delivery of a water, sanitation, and hygiene mobile health program