INTRODUCTION
eHealth can help health-service users take a more active role in decision-making and help health professionals guide the patient in this process1. It can also contribute, at the organizational level, to a more patient-centered model of healthcare2. The World Health Organization has highlighted the relevance of digital health interventions to address health needs, although they stress that these tools should always be used as an aid and improvement for health systems, not as a substitute3. Currently, most digital healthcare tools (eHealth) are accessible from mobile devices such as smartphones or tablets (mHealth).
Pregnancy and postpartum are periods of great vulnerability in which sensitivity to information and self-care is heightened. Midwives respond to this need with woman-centered health education programs, focusing not only on pregnancy care but also on women’s well-being and satisfaction, providing information, emotional support, and resources to cope with childbirth and parenting both individually and in groups4. However, access to this information and resources is limited to appointment times or group activities, while women’s needs occur continuously, with unexpected variations. Digital technology can increase the reach and effectiveness of this work by providing continuous, immediate, and personalized support5,6. Indeed, young women from differing socioeconomic backgrounds consistently use digital content as a source of information about their health, and the information they consult is positively correlated with the decisions they make about their health7. However, according to the WHO, the enthusiasm for digital health has also driven a proliferation and overwhelming diversity of digital tools, although there is still limited understanding of their impact on health systems and people’s well-being3. Some mHealth apps are not evidence-based, prioritize diseasecentered over person-centered approaches, have limited usability, and present serious concerns regarding user privacy3,5,7,8. According to the WHO, reviewing the needs of health systems and moderating expectations based on Information and Communication Technologies (ICTs) is of considerable value3.
The EMAeHealth (eHealth Maternal Education) application was designed as a complement to Maternal Education (ME) and is organized into four areas: information, communication, self-management of health, and clinical data9. Although the information area is freely accessible, entry into the self-management and clinical data areas requires authentication by the user. It allows for the assessment of perinatal psychosocial needs and promotes self-management of health and well-being in the postnatal period, proposing plans to change eating habits, exercise, behavior, and decisions. The app is being studied, both in terms of implementation and clinical efficacy, using a hybrid efficacy-implementation design9,10.
The key factors for the successful implementation of e-health applications are usability11, technological performance, and user participation3. However, few digital health apps publish the results of their usability assessment11. Usability is generally measured in terms of aspects such as user ratings of the application’s flexibility, operability, comprehensibility, ease of learning, efficiency, satisfaction, attractiveness, consistency, and error rates12. In recent years, numerous scales have emerged to measure the quality of health apps, quantitatively assessing areas such as engagement, functionality, aesthetics, information quality, potential for behavioral change11,12, and providing an overall measure of usability. However, they do not identify the user perceptions of the tool’s effectiveness, their satisfaction, their preferences, or the shortcomings they identify, problems that need to be addressed; in this sense, qualitative methods may be more useful3,11.
The aim of this study is to explore users’ perceptions of the EMAeHealth digital app about its acceptance, usability, strengths, and weaknesses for its implementation in real-world conditions.
METHODS
Design
This is an exploratory sequential mixed-methods research study13. This study design was chosen for its appropriateness in the research of health service interventions, as the qualitative methods allow for the exploration of unexpected issues while the quantitative methods address the limited generalizability of the former. In this study, both methods were used with a developmental perspective, in the sense that qualitative results informed the content of the questionnaire used for quantitative data collection. Initially, the qualitative study was carried out to discover whether an app like this could offer any advantages to already existing in-person antenatal education programs and other pregnancy apps. After this qualitative study, the quantitative study was designed based on the main findings of the qualitative study, to assess the agreement of a greater sample of users with the experiences reported by participants in the qualitative study in relation to app use and potential improvements13.
Participants
This study is part of a comprehensive research project in which the perceptions and needs of women during pregnancy, childbirth and postpartum have been analyzed, as well as the resources available to them to adapt to each moment of the process9. Eligible participants for this study were all pregnant Spanish-speaking women who had seen their midwives at the start of gestation and for 1 year after the implementation of the EMAeHealth app. For the interviews, 39 women who had used it were invited by phone following a purposive sampling. Saturation was reached after the tenth interview. The subsequent survey was launched online to 400 women who had been part of the study, collecting responses anonymously.
Data collection
The qualitative results were collected through individual semi-structured interviews, recorded, and then professionally transcribed. Data collection took place between January and March 2024 using individual semi-structured interviews. All the participating women received an information sheet about the study beforehand and signed an informed consent form. Verbal consent was also obtained for recording the interviews, which were conducted via video call. The mean duration of the interviews was about 16 minutes, with a range 13–28 minutes. The interview guide included questions related to the use of the app, some aspects of usability, comparison with other apps for pregnancy, and recommendations for improvement.
The quantitative results were collected in December 2024 through an anonymous online survey created ad hoc, based on the results of the qualitative study and therefore including questions that were part of aspects raised by the participants in the interviews. A structured questionnaire was designed, with a filter question to divide the respondents into those who had used the app and those who had not, and who were asked for the reasons for not using it. For those who had used it, the questionnaire consisted of 7 items, including Likert scale questions (1=strongly disagree to 5=strongly agree) and closed-ended questions, covering aspects of usability, along with questions to assess agreement with the improvement recommendations given by participants in the qualitative study. All the women who had activated the tool were invited to answer the questionnaire, without differentiating between pregnant women and women who had already given birth.
Data analysis
The transcriptions were analyzed following qualitative content analysis as described by Graneheim et al.14 with the support of Atlas.ti software for the coding process and the retrieval of quotations. AM and IE read all the transcriptions many times to familiarize themselves with the data, and each interview was independently double-coded by each of them to capture all the relevant information. The coding process was deductive, as identified and coded meaning units were interview extracts that were related to the questions stated in the objective, i.e. acceptance, usability, strengths, and weaknesses of the app being evaluated. The same two authors who drew up the codes carried out a preliminary sorting of codes into categories that encompassed the core meaning of the codes included in each of them. This preliminary sorting was later discussed by all the authors and modified after agreements were reached until the final version of the results described in the following section was approved by all the authors.
Some measures were implemented in this study to enhance trustworthiness. First, to achieve credibility, a relationship of trust was established with the participants by providing detailed information about data management and the measures taken to ensure confidentiality. The accuracy of the interview transcripts was ensured by hiring a professional service for this purpose, and the authors who did the coding double-checked the accuracy of the transcriptions by listening to the audios. Triangulation of the results by the five researchers improved credibility. Three of the researchers had an insider perspective of the app being assessed as they had been part of the development and testing process from the beginning, while the other two authors were not directly involved in the whole process. These different positions enriched the credibility by combining these two different perspectives. In terms of confirmability, representative quotations from participants have been included in the Results section. Furthermore, in terms of transferability, participants’ relevant characteristics have been summarized to facilitate readers’ assessment of the possible application of these results to other settings/ participants.
The analysis of the quantitative data was carried out using descriptive statistical methods, i.e. frequency and percentage, and mean with standard deviation (SD) for Likert scale responses using SPSS for Windows version 26.
RESULTS
The acceptance rate of the EMAeHealth digital app was 64%. The characteristics of the women who used the EMAeHealth app (n=254) and interviewees (n=10) are presented in Table 1. For the qualitative study, 10 interviews were carried out. In the quantitative section, 106 out of 400 women responded to the survey (response rate: 26.5%).
Table 1
Sociodemographic characteristics of women who used the EMAeHealth app (N=254) and interviewees (N=10) in the mixed-methods study conducted in the Basque Country, Spain, January–December 2024
Qualitative results
Two categories were created from the qualitative analysis. The first, ‘What makes this app stand out’, compiles all the strengths participants saw in this app, both in comparison to in-person antenatal health education programs and in relation to other available apps. The second, ‘remaining potential’, describes potential improvements that would make the app better and enhance its use.
What makes this app stand out
Interviewees overall assessed the app very positively. They considered it very intuitive and easy to manage and appreciated the inclusion of questionnaires about nutrition, physical activity, and emotional status. Accessibility, especially compared to in-person health education sessions that require asking for leave from work or, in some cases, the need to change work shifts, was one of the main benefits highlighted:
‘I think having it somewhere online is more convenient than a talk or something, because … You can look at it whenever you want, without having to miss work or change shifts.’ (P2)
Access to the app required the use of a digital certificate that is commonly used for access to different public administration services in the region. As reflected in the following quotation, although this requirement was generally considered an easy identification system, some participants mentioned that it might be a barrier for some women to use the app:
In relation to more specific features, quantity, quality, and good organization of the information, referring to the ease of finding specific information overall, and related to the different stages of the pregnancy were among the reasons for the positive valuation. More specifically, the credibility of the source of the information – healthcare professionals – was a remarkable strength of this app in comparison to other sources of information:
‘I really did like it because it is sort of organized in stages. When you want to get pregnant, then when you get pregnant, the birth, the postpartum, then the issue of nutrition … so I think it’s good … in fact, it is well explained.’ (P8)
‘You have so much information in so many places that you don’t know which one is the right one, for sure, and this is a way in the end, since there are professionals behind it, and you can see that this is the right one.’ (P9)
Finally, in relation to the content of the information provided, one participant highlighted the non-judgmental approach when providing information about the different options when feeding a newborn as very positive, and commented on how this respectful approach helped her feel supported:
‘It’s not like in other places where they present formula milk as something horrible and, ugh, what are you doing … You are going to kill your child. Here, I mean in the app, I did not feel that way, so, from that point of view, I am grateful because in the end, it’s a decision that I made that was difficult, but I decided it, and I would like to feel supported. With the app, I felt supported.’ (P8)
It is worth noting that there were some differences in the usefulness perceived depending on whether the current pregnancy was the first or subsequent, and it was more highly valued by those in their first pregnancy. Similarly, those in the first pregnancy stated they had made more use of the app than those in subsequent pregnancies and were eager to have more information. For those women who were pregnant while taking care of other children, a lack of time was a repeated barrier to making use of this or other apps:
‘It would have been useful in the first pregnancy, as in the end, in the second [pregnancy], it seems that you’ve already lived through the experience, so you don’t look at it as much.’ (P2)
Remaining potential
The main potential improvement suggested for the app was linked to its origin and ownership being part of the regional public healthcare system. Therefore, participants considered that the app had the potential to become essential if new functions related to healthcare provision were added by linking it to individual health records. Participants believed a huge difference with other existing pregnancy apps would be the incorporation into the app of all the tests each woman had to undergo during the pregnancy, with the date, reminders of the appointment, aim of the test, explanation of the procedure, results, and implications of the results:
‘Maybe it would be good to have more explanations when it comes to what type of tests they are going to do on me or what they consist of, because really, you haven’t a clue in the end.’ (P10)
The second main recommendation arose from other apps women knew and was related to making the app more individualized, for example by including notifications for each week of the pregnancy, providing information about the development of the baby in that week, healthy lifestyle recommendations for that specific moment, and individualized support and guidance for the results from the questionnaires in the app about nutrition, physical activity and emotional status:
‘[The app] could include the week you have reached, a bit of the initial information, and every week you could get a notification of a weekly news item, how your body is changing, how the baby is growing, all those things … And if you add it to a plan that you already have, well, in the end, it is quite complementary.’ (P4)
In any case, participants perceived that none of the possible improvements would completely counterbalance the lack of opportunity to share experiences with other women in the same situation and to develop networks that can offer support and companionship throughout the pregnancy and after the birth that face-to-face antenatal education provides. Yet, they believed this could be mitigated to some extent by having something similar to a chat box in the app, managed by a professional who knows who you are, where specific doubts would be answered, even if not at the moment, during the day, for example:
‘If you have a question or you are feeling down or you have a doubt, there should be a kind of chat box, someone who is on the other screen, who reads your messages, not on the same screen, but well, when they can, but they know that you are Mary, that you are 30, that you are pregnant, that you are in such-and-such a week and they know a little about who you are.’ (P3)
Finally, in comparison with other popular apps for pregnancy that participants were using, the improvement recommendations included adding some information for the partners of pregnant women, making the app more visually attractive by adding more images related to the information given, and taking into account different pregnancies, like multiple or in vitro fertilization processes:
‘Images of babies, of their size, weight … what it’s like inside the amniotic fluid; you see all of that in other apps, and that’s missing here, that’s what I think it lacks.’ (P7)
‘Or, for example, it [another app] also differentiates if it’s a twin birth because, in the end, that is also different, and it has a little section for the partner, so that they know what’s happening week by week.’ (P4)
Quantitative results
In relation to the survey, of the 106 women surveyed, 64 used the app. The main reasons for not using the app were difficulty of access, lack of time, and already using a better-liked app.
In terms of usability, the app’s aesthetics were rated as 3.70 out of 5. The majority of respondents would add more content, i.e. sections, functions, and/or questionnaires for the self-management of health, but overall, they think that the current length of the topics is fine. Those who had filled in a questionnaire for the self-management of health (86%) found it useful to reflect or make changes (rated 3.33/5), and for those who had decided to start a change plan (80%), the app was useful for them in the process (rated 2.79/5).
Finally, the improvement recommendations given by those interviewed that had the strongest agreement were: ‘That it could be more personalized (weeks of gestation, changes made …)’, ‘Include a chat box to resolve doubts with my midwife’ and ‘Connect the app to the official Health File to see appointments, test results, etc.’ with ratings of 4.38 ± 0.83, 4.28 ± 1.00 and 4.16 ± 1.10 out of 5 points, respectively (Table 2).
Table 2
Results of the ad hoc questionnaire from the mixed-methods study conducted in the Basque Country, Spain, January-December 2024, in relation to the EMAeHEALTH app use (n=106) and its usability, aesthetics and improvement suggestions according to users’ experience (n=64)
DISCUSSION
Pregnant women belong to a group that is particularly willing to seek information and advice at this stage15-17. In our study, 64% of women use EMAeHealth, a slightly lower percentage than that seen in other studies, 75%17 or 77%16. In our healthcare system, government applications are less widespread than in other countries in Northern Europe18,19. One possible interpretation for this difference is the reluctance of healthcare professionals who perhaps do not feel sufficiently confident about the information and security offered by these tools and do not recommend them or recommend them with reservations18,20.
The results of this study show that women, especially first-time mothers, appreciate the immediacy and availability of this digital tool in their daily lives, its clear and non-judgmental presentation, and the credibility lent by having been developed by midwives. However, they would like the information to be fully personalized, based on gestational age, the progress of their pregnancy, and their responses to questionnaires about diet and exercise. They also miss peer interaction through the app or, at least, direct contact with a midwife. These impressions are confirmed by the quantitative study, which rated the content and format of the app’s information section at 3.7/5. However, in the questions about completing questionnaires and self-managing health, criticism already expressed in the interviews regarding the lack of personalized responses reappeared. Although they found the questionnaires useful and rated them 3.3/5, these surveys did not use them to help change behavior, such as diet or exercise (2.79/5). Undoubtedly, behavioral change is affected by factors that go beyond knowledge or even the person’s motivation, such as the social context4. From this perspective, the application could help raise awareness about possible change, but closer intervention and follow-up would be needed to modify behavior. In line with the interviews, the survey of more than 100 women confirmed the demand for personalized responses (4.38/5) and peer interaction (4.28/5). The results are consistent with other studies conducted in similar settings16-18,21. Women who used the app particularly valued the permanent accessibility of information15 and the credibility that comes with being hosted by a public health system20. On commercial websites and apps related to pregnancy, the variability of information and lack of accuracy cause women anxiety and frustration16,22,23.
The EMAeHealth app scored highly in terms of ease of download and access, but it was difficult to locate the desired information. The content was considered reasonably good, although 22% thought it should have more features and perhaps some topics aimed at partners. These drawbacks may justify less use of the resources offered by the app (such as starting a plan to change diet or exercise). Some studies show that minimal technical difficulties determine both the acquisition and use of an app16,21,22. The intuitive use of the application is also a determining factor for participation, especially when the user is accustomed to the aesthetics of social media22,24,25. In our opinion, health authorities can do little to compete with the power of large commercial or technology companies, which are capable of continuously improving the design and interoperability of digital tools26. Commercial applications offer automated, intuitive features, and very simple language complemented by graphics and gamification17,18,27. The public system can and should strive to offer some of these resources, but there will always be delays and limitations caused by control systems. Fortunately, this does not mean that their influence on health behavior will necessarily be less. In a study in which 50 young adults were asked about the importance they attached to online health information, Briones et al.28 found that participants were critical of the online medium, using it for entertainment, as a means of finding contacts, or as a source of health information, depending on the credibility they attributed to it.
The protection of privacy and the implementation of barriers that prevent strangers from accessing clinical data are mandatory requirements in the public health system that women perceive as a difficulty19,21, but which are also highly valued by users15. This tension between the need for privacy and the importance of receiving information relevant to each individual constitutes the ‘personalization–privacy paradox’ and, as Cloarec et al.29 show, it is frequently resolved with a preference for personalization over privacy when the information is perceived to be of interest. Specifically, in our study, women felt that the app should use their relationship with individual clinical data to offer personalized information, appointments, notifications, and self-care advice tailored to their stage of pregnancy or specific condition. The lack of alignment between what the tool offers and women’s specific needs causes disengagement23, while reminder messages and information tailored to the personal situation improved engagement with the digital tool24,25.
In line with previous studies, women perceived a lack of personal connection with professionals and support groups17,25. Although the use of apps improved self-care in relation to weight, blood pressure, or blood glucose30, they preferred to have a ‘real’ point of contact during pregnancy or postpartum21,23. A mixed approach, both online and in person, has proven to be effective and satisfactory during pregnancy17,23. One difficulty the healthcare system faces in meeting this demand is that managing a peer social network requires sustained dedication and technical knowledge on the part of professionals, who often do not have the resources to moderate this environment18,20.
Despite this, the use of new technologies in women’s healthcare allows midwives to advance in providing care focused on women’s experiential needs, not just clinical ones31. Having continuous and personalized attention fosters reflection, recognition, and expression of women’s preferences and personal desires32,33. A digital tool would make it possible to adapt prenatal education to their needs, develop a birth plan according to their own wishes, and experience childbirth and parenting in a way that goes beyond what is prescribed by the medical services33.
The sociodemographic profile of women who have used the tool is also reflected in their use of other health services34. Most of these women have a medium-high level of education and live in an urban environment16,17,35. Among other factors, this selection may be related to difficulties in using the tool or understanding the language21. In addition, identification is required, which can only be obtained if certain requirements are met, thereby discriminating against non-native or unregistered groups21. Despite their potential beneficial effect in improving self-care among most citizens, these technologies may increase inequalities in the use of health services and undermine the universality of public health services26,35.
Limitations
The response rate to the questionnaire was low, and the number of women who decided to participate in the interviews was small. Women who decided to participate in the study might have been more motivated to use the app and value it more positively than those who decided not to participate, with the risk of the results being weakened by non-response bias. Even though the use of a mixed methods approach might have partially overcome this limitation by the complementary perspective provided by each research method, repeated cross-sectional studies using validated questionnaires in the coming years are necessary to increase the soundness of current findings. Furthermore, the representation of women in the population with a foreign background who used the app is limited. Similarly, no difference was examined between pregnant and postpartum women in their responses to the quantitative study. It is possible that there was a recall bias related to the time elapsed since the tool was used. Therefore, future studies in our context should address this issue by focusing specifically on this population and their needs in relation to maternal education to assess if they are being met.
CONCLUSIONS
Women value the accessibility and reliability offered by a digital tool from the public healthcare system. It is not easy for this tool to compete with other commercial tools in terms of aesthetics, gamification, etc., as it must prioritize aspects such as privacy, which are not required by the former. However, areas for improvement have been identified, such as combining digital intervention with face-to-face care, thereby reinforcing confidence in the application, or personalizing messages according to the circumstances or characteristics of each woman. This individualization, including language options, would prevent the exclusion of the most vulnerable people. The evaluation of the EMAeHealth app using a mixed-methods approach allowed us to explore which features and functionalities are most valued and subsequently confirm that this assessment is shared by a broad population. Tools like the one presented in this study offer continuous and more specifically targeted support for each woman, and encourage women to express their needs, select resources, and adapt them to their preferences, thus promoting their own healthcare. The participation of professionals would improve the effectiveness of these tools.
