Sue & Bill Gross School of Nursing

Generative Artificial Intelligence is set to revolutionize healthcare delivery by transforming traditional patient care into a more personalized, efficient, and proactive process. Chatbots, serving as interactive conversational models, will probably drive this patient-centered transformation in healthcare. Through the provision of various services, including diagnosis, personalized lifestyle recommendations, dynamic scheduling of follow-ups, and mental health support, the objective is to substantially augment patient health outcomes, all the while mitigating the workload burden on healthcare providers. The life-critical nature of healthcare applications necessitates establishing a unified and comprehensive set of evaluation metrics for conversational models. Existing evaluation metrics proposed for various generic large language models (LLMs) demonstrate a lack of comprehension regarding medical and health concepts and their significance in promoting patients’ well-being. Moreover, these metrics neglect pivotal user-centered aspects, including trust-building, ethics, personalization, empathy, user comprehension, and emotional support. The purpose of this paper is to explore state-of-the-art LLM-based evaluation metrics that are specifically applicable to the assessment of interactive conversational models in healthcare. Subsequently, we present a comprehensive set of evaluation metrics designed to thoroughly assess the performance of healthcare chatbots from an end-user perspective. These metrics encompass an evaluation of language processing abilities, impact on real-world clinical tasks, and effectiveness in user-interactive conversations. Finally, we engage in a discussion concerning the challenges associated with defining and implementing these metrics, with particular emphasis on confounding factors such as the target audience, evaluation methods, and prompt techniques involved in the evaluation process.

The aim of this project was to increase willingness to receive the influenza vaccine to the optimal rate of ≥ 70%. Low acuity adult patients who visited an Emergency Department (ED) were assessed regarding their willingness to receive the influenza vaccine before and after an educational intervention that included a provider recommendation and an educational handout. A total of seventy-six patients (n = 76) were assessed. Patients willingness to receive the influenza vaccine rose from 29% pre-intervention to 72% post-intervention without disrupting the clinical flow in a busy ED. Similar vaccine educational strategies can be applied to influenza and other vaccines in EDs  to increase vaccination willingness in patients, including those who use the ED as a primary point of contact for healthcare, decreasing the burden of influenza illness in the community.

This dataset was collected from university students before, during, and after the COVID-19 lockdown in Southern California. Data collection happened continuously for the average of 7.8 months (SD=3.8, MIN=1.0, MAX=13.4) from a population of 21 students of which 12 have also completed an exit survey, and 7 started before the California COVID-19 lockdown order. This multimodal dataset included different means of data collection such as Samsung Galaxy Watch, Oura Ring, a Life-logger app named Personicle, a questionnaire mobile app named Personicle Questions, and periodical and personalised surveys. The dataset contains raw data from Photoplethysmogram (PPG), Inertial measurement unit (IMU), and pressure sensors in addition to processed data on heart rate, heart rate variability, sleep (bedtime, sleep stages, quality), and physical activity (step, active calories, type of activity). Ecological momentary assessments were collected from participants on daily and weekly bases containing their Positive and Negative Affect Schedule (PANAS) questionnaire and their emotional responses to COVID-19 and their health. Subjective data was also collected through monthly surveys containing standard mood and mental health surveys such as Beck Depression Inventory II (BDI-II), Brief Symptom Inventory (BSI), GAD-7, Inclusion of Other in the Self Scale (IOS-Partner), Acceptability of Intervention Measure (AIM), Intervention Appropriateness Measure (IAM), Feasibility of Intervention Measure (FIM), Experiences in Close Relationships Scale Short Form (ECR-S), UCLA Three-Item Loneliness Scale (ULS), Multidimensional Scale of Perceived Social Support (MSPSS), Investment Model Scale (IMS), Conflict Management Scale (CMS), etc in addition to their response to important events and COVID-19. This dataset can be used to study emotions, mood, physical activity, and lifestyle of young adults through longitudinal subjective and objective measures. This dataset also contains valuable data regarding adjustment of lifestyle and emotions during the events of 2020 and 2021 including COVID-19 discovery and lockdown, Black Life Matter movement, 2020 US presidential elections, etc. On average, participants engaged in the EMA collection study at a rate of 86% (SD=10, MIN=65, MAX=99). Smartwatch usage saw an average participation rate of 51% (SD=20, MIN=16, MAX=88), while engagement with the Oura ring averaged at 85% (SD=12, MIN=60, MAX=99).

The profound impact of food on health necessitates advanced nutrition-oriented food recommendation services. Conventional methods often lack the crucial elements of personalization, explainability, and interactivity. While Large Language Models (LLMs) bring interpretability and explainability, their standalone use falls short of achieving true personalization. In this paper, we introduce ChatDiet, a novel LLM-powered framework designed specifically for personalized nutrition-oriented food recommendation chatbots. ChatDiet integrates personal and population models, complemented by an orchestrator, to seamlessly retrieve and process pertinent information. The personal model leverages causal discovery and inference techniques to assess personalized nutritional effects for a specific user, whereas the population model provides generalized information on food nutritional content. The orchestrator retrieves, synergizes and delivers the output of both models to the LLM, providing tailored food recommendations designed to support targeted health outcomes. The result is a dynamic delivery of personalized and explainable food recommendations, tailored to individual user preferences. Our evaluation of ChatDiet includes a compelling case study, where we establish a causal personal model to estimate individual nutrition effects. Our assessments, including a food recommendation test showcasing a 92% effectiveness rate, coupled with illustrative dialogue examples, underscore ChatDiet's strengths in explainability, personalization, and interactivity.

AIM: This paper describes the trajectory during 1 year of four patient-reported outcomes (PROs), namely, sleep, depressive symptoms, health-related quality of life (HrQoL), and well-being, in patients with heart failure (HF), their relationship and the patient characteristics associated with changes in these PROs. METHODS AND RESULTS: Data analyses of PROs from 603 patients (mean age 67 years; 29% female, 60% NYHA II) enrolled in the HF-Wii study. On short term, between baseline and 3 months, 16% of the patients experienced continuing poor sleep, 11% had sustained depressive symptoms, 13% had consistent poor HrQoL, and 13% consistent poor well-being. Across the entire 1-year period only 21% of the patients had good PRO scores at all timepoints (baseline, 3, 6, and 12 months). All others had at least one low score in any of the PROs at some timepoint during the study. Over the 12 months, 17% had consistently poor sleep, 17% had sustained symptoms of depression, 15% consistently rated a poor HrQoL, and 13% poor well-being. Different patient characteristics per PRO were associated with a poor outcomes across the 12 months. Age, education, New York Heart Association, and length of disease were related to two PRO domains and submaximal exercise capacity (6 min test), co-morbidity, and poor physical activity to one. CONCLUSION: In total, 79% of the patients with HF encountered problems related to sleep, depressive symptoms, HrQoL, and well-being at least once during a 1-year period. This underscores the need for continuous monitoring and follow-up of patients with HF and the need for dynamic adjustments in treatment and care regularly throughout the HF trajectory.

AIMS: To describe sources of health information and health-seeking behaviours of adults (aged ≥18) living in medically underserved communities in the Philippines. DESIGN: This is a secondary, quantitative analysis from a cross-sectional parent study. Participants completed a 10-item, self-report survey on their sources of health information, healthcare providers sought for health and wellness and health-seeking behaviours when ill. Responses were evaluated across two age groups (<60 vs. ≥60 years) and genders using generalized linear mixed models. RESULTS: Surveys were completed by 1202 participants in rural settings (64.6% female, mean age 49.5 ± 17.6). Friends and/or family were their key source of health information (59.6%), followed by traditional media (37%) and healthcare professionals (12.2%). For health promotion, participants went to healthcare professionals (60.9%), informal healthcare providers (17.2%) or others (7.2%). When ill, they visited a healthcare professional 69.1% of the time, self-medicated (43.9%), prayed (39.5%) or sought treatment from a rural health clinic (31.5%). We also found differences in health-seeking behaviours based on age and gender. CONCLUSIONS: Our findings highlight the need to organize programs that explicitly deliver accurate health information and adequate care for wellness and illness. Study findings emphasize the importance of integrating family, friends, media and healthcare professionals, including public health nurses, to deliver evidence-based health information, health promotion and sufficient treatment to medically underserved Filipinos. IMPLICATIONS: New knowledge provides valuable information to healthcare providers, including public health nurses, in addressing health disparities among medically underserved Filipinos. IMPACT: This study addresses the current knowledge gap in a medically vulnerable population. Healthcare professionals are not the primary sources of health information. Approximately one-third of participants do not seek them for health promotion or treatment even when ill, exacerbating health inequities. More work is necessary to support initiatives in low- and middle-income countries such as the Philippines to reduce health disparities. REPORTING METHOD: We adhered to the reporting guidelines of STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) for cross-sectional studies. PATIENT OR PUBLIC CONTRIBUTION: There was no patient or public contribution as our study design and methodology do not make this necessary.

Despite the growing number of Korean American (KA) family caregivers for persons with dementia, little is known about how acculturation might affect caregiving stress in this population. Acculturation is a variable of considerable interest in caregiving research due to its significance in understanding the impact of cultural perceptions and expectations on the caregiving role and its relation to mental health outcomes. A cross-sectional descriptive study using baseline data from an ongoing randomized controlled trial of dementia caregiver intervention was performed to examine the association between acculturation and mental health outcomes among KA caregivers (n = 32) for persons with dementia. Self-report survey questionnaires including a bidirectional acculturation scale, Center for Epidemiologic Studies Depression Scale, the Perceived Stress Scale, and the Zarit Burden Interview were administered in person in English or Korean by trained bilingual community health workers. The primary independent variable, acculturation, was assessed using a 24-item inventory. It measured two sets of cultural orientation: Korean orientation and American orientation. The mean age was 67 years (SD = 11.8) and 87% were women. Half of the caregivers were spouses of persons with dementia, while the other half were offspring caregivers. In the multiple linear regression model, caregiver acculturation toward Korean cultural orientation had a significant and positive association with depressive symptoms (β = .62; SE = 0.25; p-value = .02) and perceived stress (β = .29; SE = 0.13; p-value = .03) after adjusting for age and self-efficacy. No significant effect of American cultural orientation was found for caregiver burden, perceived stress, or depressive symptoms. Our findings suggest that exploring the role of acculturation in caregiving and its relation to outcomes, particularly caregiver distress, may be valuable for future studies aiming to understand specific elements of cultural values and practices in the acculturation process related to mental health outcomes among immigrant Korean American caregivers.

BACKGROUND: Approximately 70-80% of people experiencing homelessness in the United States use tobacco. Smoking cessation programs specifically for this population have been found to be less effective for African American participants. The purpose of this study was to explore discrimination experiences and their impact on smoking habits and readiness to quit cigarette smoking while experiencing homelessness. METHODS: In the qualitative phase of this mixed methods study, five focus groups were conducted for African Americans residing in a homeless shelter in Skid Row, Los Angeles, CA. Using a semi-structured interview guide, we asked participants about discrimination experiences, how smoking habits were impacted by these experiences, and tools needed to successfully abstain from cigarette smoking. Qualitative descriptive content analysis was used to explore discrimination experiences and its association with readiness to quit cigarette smoking. RESULTS: Of the 17 participants, 14 (82.4%) were male, and the average age was 46.8 years. Using a qualitative In Vivo coding method, three themes were revealed: Experiencing Discrimination while Black, The Psychosocial Fabric-Why Quitting Cigarette Smoking is a Challenge, and The Lesser of Two Evils-Choosing to Smoke over More Harmful Options. Participants discussed working in the blue-collar workforce while Black, identifying as a double minority, smoking to cope with stress, early exposure to cigarettes, smoking being a central part of ones belonging to a group, and the legality of cigarette smoking. DISCUSSION: Our findings show that African Americans experiencing homelessness (1) may experience discrimination in multiple settings, regardless of housing status, (2) could have grown up around cigarette smoking and remain surrounded by it while experiencing homelessness, and (3) may experience a calming effect with smoking, which slows some from reacting negatively to adverse situations. CONCLUSION: Barriers to successfully abstaining from smoking are multifactorial among African Americans experiencing homelessness and should be addressed individually. Future research should explore the cultural tailoring of interventions that support cessation efforts unique to minoritized populations to improve smoking cessation programs offered to this population.

BACKGROUND: Stress and anxiety during pregnancy are extremely prevalent and are associated with numerous poor outcomes, among the most serious of which are increased rates of preterm birth and low birth weight infants. Research supports that while in-person mindfulness training is effective in reducing pregnancy stress and anxiety, there are barriers limiting accessibility. OBJECTIVE: The aim of this paper is to determine if mindfulness meditation training with the Headspace app is effective for stress and anxiety reduction during pregnancy. METHODS: A longitudinal, single-arm trial was implemented with 20 pregnant women who were instructed to practice meditation via the Headspace app twice per day during the month-long trial. Validated scales were used to measure participants levels of stress and anxiety pre- and postintervention. Physiological measures reflective of stress (heart rate variability and sleep) were collected via the Oura Ring. RESULTS: Statistically significant reductions were found in self-reported levels of stress (P=.005), anxiety (P=.01), and pregnancy anxiety (P<.0001). Hierarchical linear modeling revealed a statistically significant reduction in the physiological data reflective of stress in 1 of 6 heart rate variability metrics, the low-frequency power band, which decreased by 13% (P=.006). A total of 65% of study participants (n=13) reported their sleep improved during the trial, and 95% (n=19) stated that learning mindfulness helped with other aspects of their lives. Participant retention was 100%, with 65% of participants (n=13) completing about two-thirds of the intervention, and 50% of participants (n=10) completing ≥95%. CONCLUSIONS: This study found evidence to support the Headspace app as an effective intervention to aid in stress and anxiety reduction during pregnancy.

We conducted a cross-sectional study, examining the mediation effects of depression and anxiety on the association between discrimination and readiness to quit cigarette smoking among African American adult cigarette smokers experiencing homelessness. Using a convenience sample, participants were recruited from a homeless shelter in Southern California. Scores of discrimination, depressive, and anxiety symptoms, and readiness to quit smoking were analyzed using linear regression modeling. We enrolled 100 participants; 58 participants were male. In the final model, discrimination had no association with readiness to quit (b = 0.02; 95% CI [-0.04, 0.08]; p = 0.47). The indirect effects of depression (b = 0.04, [0.01, 0.07]; p = 0.02) and anxiety (b = 0.03; [0.01, 0.05]; p = 0.04) reached statistical significance; the direct effects of depression (b = -0.01; [-0.09, 0.04]; p = 0.70) and anxiety (b = -0.00; [-0.09, 0.06]; p = 0.86) did not. Future studies should explore these associations to enhance smoking cessation programs for this population.