Advertisement

Electronic Patient Symptom Management Program to Support Patients Receiving Cancer Treatment at Home During the COVID-19 Pandemic

Published:March 23, 2022DOI:https://doi.org/10.1016/j.jval.2022.01.023

      Abstract

      Objectives

      Remote patient monitoring became critical for patients receiving cancer treatment during the COVID-19 pandemic. We sought to test feasibility of an electronic patient symptom management program implemented during a pandemic. We collected and analyzed the real-world data to inform practice quality improvement and understand the patient experience.

      Methods

      Eligible patients had breast, lung, or ovarian cancers, multiple myeloma, or acute myeloid leukemia and 12 weeks of planned chemotherapy. Patients were notified that a symptom survey with common symptoms derived from the National Cancer Institute’s Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events was available to complete using a smart phone, tablet, or computer. Patients recorded their symptoms and results were sent to the provider. Patients received care guidelines for mild/moderate severity symptoms and a phone call from the provider for severe reports.

      Results

      A total of 282 patients generated > 119 088 data points. Patients completed 2860 of 3248 assigned surveys (88%), and 152 of 282 patients (54%) had symptom reports that generated an immediate notification to the provider. Longitudinal data were analyzed to determine whether previous reports predicted a notification alert and whether symptoms resolved after the alert was addressed.

      Conclusions

      An electronic patient symptom management program was implemented in the midst of the COVID-19 pandemic. Enrollment of 282 patients and a high survey completion (88%) demonstrated feasibility/acceptance. Patients reported symptoms at severe levels of 54% of the time and received self-management instructions and provider phone calls that resolved or decreased the severity of the symptom. A standard approach and validated instrument provide opportunities for improving and benchmarking outcomes.

      Keywords

      Introduction

      All patients with cancer will experience symptoms from either their disease or its treatment at some point in their care often with interference in activities of daily living and decreased quality of life.
      • Reilly C.M.
      • Bruner D.W.
      • Mitchell S.A.
      • et al.
      A literature synthesis of symptom prevalence and severity in persons receiving active cancer treatment.
      It is well established that symptoms are under-reported and underassessed during clinical visits because of a variety of reasons such as time allotted for clinical visits, provider sensitivity toward specific treatment side effects, and the patients’ desire to minimize symptoms to receive treatment.
      • Basch E.
      • Jia X.
      • Heller G.
      • et al.
      Adverse symptom event reporting by patients vs clinicians: relationships with clinical outcomes.
      • Laugsand E.A.
      • Sprangers M.A.G.
      • Bjordal K.
      • Skorpen F.
      • Kaasa S.
      • Klepstad P.
      Health care providers underestimate symptom intensities of cancer patients: a multicenter European study.
      • Atkinson T.M.
      • Li Y.
      • Coffey C.W.
      • et al.
      Reliability of adverse symptom event reporting by clinicians.
      • Basch E.
      • Deal A.M.
      • Kris M.G.
      • et al.
      Symptom monitoring with Patient-Reported Outcomes during routine cancer treatment: a randomized controlled trial [published correction appears in J Clin Oncol. 2016;34(18):2198] [published correction appears in J Clin Oncol. 2019;37(6):528].
      In addition, inadequate symptom control contributes to poor clinical outcomes, diminished quality of life, lost patient and family productivity, avoidable emergency department visits, hospitalizations and readmissions, poor treatment adherence, and decreased survival.
      • Mayer D.K.
      • Travers D.
      • Wyss A.
      • Leak A.
      • Waller A.
      Why do patients with cancer visit emergency departments? Results of a 2008 population study in North Carolina.
      • Kotronoulas G.
      • Kearney N.
      • Maguire R.
      • et al.
      What is the value of the routine use of patient-reported outcome measures toward improvement of patient outcomes, processes of care, and health service outcomes in cancer care? A systematic review of controlled trials.
      • Howell D.
      • Molloy S.
      • Wilkinson
      • et al.
      Patient-reported outcomes in routine cancer clinical practice: a scoping review of use, impact on health outcomes, and implementation factors.
      • Denis F.
      • Yossi S.
      • Septans A.-L.
      • et al.
      Improving survival in patients treated for a lung cancer using self-evaluated symptoms reported through a web application.
      Multiple studies have shown proactive symptom management in response to patient reports of symptoms reduces symptom burden, decreases healthcare resource utilization, and improves survival,
      • Basch E.
      • Deal A.M.
      • Kris M.G.
      • et al.
      Symptom monitoring with Patient-Reported Outcomes during routine cancer treatment: a randomized controlled trial [published correction appears in J Clin Oncol. 2016;34(18):2198] [published correction appears in J Clin Oncol. 2019;37(6):528].
      ,
      • Basch E.
      • Deal A.M.
      • Dueck A.C.
      • et al.
      Overall survival results of a trial assessing patient-reported outcomes for symptom monitoring during routine cancer treatment.
      • Denis F.
      • Lethrosne C.
      • Pourel N.
      • et al.
      Randomized trial comparing a web-mediated follow-up with routine surveillance in lung cancer patients [published correction appears in J Natl Cancer Inst. 2018;110(4):436].
      • Fjell M.
      • Langius-Eklof A.
      • Nilsson M.
      • Wengström Y.
      • Sundberg K.
      Reduced symptom burden with the support of an interactive app during neoadjuvant chemotherapy for breast cancer- a randomized controlled trial.
      • Mierzynska J.
      • Piccinin C.
      • Pe M.
      • et al.
      Prognostic value of patient-reported outcomes from international randomised clinical trials on cancer: a systematic review.
      yet widespread clinical adoption is lacking as cancer care institutions struggle to adopt patient-reported outcomes (PROs) into their workflow. Electronic PRO solutions are being developed and tested and show improved symptom experience.
      • Fjell M.
      • Langius-Eklof A.
      • Nilsson M.
      • Wengström Y.
      • Sundberg K.
      Reduced symptom burden with the support of an interactive app during neoadjuvant chemotherapy for breast cancer- a randomized controlled trial.
      ,
      • Judson T.J.
      • Bennett A.V.
      • Rogak L.J.
      • et al.
      Feasibility of long-term patient self-reporting of toxicities from home via the internet during routine chemotherapy.
      • Ciani O.
      • Cucciniello M.
      • Petracca F.
      • et al.
      Lung Cancer App (LuCAPP) study protocol: a randomised controlled trial to evaluate a mobile supportive care app for patients with metastatic lung cancer.
      • Avery K.N.L.
      • Richards H.S.
      • Portal A.
      • et al.
      Developing a real-time electronic symptom monitoring system for patient after discharge following cancer-related surgery.
      • Gandrup J.
      • Ali S.M.
      • McBeth J.
      • van der Veer S.N.
      • Dixon W.G.
      Remote symptom monitoring integrated into electronic health records: a systematic review.
      • Greer J.A.
      • Jacobs J.M.
      • Pensak N.
      • et al.
      Randomized trial of a smartphone mobile app to improve symptoms and adherence to oral therapy for cancer.
      Carevive Systems, Inc™, developed a remote symptom management solution that includes personalized, computable algorithms for tailored, evidence-based clinical symptom management with the promise of improving care by systematic delivery of symptom assessments that allow earlier identification and management of symptoms before they escalate and require unplanned clinic or emergency department visits and preventable hospitalizations. Three cancer centers were starting implementation of the Carevive electronic patient symptom management system (ePSMS) at the beginning of the COVID-19 pandemic. It became quickly apparent that patients would continue to receive cancer treatment but would need to remain away from the clinic as much as possible to follow pandemic restrictions.
      • Moazzami B.
      • Razavi-Khorasani N.
      • Dooghale-Modghadam A.
      • et al.
      COVID-19 and telemedicine. Immediate action required for maintaining healthcare providers well-being.
      • Marandino L.
      • Necchi A.
      • Aglietta M.
      • Di Maio M.
      COVID-19 emergency and the need to speed up the adoption of electronic patient-reported outcomes in cancer clinical practice.
      • Schrag D.
      • Hershman D.L.
      • Basch E.
      Oncology practice during the COVID-19 pandemic.
      • Waisberg F.
      • Enrico D.
      • Angel M.
      • Chacón M.
      Cancer treatment adaptations in the COVID-19 era.
      The necessity for a system to monitor patients remotely, mitigate mild and moderate severity symptoms, and “alert” care providers of severe symptoms became paramount. This report details data on 284 patients who enrolled between September 2020 and October 2021.

      Methods

      Goals

      The primary goal of this study was to test feasibility of implementing an ePSMS by the number of patients enrolled at each site, survey adherence, and survey completeness. Our secondary goal was to collect and describe the alerts reported to inform practice quality improvement and better understand the real-world patient experience. Given that remote symptom management was considered standard of care at the participating sites and all participants sign an end-user license agreement to allow use of their deidentified data, the institutional review board determined the study was exempt from institutional review board oversight.

      Procedures

      The design uses rolling enrollment methodology with patient PROs collected on a weekly basis and alerts conveyed to providers on a real-time basis. Eligible patients had one of 5 cancer diagnoses (breast, lung, or ovarian cancers, multiple myeloma, or acute myeloid leukemia), had a minimum of 12 weeks of planned chemotherapy, were 18 years or older, and had access to a smartphone, tablet, or computer. The selected cancers represent the initial diagnoses supported in the ePSMS with others in development for future expansion of the program. The 12-week minimum for weekly surveys is used by the sites as a criterion to justify the effort associated with enrollment and ongoing patient management remotely. The ePSMS was implemented at 3 US cancer centers, 2 academic centers in the Southeast, and 1 community center in the Mountain West.
      Patients were enrolled in the cloud-based platform and notified via text or email that a symptom survey was available to be completed. Patients signed into a secure site to record their symptoms using a smart phone, tablet, or computer, and results were sent to the provider through the electronic medical record (EMR). Specified symptom levels such as severe or very severe or in some cases such as fever that is present, yes or no, were set to trigger a provider notification. Patients received written care guidelines for mild or moderate severity symptoms and a phone call from the care team for management of alerting symptoms. Of note, patients with acute myeloid leukemia who were hospitalized completed surveys and symptom alerts were reported to the inpatient clinical team.

      Instrument

      Patients complete weekly symptom reports as long as they are receiving treatment. Surveys delivered weekly include a core set of common symptoms derived from the National Cancer Institute’s Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE™).
      • Hay J.L.
      • Atkinson T.M.
      • Reeve B.B.
      • et al.
      Cognitive interviewing of the US National Cancer Institutes’ Patient-Reported outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE).
      ,

      Basch E, Reeve BB, Mitchell SA, et al. Development of the National Cancer Institute’s patient-reported outcomes version of the common terminology criteria for adverse events (PRO-CTCAE). J Natl Cancer Inst. 2013;106(9):dju244.

      The PRO-CTCAE library contains 124 items to measure 78 symptomatic adverse events. Reeve et al
      • Reeve B.B.
      • Mitchell S.A.
      • Dueck A.
      • et al.
      Recommended patient-reported core set of symptoms to measure in adult cancer treatment trials.
      (2014) identified a core set of 12 symptoms prevalent across diverse cancer populations (fatigue, insomnia, pain, appetite loss, dyspnea, cognitive problems, anxiety, nausea, depression [sadness], neuropathy, constipation, and diarrhea). Per the participating physicians, these symptoms, excluding cognitive problems, are part of the week survey along with some additional symptoms requested by providers (rash and mouth/throat sores).
      Given that the PRO-CTCAE is a well-established instrument with proven reliability and validity,
      • Dueck A.C.
      • Mendoza T.R.
      • Mitchell S.A.
      • et al.
      Validity and reliability of the US National Cancer Institutes’ Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE).
      patients did not receive specific instructions about the measures before starting the surveys. When the symptom occurs, severity is reported as mild, moderate, severe, or very severe.
      • Atkinson T.M.
      • Hay J.L.
      • Dueck A.C.
      • et al.
      What do “none,” “mild,” “moderate,” “severe,” and “very severe,” mean to patients with cancer? Content validity of PRO-CTCAE™ response scales.
      Additional questions about frequency and interference with daily activities are included for some symptoms. A composite score of all attributes for each symptom is derived using an algorithm for mapping individual items for any given adverse event to a single (composite) numerical grade.
      • Basch E.
      • Becker C.
      • Rogak J.
      • et al.
      Composite grading algorithm for the National Cancer Institute’s Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE).
      All data are collected either directly into the Carevive platform via integration with the site EMR and data entered directly by the site staff and from the patient surveys. A security and privacy provider manages all protected health information and any access to it. Carevive supports operations at all sites and has Health Insurance Portability and Accountability Act Business Associate Agreements in place with all data providers.

      Data Structure

      Data manipulation and analyses were performed using SAS version 9.4
      SAS version 9.4. SAS.
      and SPSS version 27.0.
      SPSS version 27.0 IBM.
      The analyses focused on 3 data structures: (1) demographics with one record per patient (number of records = 282), (2) weekly data captured across all variables for each patient within a given week (number of data points = 119 088), and (3) patient-/week-level data set where data were aggregated at the weekly report level and then nested within a given patient (number of weekly nested reports = 2790). Patients provided a range of 1 to 57 weekly surveys with an average of 12.17 weeks (SD 9.54) of data; 55 of 282 patients (20%) had completed treatment and were no longer using the platform at the time of this analysis.
      A critical focus of this work was to examine symptom levels immediately before an alerting episode (look back) and immediately after an alerting episode (look forward). We explored the relationship of the symptom report before the symptom alert report and the relationship of the symptom report after the alert report to explore whether an intervention was effective or not. Preliminary results showed mild symptom profiles accelerated within 1 week to trigger an alert. In the look back analyses, only “naive” alerts were studied, naive meaning that the alerting episode was either the first episode for a patient, or for patients with one or more alerting reports, an alert was considered naive if there was at least 1 nonalerting symptom report immediately before the alerting report of interest. In this look back process, the immediately preceding symptom levels were juxtaposed with the symptom alert levels to compute the alert change scores to indicate improvement, no change, or worsening of the symptom levels.
      In the look forward process, the symptom report immediately after an alerting report was examined for change in symptom level after an alert. In a process like the selection of naive alerts in the look back analysis, only those records after an alert that themselves were nonalerting records were examined. In this process, the record immediately after an alert was juxtaposed with the alerting levels that generate the alert, and as above, change scores were computed to indicate improvement, no change, or worsening of the symptom levels.
      In these processes (look back and look forward), the changes in symptom levels were quantified by computing change scores (Rising_delta = CompositePrealert − CompositeAlert). The same process was used to examine resolution. Change scores were computed as (Resolution_delta = CompositeAlert − Composite Postalert). These change scores could range from “worsening by 3 points” to “no change” to “improving by 3 points.” Both Rising_delta and Resolution_delta scores were examined for patterns of change overall and specifically for comparing patterns of change using divergent stacked bar charts as per recommendation by Zhou et al (2018).
      • Zhou X.
      • Eid D.
      • Gnanasakthy A.
      Methods for reporting the patient-reported outcomes version of the common terminology criteria for adverse events (PRO-CTCAE) data in cancer clinical trials.

      Results

      Participants had a mean age of 58.96 years (SD = 13.18), were mostly female (85.5%), were white (72.4%), and had a diagnosis of breast cancer (63.1%) (see Table 1). Patients completed 2860 of 3248 assigned surveys (88.0%), and 152 of 282 patients (54.0%) had symptom reports that rose to the level of an immediate notification sent to the provider. The most common symptoms reported were fatigue (84%), pain (54%), decreased appetite (52%), insomnia (51%), and nausea (50%) (see Table 2). The top symptoms that triggered alert reports were pain (42%), fatigue (32%), and insomnia (27%) (see Table 2). The top symptom alerts across the sample by cancer type, race, and sex were examined. These analyses were limited by sparse cells, and only the most robust were reported here. Cancer type was significantly associated with generation of pain alerts (χ2(4) = 11.38 P =.03.) The nature of this association was that patients with breast cancer were significantly more likely to trigger pain alerts. In addition, females were more likely than males to trigger pain alerts (χ2(2) =4.62 P =.03). Finally, we found no association between race and generation of alerts.
      Table 1Sample characteristics (N = 282).
      Mean (SD)
      Age58.96 (13.18)
      Number (%)
      Sex
       Male41 (14.4)
       Female241 (85.5)
      Race
       American native5 (1.8)
       Asian8 (2.9)
       African American59 (21.7)
       White197 (72.4)
       Other3 (1.1)
       Missing10 (3.5)
      Cancer diagnosis
       AML29 (10.3)
       Breast178 (63.1)
       Lung19 (6.7)
       Myeloma39 (13.8)
       Ovarian/gyn17 (6.0)
      AML indicates acute myeloid leukemia; gyn, gynecological.
      Table 2Incidence of symptom and alert reports.
      SymptomAlerted
      Of those reporting yes, to having a symptom, the composite score triggered an alert (notification) to the provider.


      Within the 152 who reported alerts, %
      (95% CI)Overall
      Patients who reported yes to experiencing a symptom in the past 7 days.


      Reported binary data

      With N = 282, %
      (95% CI)
      Pain42(0.34-0.50)47(0.41-0.57)
      Constipation22(0.16-0.29)48(0.40-0.56)
      Insomnia27(0.20-0.34)50(0.42-0.58)
      Fatigue32(0.25-0.40)84(0.78-0.90)
      Decreased appetite19(0.13-0.25)53(0.45-0.62)
      Nausea20(0.13-0.26)51(0.43-0.60)
      Neuropathy12(0.07-0.17)46(0.38-0.55)
      Diarrhea8(0.04-0.12)50(0.42-0.58)
      Anxiety9(0.04-0.13)47(0.38-0.58)
      Rash7(0.03-0.11)26(0.19-0.34)
      Sadness/depression5(0.02-0.09)28(0.21-0.36)
      SOB10(0.05-0.15)40(0.32-0.48)
      Mouth/throat sores5(0.01-0.08)34(0.26-0.41)
      CI indicates, confidence interval; SOB, shortness of breath.
      Of those reporting yes, to having a symptom, the composite score triggered an alert (notification) to the provider.
      Patients who reported yes to experiencing a symptom in the past 7 days.
      With regard to the look back and look forward analyses, the change scores as discussed earlier were graphed as divergent stacked bar charts and are provided in Figure 1A,B. In general, these figures show the improvement percentage and worsening percentage in composite scores between 1 week before an alert report (rising delta) and 1 week after an alert report (resolution delta) for 6 symptoms: pain, nausea, and insomnia in Figure 1A and fatigue, constipation and diarrhea in Figure 1B (graphs are separated only for ease of presentation). The graphs summarize data in a way that can be quickly compared. For instance, in Figure 1A, the change from the prepain alert to the pain alert indicated 18.5% patients showed worsening by 3 composite score points, 44.0% showed worsening by 2 points, and 24.7% showed worsening by 1 point (the negative sign to the worsening percentages are a function of how the data are graphed.) In this figure, 12.4% showed no change from the week before the alert report to the alert report. In contrast, we see in the corresponding element immediately below, in the postpain alert, 23.2% of the patients showed improvement by 3 points, 34.8% improved by 2 points, and 16.8% improved by 1 point. Notably, 23.8% showed no change and a small percentage (< 3%) showed a worsening by 1 point.
      Figure thumbnail gr1
      Figure 1Divergent stacked bar charts show percent improvement and percent worsening in composite scores between one week prior to an alert and one week after an alert report. (A) Shows pain, nausea and insomnia scores. (B) Shows fatigue, constipation and diarrhea scores.

      Discussion

      This report describes the initial results of implementation of remote symptom management using an ePRSM system. Due to the challenges of patient care during the COVID-19 pandemic, enrollment was significantly less than expected; nevertheless, adherence and completion rates were higher than anticipated and comparable with clinical trial reports of 70% to 90% adherence.
      • Basch E.
      Patient-reported outcomes-harnessing patients’ voices to improve clinical care.
      • Benze G.
      • Nauck F.
      • Alt-Epping B.
      • et al.
      PROutine: a feasibility study assessing surveillance of electronic patient reported outcomes and adherence via smartphone app in advanced cancer.
      • Wright A.A.
      • Raman N.N.
      • Staples P.
      • et al.
      The HOPE pilot study: harnessing patient-reported outcome and biometric data to enhance cancer care.
      High adherence is attributed to patients knowing that the reports were being monitored.
      In addition, known barriers to adoption such as a lack of EMR integration and a lack of effective processes for clinical workflow integration
      • Basch E.
      Patient-reported outcomes-harnessing patients’ voices to improve clinical care.
      ,
      • Harle C.A.
      • Listhaus A.
      • Covarrubias C.M.
      • et al.
      Overcoming barriers to implementing patient-reported outcomes in an electronic health record: a case report.
      were overcome through integration of the ePSMS with the EMR and intense support for workflow design during the implementation period. Additional funds for staff support were provided and welcomed during the time of staffing shortages and reassignment. Other keys to adoption have been identified such as the immediate scoring and reporting, longitudinal display, clear workflow as to who receives and responds to alerts, and guideline-supported recommendations available.
      • Duncan E.A.
      • Murray J.
      The barriers and facilitators to routine outcome measurement by allied health professionals in practice: a systematic review.
      • Jensen R.E.
      • Rothrock N.E.
      • DeWitt E.M.
      • et al.
      The role of technical advances in the adoption and integration of patient-reported outcomes in clinical care.
      • Mooney K.
      • Berry D.
      • Whisenant M.
      • Sjoberg D.
      Improving cancer care through the patient experience: how to use patient-reported outcomes in clinical practice.
      The ePSMS includes each of these features.
      The acceptability of digital PRO systems is growing with many clinical trials demonstrating decreased symptom severity with automated self-management and calls for poorly managed symptoms.
      • Mooney K.
      • Berry D.
      • Whisenant M.
      • Sjoberg D.
      Improving cancer care through the patient experience: how to use patient-reported outcomes in clinical practice.
      • Basch E.
      • Stover A.
      • Schrag D.
      • et al.
      Clinical utility and user perceptions of a digital system for electronic patient-reported symptom monitoring during routine cancer care: findings from the PRO-TECT trial.
      • Cleeland C.S.
      • Wang X.S.
      • Shi Q.
      • et al.
      Automated symptom alerts reduce postoperative symptom severity after cancer surgery: a randomized controlled clinical trial.
      • Chung A.E.
      • Basch E.M.
      Incorporating the patient’s voice into electronic health records through patient-reported outcomes as the “review of symptoms”.
      Analyzing the longitudinal symptom reports to track response to interventions provides the clinical practice feedback needed for continuous quality improvement. With more patients and further analysis of specific interventions, the potential usefulness of the data increases. In the look back for each symptom, patients who reported the symptom that did not rise to the level of an alert in the week before the alerting symptom report can be examined for effectiveness of the self-management measures and potential need for earlier calls by the care team. The look forward data can provide insight into the effectiveness (or ineffectiveness) of interventions after the alert. Using a standard approach and a validated PRO symptom instrument provides opportunities for benchmarking and sharing best practices.
      Implementing the ePSMS in 3 institutions during the COVID-19 pandemic provided both opportunities and challenges. Patients with cancer are more at risk of morbidity and mortality with COVID-19 infection because of age, comorbidities, and the immunosuppression related to the tumor itself or treatment effects.
      • Marandino L.
      • Necchi A.
      • Aglietta M.
      • Di Maio M.
      COVID-19 emergency and the need to speed up the adoption of electronic patient-reported outcomes in cancer clinical practice.
      Millions of patients were transitioned to telemedicine visits to decrease risk for patients and staff alike,
      • Schrag D.
      • Hershman D.L.
      • Basch E.
      Oncology practice during the COVID-19 pandemic.
      and treatments were modified or delayed to reduce in person patient visits.
      • Waisberg F.
      • Enrico D.
      • Angel M.
      • Chacón M.
      Cancer treatment adaptations in the COVID-19 era.
      Not surprisingly, patients reported higher symptom burden and stress during the COVID-19 pandemic.
      • Miaskowski C.
      • Paul S.M.
      • Snowberg K.
      • et al.
      Stress and symptom burden in oncology patients during the COVID-19 pandemic.
      The ePSMS was welcomed as a tool to support patient care, yet widespread adoption was hampered by reduced personnel because of positive test results or reassigning staff to the support the increased demand for inpatient care. Staff spent enormous effort rescheduling patients for telemedicine visits, prescreening those who continued receiving treatments for COVID-19 symptoms, and adapting to evolving procedure and policy changes. Although these barriers were not addressed with ePSMS, enrollment of patients into the ePSMS supported efficient screening of patients and symptom management at home. In addition, staff could also quarantine at home and securely access the patient reports to assure continuity of care.
      There are several limitations in this study. Patients were not educated on the meaning of symptom responses before taking a survey; therefore, internal consistency and reliability were not established. In addition, self-reporting of symptoms can potentially bias the responses. There is a preponderance of women in the study because of the diagnoses of breast and gynecological cancer and the sex imbalance may affect results. Eligibility required access to a smartphone, tablet, or computer. Implementation of this symptom reporting and management system may not be feasible or accessible to underserved and disadvantaged populations.
      The look back and look forward analyses as portrayed in the divergent stacked bar graphs provide a useful snapshot of the rise and resolution of symptoms. Future analyses will focus on the interventions made in response to the alerting symptom reports to determine the effectiveness of the interventions in resolving the symptom. We will investigate unexpected findings such as the relatively large percentage of patients who did not improve or got worse after an alert, including the clinical and demographic factors associated with this lack of change. With increased sample size with continual enrollment, we expect to extend the prealert measurement to more than a single prealert measure so that we can better quantify the process of symptom change leading to an alert. In addition, we anticipate the use of clustering techniques to uncover latent groups of individuals based on change scores and to examine predictors of cluster membership and the impact of cluster membership on outcomes.

      Conclusions

      This article describes an ePSMS implemented at 3 institutions in the midst of the COVID-19 pandemic. Feasibility and acceptance were demonstrated through enrollment of 282 patients undergoing cancer treatment with a high rate of weekly survey completion. Patients reported symptoms at severe levels of 54% of the time and received self-management instructions and provider phone calls that resolved or decreased the severity of the symptom. Continued aggregation of homogenously reported symptoms provides opportunities for quality improvement, benchmarking outcomes, and sharing of best practices.

      Article and Author Information

      Author Contributions: Concept and design: Wujcik, W. Dudley, Brant
      Acquisition of data: Wujcik, Gupta, Brandt
      Analysis and interpretation of data: Wujcik, W. Dudley, M. Dudley, Gupta, Brant
      Drafting of the manuscript: Wujcik, W. Dudley, M. Dudley
      Critical revision of the paper for important intellectual content: Wujcik, W. Dudley, Brant
      Statistical analysis: W. Dudley, M. Dudley, Gupta
      Provision of study materials or patients: Wujcik
      Obtaining funding: Wujcik
      Administrative, technical, or logistic support: Wujcik
      Supervision: Wujcik
      Conflict of Interest Disclosures: Dr Wujcik and Ms Gupta are employed by Carevive Systems Inc. Dr W. Dudley and Mr M. Dudley reported providing statistical support through Piedmont Research Strategies Inc. No other disclosures were reported.
      Funding/Support: The authors received no financial support for this research.

      References

        • Reilly C.M.
        • Bruner D.W.
        • Mitchell S.A.
        • et al.
        A literature synthesis of symptom prevalence and severity in persons receiving active cancer treatment.
        Support Care Cancer. 2013; 21: 1525-1550
        • Basch E.
        • Jia X.
        • Heller G.
        • et al.
        Adverse symptom event reporting by patients vs clinicians: relationships with clinical outcomes.
        J Natl Cancer Inst. 2009; 101: 1624-1632
        • Laugsand E.A.
        • Sprangers M.A.G.
        • Bjordal K.
        • Skorpen F.
        • Kaasa S.
        • Klepstad P.
        Health care providers underestimate symptom intensities of cancer patients: a multicenter European study.
        Health Qual Life Outcomes. 2010; 8: 104-118
        • Atkinson T.M.
        • Li Y.
        • Coffey C.W.
        • et al.
        Reliability of adverse symptom event reporting by clinicians.
        Qual Life Res. 2012; 21: 1159-1164
        • Basch E.
        • Deal A.M.
        • Kris M.G.
        • et al.
        Symptom monitoring with Patient-Reported Outcomes during routine cancer treatment: a randomized controlled trial [published correction appears in J Clin Oncol. 2016;34(18):2198] [published correction appears in J Clin Oncol. 2019;37(6):528].
        J Clin Oncol. 2016; 34: 557-565
        • Mayer D.K.
        • Travers D.
        • Wyss A.
        • Leak A.
        • Waller A.
        Why do patients with cancer visit emergency departments? Results of a 2008 population study in North Carolina.
        J Clin Oncol. 2011; 29: 2683-2688
        • Kotronoulas G.
        • Kearney N.
        • Maguire R.
        • et al.
        What is the value of the routine use of patient-reported outcome measures toward improvement of patient outcomes, processes of care, and health service outcomes in cancer care? A systematic review of controlled trials.
        J Clin Oncol. 2014; 32: 1480-1501
        • Howell D.
        • Molloy S.
        • Wilkinson
        • et al.
        Patient-reported outcomes in routine cancer clinical practice: a scoping review of use, impact on health outcomes, and implementation factors.
        Ann Oncol. 2015; 26: 1846-1858
        • Denis F.
        • Yossi S.
        • Septans A.-L.
        • et al.
        Improving survival in patients treated for a lung cancer using self-evaluated symptoms reported through a web application.
        Am J Clin Oncol. 2017; 40: 464-469
        • Basch E.
        • Deal A.M.
        • Dueck A.C.
        • et al.
        Overall survival results of a trial assessing patient-reported outcomes for symptom monitoring during routine cancer treatment.
        JAMA. 2017; 318: 197-198
        • Denis F.
        • Lethrosne C.
        • Pourel N.
        • et al.
        Randomized trial comparing a web-mediated follow-up with routine surveillance in lung cancer patients [published correction appears in J Natl Cancer Inst. 2018;110(4):436].
        J Natl Cancer Inst. 2017; 109https://doi.org/10.1093/jnci/djx029
        • Fjell M.
        • Langius-Eklof A.
        • Nilsson M.
        • Wengström Y.
        • Sundberg K.
        Reduced symptom burden with the support of an interactive app during neoadjuvant chemotherapy for breast cancer- a randomized controlled trial.
        Breast. 2020; 51: 85-93
        • Mierzynska J.
        • Piccinin C.
        • Pe M.
        • et al.
        Prognostic value of patient-reported outcomes from international randomised clinical trials on cancer: a systematic review.
        Lancet Oncol. 2019; 20: e685-e698
        • Judson T.J.
        • Bennett A.V.
        • Rogak L.J.
        • et al.
        Feasibility of long-term patient self-reporting of toxicities from home via the internet during routine chemotherapy.
        J Clin Oncol. 2013; 31: 2580-2586
        • Ciani O.
        • Cucciniello M.
        • Petracca F.
        • et al.
        Lung Cancer App (LuCAPP) study protocol: a randomised controlled trial to evaluate a mobile supportive care app for patients with metastatic lung cancer.
        BMJ Open. 2019; 9e025483
        • Avery K.N.L.
        • Richards H.S.
        • Portal A.
        • et al.
        Developing a real-time electronic symptom monitoring system for patient after discharge following cancer-related surgery.
        BMC Cancer. 2019; 10: 463-478
        • Gandrup J.
        • Ali S.M.
        • McBeth J.
        • van der Veer S.N.
        • Dixon W.G.
        Remote symptom monitoring integrated into electronic health records: a systematic review.
        J Am Med Inform Assoc. 2020; 27: 1752-1763
        • Greer J.A.
        • Jacobs J.M.
        • Pensak N.
        • et al.
        Randomized trial of a smartphone mobile app to improve symptoms and adherence to oral therapy for cancer.
        J Natl Compr Canc Netw. 2020; 18: 133-141
        • Moazzami B.
        • Razavi-Khorasani N.
        • Dooghale-Modghadam A.
        • et al.
        COVID-19 and telemedicine. Immediate action required for maintaining healthcare providers well-being.
        J Clinic Virol. 2020; 126104345
        • Marandino L.
        • Necchi A.
        • Aglietta M.
        • Di Maio M.
        COVID-19 emergency and the need to speed up the adoption of electronic patient-reported outcomes in cancer clinical practice.
        JCO Oncol Prac. 2020; 16: 295-299
        • Schrag D.
        • Hershman D.L.
        • Basch E.
        Oncology practice during the COVID-19 pandemic.
        JAMA. 2020; 323: 2005-2006
      1. COVID-19 & telemedicine FAQs. AMA.
        • Waisberg F.
        • Enrico D.
        • Angel M.
        • Chacón M.
        Cancer treatment adaptations in the COVID-19 era.
        JCO Oncol Prac. 2020; 6: 305-307
        • Hay J.L.
        • Atkinson T.M.
        • Reeve B.B.
        • et al.
        Cognitive interviewing of the US National Cancer Institutes’ Patient-Reported outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE).
        Qual Life Res. 2013; 23: 257-269
      2. Basch E, Reeve BB, Mitchell SA, et al. Development of the National Cancer Institute’s patient-reported outcomes version of the common terminology criteria for adverse events (PRO-CTCAE). J Natl Cancer Inst. 2013;106(9):dju244.

        • Reeve B.B.
        • Mitchell S.A.
        • Dueck A.
        • et al.
        Recommended patient-reported core set of symptoms to measure in adult cancer treatment trials.
        J Natl Cancer Inst. 2014; 106dju129
        • Dueck A.C.
        • Mendoza T.R.
        • Mitchell S.A.
        • et al.
        Validity and reliability of the US National Cancer Institutes’ Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE).
        JAMA Oncol. 2015; 1: 1051-1059
        • Atkinson T.M.
        • Hay J.L.
        • Dueck A.C.
        • et al.
        What do “none,” “mild,” “moderate,” “severe,” and “very severe,” mean to patients with cancer? Content validity of PRO-CTCAE™ response scales.
        J Pain Symptom Manag. 2018; 55: e3-e6
        • Basch E.
        • Becker C.
        • Rogak J.
        • et al.
        Composite grading algorithm for the National Cancer Institute’s Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE).
        Clin Trials. 2021; 18: 104-114
      3. SAS version 9.4. SAS.
        www.SAS.com
        Date accessed: June 26, 2021
      4. SPSS version 27.0 IBM.
        • Zhou X.
        • Eid D.
        • Gnanasakthy A.
        Methods for reporting the patient-reported outcomes version of the common terminology criteria for adverse events (PRO-CTCAE) data in cancer clinical trials.
        Value Health. 2018; 21: S226
        • Basch E.
        Patient-reported outcomes-harnessing patients’ voices to improve clinical care.
        N Engl J Med. 2017; 376: 105-108
        • Benze G.
        • Nauck F.
        • Alt-Epping B.
        • et al.
        PROutine: a feasibility study assessing surveillance of electronic patient reported outcomes and adherence via smartphone app in advanced cancer.
        Ann Palliat Med. 2019; 8: 104-111
        • Wright A.A.
        • Raman N.N.
        • Staples P.
        • et al.
        The HOPE pilot study: harnessing patient-reported outcome and biometric data to enhance cancer care.
        JCO Clin Cancer Inform. 2018; 2: 1-12
        • Harle C.A.
        • Listhaus A.
        • Covarrubias C.M.
        • et al.
        Overcoming barriers to implementing patient-reported outcomes in an electronic health record: a case report.
        J Am Med Inform Assoc. 2016; 23: 74-79
        • Duncan E.A.
        • Murray J.
        The barriers and facilitators to routine outcome measurement by allied health professionals in practice: a systematic review.
        BMC Health Serv Res. 2012; 12: 96-105
        • Jensen R.E.
        • Rothrock N.E.
        • DeWitt E.M.
        • et al.
        The role of technical advances in the adoption and integration of patient-reported outcomes in clinical care.
        Med Care. 2015; 53: 153-159
        • Mooney K.
        • Berry D.
        • Whisenant M.
        • Sjoberg D.
        Improving cancer care through the patient experience: how to use patient-reported outcomes in clinical practice.
        Am Soc Clin Oncol Educ Book. 2017; 37: 695-704
        • Basch E.
        • Stover A.
        • Schrag D.
        • et al.
        Clinical utility and user perceptions of a digital system for electronic patient-reported symptom monitoring during routine cancer care: findings from the PRO-TECT trial.
        JCO J Clin Cancer Inform. 2020; 4: 947-957
        • Cleeland C.S.
        • Wang X.S.
        • Shi Q.
        • et al.
        Automated symptom alerts reduce postoperative symptom severity after cancer surgery: a randomized controlled clinical trial.
        J Clin Oncol. 2011; 29: 994-1000
        • Chung A.E.
        • Basch E.M.
        Incorporating the patient’s voice into electronic health records through patient-reported outcomes as the “review of symptoms”.
        J Am Med Inform Assoc. 2015; 22: 914-916
        • Miaskowski C.
        • Paul S.M.
        • Snowberg K.
        • et al.
        Stress and symptom burden in oncology patients during the COVID-19 pandemic.
        J Pain Symptom Manag. 2020; 60: e25-e33