RESPONSIBLE AI FOR PALLIATIVE CARE.
BACkground.
This project constituted the majority of my Master’s thesis in Human Computer Interaction. It involved the design of an AI driven decision-making assistance tool for use in palliative care following a series of user interviews with palliative care clinicians and hospital directors and culminated in a series of User Requirements and Key Features for this prospective tool.
Before undertaking this project, I found that the lion’s share of research devoted to Artificial Intelligence in the medical domain was tackling issues taking place earlier in a patient’s illness timeline, such as tumor detection on CT scans and management of patient’s Electronic Health Records (EHR).
The goal of this project was to extend AI’s predictive and analytical capabilities further along down a patient’s illness timeline, and provide palliative care clinicians assistance with the often difficult decisions they make on a daily basis, as well as enhance their ability to communicate these decisions to non-specialist clinicians and patients alike.
Any decision where a clinician must weigh the possible benefits of an intervention with the stress caused by performing it could benefit from better predictions, for example: whether or not systemic treatments like palliative chemotherapy will result in a quicker decline in quality of life than the natural progression of the disease itself for a given patient.
Timeline: 2.5 Months
ReSEARCH.
Upon completing my extensive literature review, I decided that given the three month time constraint, User Interviews would be the best method for gathering a rich picture from which to base my User Requirements, as methods like Ethnography would take excess time, and require placement in a palliative care ward to conduct observations, which would have involved lengthy ethical considerations that would have undoubtedly ground progress to a halt.
After initial testing and two rounds of interview piloting, I reached out to the brilliant team at Nottingham City Hospital’s Hayward House in-patient palliative care unit, who agreed to participate in the research as my primary stakeholders. Five 30-minute semi-structured interviews took place over the course of a month and a half, resulting in 5 anonymized transcripts.
These transcripts were then coded using NVivo, and prepped for Thematic Analysis. The reason I chose Thematic Analysis was twofold: 1. Each theme would serve as the basis for on, or several user requirements, 2. All codes and their respective themes are transparent, which enhances credibility.
RESULTS.
Following Thematic Analysis, I codified my results into 3 overarching main themes: “Specialized Knowledge” which highlighted the consultatory practices of palliative care clinicians, their highly varied patient cohort, and the implicit measures they make to assess a patient’s status that aren’t traditionally reflected in available clinical data. “Patient over Treatment” focused on the values placed on comfort and quality of life/death in palliative care, and the necessity of reducing overzealous and unnecessary curative treatments that hamper the patient’s final moments. Finally, “Bridging the Gap” assessed the current practical limitations on staffing, ethical considerations, and data collection and storage practices.
These 3 themes formed the basis of an initial 8 User Requirements:
1. Contextual Output – High Priority
As mentioned in the Thematic Analysis, it is vitally important that the output for any predictions made by an AI decision making tool reflect the person who will act on them, either the palliative care clinician, a member of a parent team, or the patient and their family, each output has its own specific requirements so that it can both be understood and acted upon effectively. These outputs can be divided into two categories: clinician facing, and patient facing. Both categories share similar features yet differ in small but key areas. The best modality for the output of a prediction in both categories would be in the form of a bar chart, with the length of each bar indicating the commonality of that symptom or feature in the input data with the training and sample database, and a color indicating its severity in impacting the resultant prediction, with red denoting severe impact, yellow for moderate impact, and green denoting no impact. When displayed to a clinician, these factors or symptoms can be associated with a corresponding integer percentage showing similarity in a more accurate way than simple bar length, but these percentages should not be shown when communicating the output to a patient, as percentage values have been shown in the Thematic Analysis to have a negative impact on patients whose options are limited.
2. Transparency and Trust – High Priority
Following on from Contextual Output, Transparency and Trust are vital for the sustained and effective use of an AI decision-making tool. The decisions that an AI will assist with via predicting possible outcomes may very well involve life or death decisions and have direct impact on the quality of life for any patient affected by said predictions. The severity underlying the context of use for any tool like this one highlights why it is so important for clinicians and patients to be able to trust both the results and the model’s behavior, as an incorrect prediction or unpredictable behavior may have a profound impact on budgets, clinician time, and most importantly, patient lives. Contextual Output is vital for building trust, as simple integer readouts on probabilities would be far too opaque. Allowing for users to see what factors influenced a decision and by how much means that a non-specialist operator can see how an AI is behaving and make a judgement on the AI’s predictions based on the factors it considers relevant to the output.
3. Bi-Directional Input – Medium Priority
Whilst outputs are incredibly important for the applicability of this technology for the end-user, inputs are similarly relevant to the usability and viability of large-scale implementations. For all treatment decisions, both parent teams and palliative care specialists strive to involve the patient in both the discussion and application of chosen treatments, as well as relying on the patient’s recognition of their condition as a basis to launch into end-of-life discussions and weighing the benefits and detriments of any symptom control based on how patients feel, as well as how the wish to feel in the future. To account for this patient-centered treatment focus, ADMT should provide a conversational model so patients can have equal stake in the data analyzed by the AI when making a prediction, or for general live monitoring purposes. Without a way for patients to have some stake, clinicians may run the risk of overreliance on EHR data, which may prove to alienate the patient from their treatment, relegating them to a series of data points rather than a stakeholder.
4. Mixed Media Input – Medium Priority
Similar to including a conversational model powered by NLP, an AI tasked with predicting outcomes of such importance must be able to analyze all forms of input: natural language, images, sensor data, time data, numerical data, etc. As any predictions made that do not reflect the data rich environment present in the real world would not be able to effectively meet the expectations of clinicians who already perform those same predictions without the use of artificial intelligence on a daily basis, rendering any assistance delivered in a subpar manner redundant.
5. Dataset Regulation – High Priority
The ability to provide an Ai decision making tool with an accurate, up-to-date, and standardized dataset for training is essential for accurate predictions. Currently datasets are heterogeneous and disorganized, with one hospital network using different recording practices than another. This has to be remedied, as a predictive intelligence requires a large enough pool of data to be able to make accurate predictions, and when accuracy is so paramount the quantity of available data has to be sufficient to allow for accurate generalizations and predictable behavior. Without data regulations in place for EHR and other monitoring data, the pool of available data is far too small and irregular to be able to make any reasonably accurate predictive AI that can be deployed on a large scale.
6. Patient Reflective Training Data – Medium Priority
In addition to a regulated and standardized data set, training data must be reflective of the unique cohort of patients present in palliative care clinics. These patients have several differentiating factors that make them unique from the general population of patients in the average hospital ward. Patients in palliative care are already severely compromised, facing death, discomfort, and further deterioration, this makes treatment of these patients difficult, as traditional curative techniques may not be reasonable to apply to patients who have no potential for recovery. In order for an AI to weight the pros and cons of a prospective treatment, it must be aware of these limitations, for example: when treating a secondary infection during a hospital stay for a patient undergoing a systemic treatment like chemotherapy or radiotherapy, it is often advisable to prescribe antibiotics to attempt to cure the infection, however, when facing a patient whose death is guaranteed, its often more advisable to treat the symptoms of such an infection and providing comfort, rather than risk putting the dying patient in an even more compromised situation. This is why it is essential for an AI to be trained on data from patients who are in palliative care, not just on the general population of admitted patients, as often it is not the cure that is most important, but how it affects that person’s remaining time.
7. Implicit Measures – Medium Priority
There are several oversights in traditional monitoring and data recording strategies when seeking to use this data to train an AI to make predictions on patient outcomes. In many cases, palliative care clinicians must make judgements on patient deterioration when all traditional data monitoring tools are giving identical readouts for two patients who would face completely different outcomes if similar treatments were pursued. They must rely on implicit, less binary status measures that are not traditionally considered by software led approaches. For instance, one participant stated that they will use clinic attendance as a marker for deterioration and performance status when facing a patient whose data is inconclusive, whether or not that patient made it to their office, or if they had to attend clinic virtually if at all. AI must be cognizant of these measures, and data monitoring must also improve to include a wider variety of possible data points to increase the accuracy of any solution that relies upon said data to function.
8. Live Monitoring – High Priority
Timeliness, responsiveness, and accessibility are all key to ensuring regular use of AI tools in palliative medicine. To facilitate this, AI must be ever present and ever vigilant, as relying on the human element to proactively engage with the technology may result in the same pitfalls as contemporary analog solutions, for example: referral timeliness may not improve due to clinician failure to recognize the appropriate situation to apply the tool, similar to the checklists used today. This limitation could be remedied through an increase in technological literacy, but that may be costly and time consuming. Instead, the AI should be proactive in engaging with clinicians and patients alike, taking the onus of recognizing deterioration out of the already overburdened hands of parent teams and placing it in the hands of an always-on solution.