Variables
There were a number of similarities between variables in the AIME and previously described models. For example; antiarrhythmic agents were significant in the model by McElnay [44] and antipsychotics in models by Trivalle and Nguyen et al. [12, 46]. Previous hospitalisation also featured in the model by Nguyen [46], and previous drug allergy was a variable in the GerontoNet ADR risk score, and the MOAT model [37, 41]. All models included the ‘number of medications’ as a predictor which highlights the importance of deprescribing.
A clinically informative model development strategy must assess a comprehensive range of variables and this was a strength of our study. To the best of our knowledge, no other study has included laboratory tests assessing coagulation indices such as INR and aPTT, yet they are commonly used by clinicians to identify high-risk patients for urgent assessment and diagnosis. By leveraging the digital capabilities of our hospital’s EMR we obtained a comprehensive dataset. This resulted in a final model which included serum sodium and INR, as predictors. Given that these variables have been identified as clinically important for patient prioritisation [24] and are modifiable, we anticipate that their inclusion will enhance the application of the AIME model and its’ translation into practice. Whilst variables such thromboembolism or supratherapeutic INR may occur later during the course of hospitalisation, the advantages of “real-time” digital surveillance means that as the patient’s variables fluctuate so does the probability of medication harm. This is consistent with the dynamic nature of patient risk and a predictive model such as the AIME can assist with ongoing clinical prioritisation.

Limitations

Despite our best attempt to develop a model with an EPV of 10 or greater we were unable to achieve this given the larger-than expected number of variables included in the multivariable modelling phase of our study. However, simulation studies have shown that lower EPVs can produce stable models, and that model instability occurs with EPVs below 4 [47].
For clinical applicability three variables (INR, serum sodium and number of medications) were categorised. To minimise the risk of bias we examined variables and selected thresholds based on optimal cut-off in ROC analysis (for number of medications), and clinical relevance as guided by pharmacist survey and focus groups.
Whilst we evaluated a comprehensive set of variables there were others, such as frailty which has been correlated with polypharmacy and non-adherence, that may have warranted inclusion[48]. However, at the time this study was conducted we were unable to quantify these variables with precision. Plans for external evaluation of the AIME model include testing these variables, as we anticipate they could improve model performance.
The retrospective nature of our study means that it is possible that medication-harm events were undetected. We chose this method as a practical means of evaluating harm in a reasonably sized patient cohort. Studies have used ICD-10 coding as an efficient approach to evaluate the impact of serious harm events in the hospital setting[49]. In addition to using ICD-10 codes, we comprehensively examined the medical records of flagged patients to detect any additional medication events that may not have been documented or coded. The hospital’s incident database was also reviewed to identify medication incidents that resulted in actual patient harm.
The AIME includes two variables (antipsychotic use and history of medication allergy) that marginally exceed the traditional 5% level of significance. This stems from the principal that prognosis (as opposed to causality assessment), is based on risk estimation, and thus it is statistically acceptable (and recommended) to include predictors with p-values greater than the 5% threshold. This is especially true if the predictor is known to be correlated with the outcome from prior research and its inclusion has a large effect size and enhances model performance [27]. Other predictive models such as the BADRI ADR score and the MOAT model have followed a similar approach [11, 37].

Future direction

The delivery of a comprehensive medication management services, for every inpatient, is not feasible in busy Australian public hospitals which serve a growing and aging population. Therefore, we must prioritise those at greatest risk of medication harm for timely interventions. To date, a limited number of risk prediction models have been developed for identifying hospital inpatients at high-risk of medication harm and none have undergone impact evaluation. Our study shows that the AIME model has an acceptable degree of predictive accuracy and potential clinical utility.
The availability of complete EMRs means that predictive models could be embedded into digital systems. This would enable patient risk to be iteratively estimated and monitored in real time using approaches such as surveillance dashboards, available to all clinicians at any time. The increasing availability of hospital digital data and machine learning methods of modelling provide an exciting opportunity to gain deeper insights and improve the quality of patient care [50]. The role of e-health in improving medication safety is predicted to be fundamental. Clinicians must embrace opportunities to utilise a hospital’s digital capabilities to mitigate avoidable harm, improve patient outcomes and optimise health system efficiencies, and the AIME model offers this opportunity.

Acknowledgements

The authors would like to express their sincere thanks and appreciation to Professor Bill Venables for his expert statistical guidance, and to Mr Karl Winckel and Dr Christopher Morris who assisted with review and rating of medication events.
References
(1) Budnitz, D.S., Lovegrove, M.C., Shehab, N. & Richards, C.L. Emergency hospitalizations for adverse drug events in older Americans.N. Engl. J. Med. 365 , 2002-12 (2011).
(2) Leendertse, A.J., Van Den Bemt, P.M.L.A., Bart Poolman, J., Stoker, L.J., Egberts, A.C.G. & Postma, M.J. Preventable Hospital Admissions Related to Medication (HARM): Cost analysis of the HARM study.Value. Health. 14 , 34-40 (2011).
(3) Classen, D.C., Pestotnik, S.L., Evans, R.S., Lloyd, J.F. & Burke, J.P. Adverse drug events in hospitalized patients: Excess length of stay, extra costs, and attributable mortality. JAMA.277 , 301-6 (1997).
(4) Dhingra, N. WHO Global Patient Safety Challange - Medication safety . <http://www.who.int/patientsafety/campaigns/en/> (2016). Accessed April 2017 2017.
(5) Roughead, E.E., Semple, S.J. & Rosenfeld, E. The extent of medication errors and adverse drug reactions throughout the patient journey in acute care in Australia. Int. J. Evid. Based. Health.14 , 113-22 (2016).
(6) Paradissis, C., Coombes, I.D., Donovan, P., Doran, E., Mckean, M. & Barras, M.A. The type and incidence of adverse drug events in ageing medical inpatients and their effect on length of hospital stay.JPPR. 47 , 347-54 (2017).
(7) Davies, E.C., Green, C.F., Taylor, S., Williamson, P.R., Mottram, D.R. & Pirmohamed, M. Adverse drug reactions in hospital in-patients: a prospective analysis of 3695 patient-episodes. PLoS. ONE.4 , e4439 (2009).
(8) O’Sullivan, D. et al. The impact of a structured pharmacist intervention on the appropriateness of prescribing in older hospitalized patients. Drugs. Aging. 31 , 471-81 (2014).
(9) Australian Commission on Safety and Qaulity in Healthcare (ACSQHC).Hospital-acquired complications . <https://www.safetyandquality.gov.au/our-work/indicators/hospital-acquired-complications/> (2019).
(10) Shekelle, P., Wachter, R.M., Provonost, P., Schoelles, K., Mcdonald, K. & et al. (2013). Making Health Care Safer 2: An Updated Critical Analysis of Patient Safety Practices. Comparative Effectiveness Review No. 211 (Agency for Healthcare Research and Quality, Rockville, MD, 2013).
(11) Tangiisuran, B. et al. Development and validation of a risk model for predicting adverse drug reactions in older people during hospital stay: Brighton Adverse Drug Reactions Risk (BADRI) model.PLoS. One. 9 , e111254 (2014).
(12) Trivalle, C., Burlaud, A. & Ducimetière, P. Risk factors for adverse drug events in hospitalized elderly patients: A geriatric score.Eur. Geriatr. Med. 2 , 284-9 (2011).
(13) Moons, K.G.M. et al. Risk prediction models: I. Development, internal validation, and assessing the incremental value of a new (bio)marker. Heart. 98 , 691e8 (2012).
(14) Anderson, K.M., Odell, P.M., Wilson, P.W. & Kannel, W.B. Cardiovascular disease risk profiles. Am. Heart. J. 121(1 Pt 2) , 293-8 (1991).
(15) Lane, D.A. & Lip, G. Use of the CHA2DS2-VASc and HAS-BLED Scores to Aid Decision Making for Thromboprophylaxis in Nonvalvular Atrial Fibrillation. Circulation. 126 , 860–5 (2012).
(16) Pisters, R., Lane, D.A., Nieuwlaat, R., de Vos, C.B., Crijns, H. & Lip, G. A novel user-friendly score (HAS-BLED) to assess 1-year risk of major bleeding in patients with atrial fibrillation. The Euro Heart Survey. Chest 138 , 1093-100 (2010).
(17) Falconer, N., Barras, M.A. & Cottrell, W.N. Systematic review of predictive risk models for adverse drug events in hospitalised patients.Bri. J. Clin. Pharmacol. 84 , 846-64 (2018).
(18) Bates, D.W., Boyle, D.L., Vander Vliet, M.B., Schneider, J. & Leape, L. Relationship between Medication Errors and Adverse Drug EventsJ. Gen. Intern. Med. 10 , 199-205 (1995b).
(19) Edwards, I.R. & Aronson, J.K. Adverse drug reactions: definitions, diagnosis and management. Lancet. 356 , 1255-9 (2000).
(20) Aronson, J.K. & ferner, R.E. Classification of Terminology in Drug Safety. Drug. Saf. 28 , 851-70 (2005).
(21) Hallas, J. et al. Drug related hospital admissions: the role of definitions and intensity of data collection, and the possibility of prevention. J. Intern. Med. 228 , 83-90 (1990).
(22) Morimoto, T., Gandhi, T.K., Segar, A.C., Hsieh, T.C. & Bates, D.W. Adverse Drug Events and Medication Errors: detection and classification methods. Qual. Saf. health. Care. 13 , 306-14 (2004).
(23) Schumock, G.T. & Thornton, J.P. Focusing on the preventability of adverse drug reactions. Hosp. Pharm. 27 , 538 (1992).
(24) Falconer, N., Barras, M. & Cottrell, N. How hospital pharmacists prioritise patients at high risk for medication harm. Res. Soc. Admin. Pharm. 15 , 1266-73 (2018).
(25) Australian Medicines Handbook 2017 (online) (Australian Medicines Handbook Pty Ltd: Adelaide, 2017).
(26) Levey, A.S. et al. A new equation to estimate glomerular filtration rate. Ann Intern Med 150 , 604-12 (2009).
(27) Steyerberg, E.W. Clinical Prediction Models : a practical approach to development validation and updating. (2009).
(28) Moon, K.G.M. et al. Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies: The CHARMS Checklist. PLoS. Med. 11 , e1001744 (2014).
(29) Heinz, G., Wallisch, C. & Dunkler, D. Variable selection – A review and recommendations for the practicing statistician.Biometrical 431-49 (2017).
(30) R Core Team. R: A language and environment for statistical computing. (R Foundation for Statistical Computing, Vienna, Austria, 2018).
(31) R: A language and environment for statistical computing. . (R Core Team (2013), R Foundation for Statistical Computing, Vienna, Austria., 2013).
(32) Collins, G.S., Reitsma, J.B., Altman, D.G. & Moons, K.G.M. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD Statement. Eur. J. Clin. Invest. 45 204–14 (2015).
(33) Parekh, N. et al. Medication-related harm in older adults following hospital discharge: development and validation of a prediction tool. BMJ. Qual. Saf. , bmjqs-2019-009587 (2019).
(34) Vickers, A.J. & Elkin, E.B. Decision Curve Analysis: A Novel Method for Evaluating Prediction Models. Med. Decis. Making.26 , 565-74 (2006).
(35) Scott, I.A., Hilmer, S.N. & Reeve, E. Reducing Inappropriate Polypharmacy: The Process of Deprescribing. JAMA. Intern. Med.175 , 827-34 (2015).
(36) Onder, G. et al. Development and validation of a score to assess risk of adverse drug reactions among in-hospital patients 65 years or older: The GerontoNet ADR risk score. Arch. Int. Med.170 , 1142-8 (2010).
(37) Geeson, C., Wei, L. & Dean Franklin, B. Development and performance evaluation of the Medicines Optimisation Assessment Tool (MOAT): a prognostic model to target hospital pharmacists’ input to prevent medication-related problems,. BMJ. Qual. Saf. , 1-12 (2019).
(38) Sakuma, M., Bates, D.W. & Morimoto, T. Clinical prediction rule to identify high-risk inpatients for adverse drug events: the JADE Study.Pharmacoepidemiol. Drug. Saf. 21 , 1221-6 (2012).
(39) O’Mahony, D., O’Connor, M.N., Eustace, J., Byrne, S., Petrovic, M. & Gallagher, P. The adverse drug reaction risk in older persons (ADRROP) prediction scale: derivation and prospective validation of an ADR risk assessment tool in older multi-morbid patients. Eur. Ger. Med. 9 , 191-9 (2018).
(40) Bates, D.W. et al. Incidence of Adverse Drug Events and Potential Adverse Drug Events: Implications for Prevention. JAMA274 , 29-34 (1995).
(41) Onder, G. et al. Development and validation of a score to assess risk of adverse drug reactions among in-hospital patients 65 years or older: the GerontoNet ADR risk score. Arch. Intern. Med.170 , 1142-8 (2010).
(42) Schneider, J.K., Mion, L.C. & Frengley, J.D. Adverse drug reactions in an elderly outpatient population. Am. J. Hosp. Pharm. 49 , 90-6 (1992).
(43) Parameswaran Nair, N., Chalmers, L., Peterson, G.M., Bereznicki, B.J., Castelino, R.L. & Bereznicki, L.R. Hospitalization in older patients due to adverse drug reactions – the need for a prediction tool. Clin. Interv. Aging. 11 , 497-505 (2016).
(44) McElnay, J.C., McCallion, C.R., Al-Deagi, F. & Scott, M.G. Development of a Risk Model For Adverse Drug Events in the Elderly.Clin. Drug. Investig. 13 , 47-55 (1997).
(45) Kaiser, E.A., Lotze, U. & Schäfer, H.H. Increasing complexity: which drug class to choose for treatment of hypertension in the elderly?Clin. Interv. Aging. 9 , 459-75 (2014).
(46) Nguyen, T. et al. Improving medication safety: Developement and impact of a multivariate model-based strategy to target high-risk patients. PLOS. One. 12 , 1-13 (2017).
(47) Vittinghoff, E. & McCulloch, C.E. Relaxing the rule of ten events per variable in logistic and Cox regression. Am. J. Epidemiol.165 , 710-8 (2007).
(48) Scott, I.A., Gray, L.C., Martin, J.H. & Mitchell, C.A. Minimizing Inappropriate Medications in Older Populations: A 10-step Conceptual Framework. Am. J. Med. 125 , 529-37.e4 (2012).
(49) Walter, S.R., Day, R.O., Gallego, B. & Westbrook, J.I. The impact of serious adverse drug reactions: a population-based study of a decade of hospital admissions in New South Wales, Australia. Br. J. Clin. Pharmacol. 83 , 416-26 (2017).
(50) Hutchinson, L. et al. Models and Machines: How Deep Learning Will Take Clinical Pharmacology to the Next Level. Clin Pharmaco. Ther. 8 , 131-4 (2019).