Data Collection and Measures
Quantitative
Study co-investigators from Qualis Health (JH, RH) created the Population Health Information Technology Assessment (PHITA) based on PF observations in prior work 13. The PHITA scale assesses 1) the technical capability of software, and 2) the available analytic skill set of the staff. Software Capability refers to data capture and analytic functionality in the EHR or affiliated software. Health Information Technology (HIT) Skill Set refers to the data management, query writing and analytic skills available to the practice. The categories were converted to 3-point scales in which 1 represented the lowest preparedness level and 3 the highest. See Table 1 for definitions. To assist the facilitator in prioritizing their work, the assessment describes three levels of preparedness (low, medium, and high) for each of these two sub-scales.
During the second half of H2N’s 15-month TA intervention (July - December 2016), two of the authors (JH, RH) administered the PHITA by interviewing all fifteen PFs so that each practice received sub-scale and total PHITA score.
We compared each practice’s PHITA score with its ability to submit data for ABCS CQMs. Ability to submit data was scored on a 5-point scale, from 0 to 4, defined by the number of ABCS CQMs submitted during the second quarter of 2017. H2N required a rolling 12-month lookback and excluded patients from the denominator with no office visit within 12 months. Criteria for each CQM are described in the primary outcomes paper 12. For the analyses we created a dichotomous outcome variable for whether a practice was able to report two or more (vs fewer than two) ABCS measures to improve interpretabality of the findings.
To examine the relationship between PHITA scores and practice characterstics, we classified clinics geographically as rural or urban (RUCA codes 4-10 vs 1-3) 14, by practice size (solo clinician, 2-4 clinicians, or ≥ 5 clinicians), and ownership (independent, part of a hospital or health system, Federally Qualified Health Center (FQHC)/migrant health clinic, or Tribal clinic).
Qualitative:
PFs documented all contacts with the 259 practices over the 15-month intervention and at 18 and 21 months after a baseline visit 12 using a password-protected online-database. Documentation included closed-ended and free-text entries after each in-person visit, phone call, or email encounter. In total, 4128 contacts were included in the database along with free-text documentation per contact from 1-5 pages in length. Qualitative data included PF notes about the practices’ QI and HIT progress and challenges, descriptions of rapid process improvement cycles, and general comments.  This qualitative subanalysis only included practices that were scored as low or high PHITA scores.
Analysis :
Quantitative
We used Chi-square tests to determine whether scores for the PHITA subscales differed by practice characteristics. Spearman correlation coefficients were used to assess relationships between the two PHITA sub-scale scores. We used relative risk regression models to estimate the association between the PHITA score (by domain and the total score) and the ability of practice sites to report two or more CQMs for the 12-month period ending within the 2nd quarter of 2016. Preliminary analyses indicated that independently owned practice sites scored significantly lower than other ownership categories on both sub-scales. We therefore stratified the analysis of the relationship between a practice’s total PHITA score and its ability to produce two or more CQM reports by independent vs non-independent ownership.
Qualitative
Members of the study team with qualitative research training and expertise reviewed and coded all free text data for all contacts. An inductive approach was used to develop the code list 15 and an iterative process of coding, comparing, and discussing coding decisions was used to refine the code list and increase shared understanding of the data and codes. Each coder was assigned practices to code as sole coder. The coding team met regularly to discuss coding questions. All coded data were entered into Atlas.ti.16. To identify subthemes and synthesize the data, two qualitative analysts (LT, KK) reviewed the coded data and created a coding memo with a summary of the themes. The analysts met to discuss the themes and used the final iteration to structure the qualitative findings. Our methods for analyzing this rich qualitative set of data were consistent with the consolidated criteria for reporting qualitative research (COREQ) checklist for interview and focus groups 17.
KPWHRI’s Institutional Review Board approved this study.