Data Collection and Measures
Quantitative
Study co-investigators from Qualis Health (JH, RH) created the
Population Health Information Technology Assessment (PHITA) based on PF
observations in prior work 13. The PHITA scale
assesses 1) the technical capability of software, and 2) the available
analytic skill set of the staff. Software Capability refers to data
capture and analytic functionality in the EHR or affiliated software.
Health Information Technology (HIT) Skill Set refers to the data
management, query writing and analytic skills available to the practice.
The categories were converted to 3-point scales in which 1 represented
the lowest preparedness level and 3 the highest. See Table 1 for
definitions. To assist the facilitator in prioritizing their work, the
assessment describes three levels of preparedness (low, medium, and
high) for each of these two sub-scales.
During the second half of H2N’s 15-month TA intervention (July -
December 2016), two of the authors (JH, RH) administered the PHITA by
interviewing all fifteen PFs so that each practice received sub-scale
and total PHITA score.
We compared each practice’s PHITA score with its ability to submit data
for ABCS CQMs. Ability to submit data was scored on a 5-point scale,
from 0 to 4, defined by the number of ABCS CQMs submitted during the
second quarter of 2017. H2N required a rolling 12-month lookback and
excluded patients from the denominator with no office visit within 12
months. Criteria for each CQM are described in the primary outcomes
paper 12. For the analyses we created a dichotomous
outcome variable for whether a practice was able to report two or more
(vs fewer than two) ABCS measures to improve interpretabality of the
findings.
To examine the relationship between PHITA scores and practice
characterstics, we classified clinics geographically as rural or urban
(RUCA codes 4-10 vs 1-3) 14, by practice size (solo
clinician, 2-4 clinicians, or ≥ 5 clinicians), and ownership
(independent, part of a hospital or health system, Federally Qualified
Health Center (FQHC)/migrant health clinic, or Tribal clinic).
Qualitative:
PFs documented all contacts with the 259 practices over the 15-month
intervention and at 18 and 21 months after a baseline visit
12 using a password-protected online-database.
Documentation included closed-ended and free-text entries after each
in-person visit, phone call, or email encounter. In total, 4128 contacts
were included in the database along with free-text documentation per
contact from 1-5 pages in length. Qualitative data included PF notes
about the practices’ QI and HIT progress and challenges, descriptions of
rapid process improvement cycles, and general comments. This
qualitative subanalysis only included practices that were scored as low
or high PHITA scores.
Analysis :
Quantitative
We used Chi-square tests to determine whether scores for the PHITA
subscales differed by practice characteristics. Spearman correlation
coefficients were used to assess relationships between the two PHITA
sub-scale scores. We used relative risk regression models to estimate
the association between the PHITA score (by domain and the total score)
and the ability of practice sites to report two or more CQMs for the
12-month period ending within the 2nd quarter of 2016.
Preliminary analyses indicated that independently owned practice sites
scored significantly lower than other ownership categories on both
sub-scales. We therefore stratified the analysis of the relationship
between a practice’s total PHITA score and its ability to produce two or
more CQM reports by independent vs non-independent ownership.
Qualitative
Members of the study team with qualitative research training and
expertise reviewed and coded all free text data for all contacts. An
inductive approach was used to develop the code list
15 and an iterative process of coding, comparing, and
discussing coding decisions was used to refine the code list and
increase shared understanding of the data and codes. Each coder was
assigned practices to code as sole coder. The coding team met regularly
to discuss coding questions. All coded data were entered into
Atlas.ti.16. To identify subthemes and synthesize the
data, two qualitative analysts (LT, KK) reviewed the coded data and
created a coding memo with a summary of the themes. The analysts met to
discuss the themes and used the final iteration to structure the
qualitative findings. Our methods for analyzing this rich qualitative
set of data were consistent with the consolidated criteria for reporting
qualitative research (COREQ) checklist for interview and focus groups
17.
KPWHRI’s Institutional Review Board approved this study.