Procedure
The people with the desired knowledge in this study are
This study will employ techniques from the field of cognitive task analysis (CTA), a field that specializes in understanding how “cognition makes it possible for humans to get things done and then turning that understanding into aids–low or high tech–for helping people get things done better” (Crandall, Klein, and Hoffman 2006, 2). Cognitive task analysis attempts to capture experts’ thoughts, what they pay attention to, their decision-making strategies, their goals, and what they know about how a process works. The three components of CTA are knowledge elicitation, data analysis, and knowledge representation ( 7).
Structured interviews are a common method used for CTA data elicitation. They allow for a diverse range of issues and skills to emerge, but are best treated as exploratory data (Crandall, Klein, and Hoffman 2006, 13), which is appropriate as this project is an initial systematic look at textbook development across regions and languages. This project will use a modified version of the Critical Decision Method, an elicitation technique that employs case-specific, multitrial retrospection from the participant’s own experience to organize the interview (Hoffman, Crandall, and Shadbolt 1998, 257). Participants are guided through a series of steps, including:
Incident Selection
The first question describes a particular situation and asks for an example where the participant’s decision-making changed the outcome, or in which the participant’s skills were challenged
Incident Recall
The participant tells the story of the situation, while the researcher allows the participant to structure the account
Incident Retelling
The researcher tells the story back to the participant, at which point, the participant may share additional details and correct misunderstandings
Time Line Verification and Decision Point Identification
The participant goes back over the incident to give a time line that identifies when key decision points occurred
Progressive Deepening
Researcher asks probe questions about the information and cues that the participant used to make decisions, as well as seeking clarification about the participant’s goals and actions. Probe questions may include “What information did you use in making this decision?” What were your specific goals at the time?” or “Were you reminded of any previous experience?”
“What-if?” Queries
The researcher asks the expert to reflect on what might have happened differently if they’d made different decisions at key points, as well as what information they would have needed in order to make more informed decisions. (Hoffman, Crandall, and Shadbolt 1998, 272)
There are many strengths to this method. The first is that experts love to tell stories. They may even expect younger practitioners to learn from these stories (Hoffman, Crandall, and Shadbolt 1998, 269), so my position as a younger member of the field of international education establishes me as a natural listener, a person who is seeking to understand their experiences. Another is that structured interviews have been found to be somewhat more efficient than unstructured interviews, and the probe questions yielded significantly more information about the expert’s cognitive activities and which cues they paid attention to ( 265, 269). The third is that by allowing participants to structure the interview around a specific event in their own lives, the researcher’s biases are minimized ( 267).
This study will use a modified version of the Critical Decision Method to collect semi-structured interview data. The classical form of the method focuses on special incidents, where the expert changed the course of the project, or where their skills were specially challenged. While this has advantages, it may also bias the project that the experts describe towards especially easy or difficult textbooks. Instead, I choose to elicit projects “that used all of the expert’s skills,” a phrase intended to elicit projects that are occur within a “normal” range, but still have interesting characteristics.
Because of the geographic dispersion of potential participants, most interviews will be conducted using video technology such as Skype or Google Hangouts. Although the body of research surrounding online interviews is small, it is growing (Deakin and Wakefield, 2014, 604). Seitz (2015) found that video-chat technology has been successfully used for projects with less-personal topics such as study abroad experiences ( 5). Because this study involves mostly professional decisions, rather than personal life experience, Skype is an appropriate choice. It is recommended in cases where participants are too far apart to allow for face-to-face interviews ( 2). This method minimizes financial costs, and in one study, several participants actually chose to interview over Skype, indicating that its use was normal to them (Deakin and Wakefield, 2014, 607).
Although the limited contact inherent in video interviews can make it harder to establish rapport, some participants felt more comfortable because they could participate while staying in their own space (Seitz, 2015, 2). Research suggests that slowing down and clarifying talk, repeating answers and questions, paying close attention to facial expressions, and e-mailing before the interview to establish rapport can all contribute to more successful interviews (Seitz, 2015, 5). Deakin and Wakefield found that Skype participants were somewhat more likely to be absent at their interview time, and suggest that this effect can be lessened by establishing rapport with participants through e-mailing several times before the interview.
Because not all potential participants will have access to Skype, and some of those that do may not have enough bandwidth to use the video function (Seitz, 2015, 5), where necessary, interviews will be conducted with audio only or over the phone. Deakin and Wakefield found that when video and audio were both non-functional, they were still able to carry on with the interview by using Skype’s text function (2015, 611). While this is not ideal, it will be done as necessary to capture the broadest amount of information possible.