5.2 Evaluation
A round of Aqua Republica consists of 14 turns that represent 14 years (from 2016-2030). Each participant was given a pre-game and a post-game survey. The pre-game survey was used to obtain basic data such as age/gender/profession etc. as well as to gauge participants interest in attending the simulation gaming event and their familiarity with technology. The post-game survey was given to learn any post game insights about the game and their team interactions. Both surveys were modified from Zhou’s 2012 thesis on serious games, The Princess in the Castle (2012) with additional questions added by the research team.
The pre-game survey was created to obtain baseline background information from the participants. It also included a questionnaire aimed at determining people’s openness to participating in a game simulation as well as to determine their level of comfort using novel technologies such as serious games. These were included to determine whether technological expertise is a barrier to interactions in a simulation gaming setting.
The post-game survey asked participants to reflect on their interactions with other members of their groups. It also asked participants about their prior interactions with other members of their teams. This is valuable information as it allowed researchers to determine the effect familiarity had on group dynamics (i.e. do participants tend only to interact with those they have pre-existing relationships with).
5.2.1 Interaction Analysis
Interaction analysis (IA) was performed using audio or video recordings from the events. The IA performed was based on the Jordan and Henderson (1995), Interaction Analysis: Foundations and Practice. IA refers to an interdisciplinary method in which an empirical investigation of interactions between players is performed (Jordan and Henderson, 1995). This is done by observing speech, nonverbal interactions and the use of artifacts and technology (Jordan and Henderson, 1995). For the events in the Maritimes only audio recordings are available due to certain participants feeling uncomfortable being filmed.
Interactions were measured over the course of three phases. The first phase representing the first 20-minutes of gameplay, the second phase representing a 20-minute period in the middle of the game event and the final phase representing the final 20-minutes of game play. Interactions, in terms of this analysis, are classified in two ways, directed interactions and group interactions:
* • Directed interactions are defined as: Any interaction between two individuals. Usually involving direct eye-contact and either take the form of a question or a response to a question.
* • Group interactions are defined as: Any interaction by one individual that is not specifically addressed to another individual i.e. broad, open questions; general statements about the game.
Collaborative interactions such as the ones that occur while playing Aqua Republica are extremely valuable. Barnes and Todd (1977) explain that:
[Collaborative] moves are mutually supportive: by taking the trouble to elicit an opinion from someone else, or by utilizing what has been said by extending it further, the group members ascribe meaningfulness to one another’s attempts make sense of the world.
These collaborative moves as Barnes and Todd (1977) refer to them are of extreme importance in terms of social learning. By interacting with other participants, players can gain insight into others’ world-views, perspectives and values.
5.2.3 Social Network Analysis
Players were also asked in surveys about their previous interactions with other participants. This information was used to perform a simple social network analysis allowing a visually representation of how well participants knew each other as well as a method of identifying any gaps within the group that need to be worked on in the future. GEPHI software was used again to visually display the social networks. Pre-existing relationships may also alienate participants who are less familiar with each other.
6. Results
This section of the paper presents the data and results from each of the research activities conducted before, during and after the game simulation events. Each sub section will address the first four research questions (i to iv) that have been developed to achieve the overall objective of this study. All research questions for this study are interrelated and embedded, the fifth and last research question (v) will be addressed in more detailed in the Discussion Section while it brings all results and insights from the earlier research questions together.
The research questions themselves follow a hierarchical flow back to the main research goal of determining whether serious games can provide a space for social learning to occur and for Nonaka and Takeuchi’s (1994) knowledge conversion cycle to take place. They are best visualized using the graphic below.