game research methodologies

This note last modified September 1, 2024
  • used theories of learning to analyze existing eSport helpers
  • qualitative, thematic, analysis of mental models of AI
  • created an embedding to determine if someone was in elo hell
  • research through design on the creation of a queer game
  • inductive deductive grounded theory to analyze Minecraft server rules
  • developed a tool that automatically generates cutscenes
  • RTD and interviews on a reflection game
  • mixed methods to analyze team communication in LoL, for the qualitative bit they used Lag Sequence Analysis which is used to understand team communication.
  • mixed methods to analyze daily quests. Inductive thematic analysis and a survey using Vallerand's passion scale.
  • PRISMA method and inductive thematic analysis to determine a definition of eSports
  • Used physiological measures on 30 participants to find evidence that hexad-based gamification personalization was effective
  • measured performance of ~250ish people playing a shooting game under a variety of latency conditions.
  • built a VR city builder game, saw whether gamers felt supported by PCG. n=55, used surveys to determine self-reported feelings of engagement and creativity support.
  • scraped reddit comments and created a survey to determine community feelings on performance enhancing drugs in eSports.
  • Used a quiz on ~20 participants to determine whether their privacy policy game was effective in teaching. They also did an interview, but that section wasn’t that long.
  • Semi structured interviews of 27 game developers.
  • Designed wearables and tested w/ WOZ study + thematic analysis for the interviews
  • Iterative design w/ participants to develop anti-toxicity tools. Used some interesting proxies rather than implement their tools in a real game.
  • the methods were beyond me, but they essentially tried to make a gameful version of a validated survey.
  • Made an impressive prototype game to study visual cues in VR. 20 participants, used a visual cues survey, and a qualitative analysis method (that I couldn’t understand because it was written in German)
  • Qualitative interviewing of 20ish participants to analyze a board game about gut health.
  • mobile AR game, 91 students in classrooms, interviews and surveying.
  • made two game about attention, one with integrated design, one without, and looked at a variety of scores.
  • Measured reaction timing of 30 participants when using an LED strip to aid in their gaming experience. PXI and a bunch of other questionnaires.
  • Did a pilot community survey, interviewed an expert, and had 13 participants look at a prototype + qualitative data analysis.
  • Used System Usability Scale and their own questionnaire with 16 participants to analyze a new behavior tree visualization tool.
  • Had 25 people take some presence and sickness questionnaires when looking at a VR locomotion technique. (also created custom questions)
  • Content analyzed 2.8K reddit posts about pay to win unfairness.
  • Put a game about civility into 6 11th grade classes, had some qualitative feedback (also measured time played outside class as a metric of general fun). They made some mathematical models that I didn’t really understand because I skimmed.
  • Analyzed reddit posts, used machine learning to do pre-processing, followed by thematic analysis.
  • Thematic analysis of 32 codes of conduct
  • Semi structured interviews with 15 pinball players.
  • Qualitative survey of 91 people about whether they’d want to influence a streamer’s environment (lighting, gadgets, etc.). Afterwards, participants acted as streamers and they WOZ’d an audience that controlled their setup.
  • Measured physiology of a variation on beatsaber, N=54, some qualitative.
  • Looked at video recordings of people eating together in Covid to understand how people play in these scenarios. Used interaction analysis (Jordan, Henderson, 1995 interaction analysis journal of learning sciences)
  • Built stuffed animal game controllers and had 15 participants go through TLX / CSI
  • Used functional near infrared spectroscopy to look at cognitive effort in playing video games.
  • Narrative literature review looking at how values affect various parts of the game development life cycle.
  • Created an eSport game database, evaluated usability with 4 esport experts.
  • Development and evaluation of an 11 item version of the player experience inventory.

Empirical-Qualitative

  • used theories of learning to analyze existing eSport helpers
  • qualitative, thematic, analysis of mental models of AI
  • inductive deductive grounded theory to analyze Minecraft server rules
  • PRISMA method and inductive thematic analysis to determine a definition of eSports
  • Semi structured interviews of 27 game developers.
  • Did a pilot community survey, interviewed an expert, and had 13 participants look at a prototype + qualitative data analysis.
  • Content analyzed 2.8K reddit posts about pay to win unfairness.
  • Analyzed reddit posts, used machine learning to do pre-processing, followed by thematic analysis.
  • Thematic analysis of 32 codes of conduct
  • Semi structured interviews with 15 pinball players.
  • Looked at video recordings of people eating together in Covid to understand how people play in these scenarios. Used interaction analysis (Jordan, Henderson, 1995 interaction analysis journal of learning sciences)

Empirical-Quantitative

  • Used physiological measures on 30 participants to find evidence that hexad-based gamification personalization was effective
  • measured performance of ~250ish people playing a shooting game under a variety of latency conditions.
  • built a VR city builder game, saw whether gamers felt supported by PCG. n=55, used surveys to determine self-reported feelings of engagement and creativity support.
  • made two game about attention, one with integrated design, one without, and looked at a variety of scores.
  • Measured reaction timing of 30 participants when using an LED strip to aid in their gaming experience. PXI and a bunch of other questionnaires.
  • Used System Usability Scale and their own questionnaire with 16 participants to analyze a new behavior tree visualization tool.
  • Had 25 people take some presence and sickness questionnaires when looking at a VR locomotion technique. (also created custom questions)
  • Used functional near infrared spectroscopy to look at cognitive effort in playing video games.

Empirical-Mixed Methods

  • mixed methods to analyze team communication in LoL, for the qualitative bit they used Lag Sequence Analysis which is used to understand team communication.
  • mixed methods to analyze daily quests. Inductive thematic analysis and a survey using Vallerand's passion scale.
  • scraped reddit comments and created a survey to determine community feelings on performance enhancing drugs in eSports.
  • Used a quiz on ~20 participants to determine whether their privacy policy game was effective in teaching. They also did an interview, but that section wasn’t that long.
  • Made an impressive prototype game to study visual cues in VR. 20 participants, used a visual cues survey, and a qualitative analysis method (that I couldn’t understand because it was written in German)
  • Put a game about civility into 6 11th grade classes, had some qualitative feedback (also measured time played outside class as a metric of general fun). They made some mathematical models that I didn’t really understand because I skimmed.
  • Qualitative survey of 91 people about whether they’d want to influence a streamer’s environment (lighting, gadgets, etc.). Afterwards, participants acted as streamers and they WOZ’d an audience that controlled their setup.
  • Measured physiology of a variation on beatsaber, N=54, some qualitative.

Artefact-Technical

  • created an embedding to determine if someone was in elo hell
  • the methods were beyond me, but they essentially tried to make a gameful version of a validated survey.
  • Created an eSport game database, evaluated usability with 4 esport experts.

Artefact-Design

  • research through design on the creation of a queer game
  • RTD and interviews on a reflection game
  • Iterative design w/ participants to develop anti-toxicity tools. Used some interesting proxies rather than implement their tools in a real game.
  • Qualitative interviewing of 20ish participants to analyze a board game about gut health.
  • mobile AR game, 91 students in classrooms, interviews and surveying.
  • Built stuffed animal game controllers and had 15 participants go through TLX / CSI

Theoretical

  • Narrative literature review looking at how values affect various parts of the game development life cycle.
  • Development and evaluation of an 11 item version of the player experience inventory.

Meta-Research

Thoughts

  • lots of quant studies with smaller research sizes than I thought.
  • qual studies usually had 20ish people
  • quant studies had surprisingly variable
  • surveys usually had ~100ish people

Methods

  • Wizard of Oz
  • Thematic analysis
  • Grounded theory
  • Validated Survey
logo