Methodology and Data Collection Instruments

Our work seeks to answer the following research questions:

  1. How can digital/virtual learning experiences be designed and deployed such that they trigger interest in STEM? (design of triggers)
  2. What features of informal learning experiences best frame science learning and encourage reengagement with content over time? (framing of learning)
  3. What pedagogical strategies, as delivered by pedagogical agents, are most effective in promoting STEM interest and learning? (promoting interest)
  4. How can a technology infrastructure be used to monitor and track changes in STEM interest over time, specifically for groups who are underrepresented in STEM? (monitoring of interest)

As such we’ve assembled an array of research instruments to collect data to approach research questions from an affective, behavioral and cognitive perspective:

InstrumentResearch QuestionData and Analysis
Surveys1, 2, 3Astronomy knowledge, STEM interests and Minecraft experience, AI learning survey
Log data1, 2, 3, 4In-game observations and science tools usage, exploration and NPC/AI interaction patterns
Self Explanations4Conceptual and content knowledge checks in the form of self explanations (scoring) derived from Renninger and Hidi (2022)
Exoplanet drawings2Content analysis evidence of scientific knowledge and reasoning
Fieldnotes1, 2, 3What if questions, STEM career interest, engagement, observation of in-game AI agent use (guide/template)
Interviews1, 2, 3, 4Astronomy and Minecraft interest and knowledge (protocol available)
Explanations for why each of these instruments was chosen and how they’ve transformed over time can be found in <this publication Matt will link>