What-If Hypothetical Implementations in Minecraft (WHIMC)
WHIMC is a National Science Foundation-funded research project and interdisciplinary collaboration between several organizations dedicated to cutting-edge and impactful informal learning.
The goal of WHIMC is to develop computer simulations that engage, excite, and generate interest and engagement in STEM. WHIMC leverages Minecraft Java Edition as a learning environment for learners to interactively explore the scientific consequences of alternative versions of Earth via “what if?” questions, such as “What if the earth had no moon?” or “What if the earth were twice its current size?” as well as other emerging astronomy inquires, like “What would it take to terraform mars?” or “How do we mine asteroids?” Kids get to explore these as aspiring scientists and engineers on an interactive server that changes to match their interests. Check out our worlds! Or read more about the research.
News
May 2024
Our student Arunima Suri presented her work helping us to do user research and curricular design at the University of Illinois Undergraduate Research Showcase this past week. She helped us to create a content guide for the new Barrelbot.
April 2024 – Recent AI publications and videos
We’ve been continuing to develop our AI-driven learner support tools into 2024, here’s the latest:
Hum, S. Gadbury, M., Shipley, E., Lane, H.C., Ginger, J. (2024). Mars, Minecraft, and AI: A Deep Learning Approach to Improve Learning by Building. 25th International Conference on Artificial Intelligence in Education, Recife, Brazil.
Mahajan, J., Hum, S., Henhapl, J., Yunus, D., Gadbury, M,. Brown, E., Ginger, J., Lane, H.C. (2024). MineObserver 2.0: A Deep Learning & In-Game Framework for Assessing Natural Language Descriptions of Minecraft Imagery. Association for the Advancement of Artificial Intelligence 2024 Conference, Vancouver.
March 2024 – Public Events
WHIMC doesn’t just do summer camps – we joined forces with Fiske Planetarium and Science Discovery for interactive demos at STEAM Fest this past week. Hundreds of people attended but we didn’t have time to take photos during all the action.
November 2023 – Introducing Barrelbot
An entirely new way to learn how to code, built on the WHIMC platform – WAY better for collecting data about learners and enabling them to build and collaborate as part of the process of learning to code.
We ran our first pilot camp with it with our partners in Hemet, CA.
September 2023 – Summer Camp Recap
Another successful summer of WHIMC camps! 80 participants made over 1500 observations and 2000 science tools measures across 6 summer camps. Several changes this time though:
We ran camps with our youngest learners yet (grades 4-6) after some adjustment to curriculum based on emphasis on Universal Design Learning and Social Emotional Learning.
Our partnership with the Science Discovery program at CU Boulder continues through training several staff at Fiske Planetarium to run self-sustaining camps beyond the conclusion of our grant.
Kids had a chance to make use of AI tools as a normalized component of camps this time around allowing us to collect better data and improve learning experiences.
August 2023 – New ERC Grant
Luc Paquette and H Chad Lane were recently awarded a new grant to continue working with the WHIMC platform!
Collaborative Research: Advancing the Science of STEM Interest Development through Educational Gameplay with Machine Learning and Data-driven Interviews
WHIMC is one of the informing systems for the new INVITE Institute which asks “What can AI do to help achieve Education for All?” INVITE seeks to reposition AI as a powerful tool to promote fairness in technology-enhanced K-12 STEM learning by producing intelligent learning technologies that understand and support skills known to underlie effective learning. The institute is designed to serve as a nexus for building capacity and collaboration among researchers, and as resource for schools and teachers interested in leveraging advanced STEM learning technologies.
We look forward to processing WHIMC data to analyze important learner skills like persistence and collaboration and being one of the first contributions to a new repository.
April 2023 – Summer Camp Sign Up!
CU Science Discovery and Fiske Planetarium are offering WHIMC summer camps for kids who would like to visit our crazy-cool “What-if” worlds and exoplanets and also work in teams to build mars habitats to later share on the theater dome. We also do all kinds of other STEM activities, including imagining and drawing exoplanets, making unique astronaut skins, experimenting with Universe Sandbox, taking real life science measures and more. We still have some spots left but not for long so sign up before it’s too late!
The mars map has the actual shell from the Hassell mars competition submission
An entirely redesigned Earth Tilted 90 degrees scenario and improvements to others
Just to name a few – always much in the works.
December 2022 – Fiske Planetarium is excited to announce the world release of our FREE full-dome production WORLDS OF CURIOSITY!
This 15-minute film dives into the questions of “What it would be like to live on an Earth with no Moon?” or “What if the Earth was tilted on its side (like Uranus)?” Join Mateo and Dr. Alicia Woods as they explore how our lives would be different on these Earths, talk about other hypothetical planets that could exist, and marvel at the even stranger worlds that astronomers have discovered beyond our solar system.
The full-dome masters (1K, 2K, and 4K) or a 16:9 version, plus a promotional kit are available for FREE download.
November 2022 – We had over 250 people attend Worlds of Curiosity: Minecraft Astronomy Day at the Fiske Planetarium! The event included:
The debut of our new Planetarium Show, Worlds of Curiosity, followed by an exploration of what-if worlds and exoplanets in 360 degrees!
Imagine, describe and draw your own exoplanet in the exhibition space
Learn how we discover and study exoplanets to rehome lost aliens with PBS Nova Labs
Show off your Minecraft builder skills and science knowledge to build the best Mars habitat
Speaking of, if you started working on a Mars world at the event and would like to continue working on it here’s how to find it on the server:
Need to get setup with computer-based Minecraft? Head on over to the Get Started page.
August 2022 – We’ve had a very busy spring and summer with the WHIMC grant! Activities have included half and full day summer camps with our partners:
Fiske Planetarium, InventHQ Makerspace in Broomfield and Science Discovery at CU Boulder in Colorado
Urbana Neighborhood Connections Center and Mahomet Junior High School in Illinois
The Versant Power Astronomy Center at the University of Maine
In total 95 unique campers used science tools like /atmosphere and /gravity to collect 2221 data points, which combined with over 1927 descriptive, comparative and inferential observations! We’ll have more research and learning outcomes to report on soon.
July 2022 –
Our Artificial Intelligence team has been working hard on a prototype to track observations. We introduce MineObserver. This framework uses state of the art methods in Computer Vision and Natural Language Processing to check student’s observations in WHIMC.
The framework consists of 3 major components: the photographer, the student, and the AI system. First, the student takes an observation to be assed. Then the photographer takes the student’s POV and observation which is eventually fed into our AI system. The AI system evaluates how accurately the observation is based on the student’s view and returns feedback to the student. The entire in-game framework can be described by the images below and an example of this framework is shown in this video:
The AI system can be broken down into two major components. First, we allow the AI to generate a caption of the student’s POV. This generated caption acts as an expert answer / correct observation. From there we use RoBERTa and compare the generated caption against the student’s observation. Finally, we return feedback to the student based on the cosine similarity and keyword detection.
April 2022 – We’re hosting the first-ever Exoplanet build competition in collaboration with Planet Minecraft! Winners will be hosted on our server. Read more.
March 2022 – We found preliminary findings from our summer camps in 2021 to be very encouraging, in terms of engagement and interest development. We had 19 participants across two weeks join us to play on the WHIMC server. In total, participants made 270 in-game scientific observations, with 66% of all observations using mid- or high-level sensemaking through descriptions, comparisons, and inferences. We had 173 responses to in-game conceptual knowledge prompts delivered at the end of exploring various worlds, and 80% of responses were scored as well-reasoned and evidence-based. Finally, from coded interviews with all participants we found 215 episodes of STEM-related interest behaviors, including an intention to join the camp again next summer, wanting to visit a planetarium, sharing new astronomy knowledge with family and friends, and conducting voluntary searches in their free time to learn more about outer space.
January 2022 – WHIMC is updating to version 1.18 with even better exoplanets and new “galaxy-scale” maps to help participants understand and explore the origins of our solar system.
January 2022 – We’re excited to share a virtual field trip hosted by our partners at PBS Nova Labs: Searching for Life on Exoplanets.
Over the last few decades, scientists have found thousands of planets beyond our own. Some of those planets might be habitable, and perhaps even inhabited already; but how can we tell? Clara Sousa-Silva (Center for Astrophysics | Harvard- Smithsonian) looks for signs of life on other planets, using astronomical tools such as spectroscopy to detect the faint signals emitted by potential alien biospheres. In this presentation, she will answer the question: “Would we know life if we saw it?”, drawing on her experience investigating strange molecules on strange planets, such as her recent work trying to detect phosphine on Venus. This event is the third in a series of three virtual field trips highlighting the new five-part series “NOVA Universe Revealed.” For more information about the series, visit https://www.pbs.org/wgbh/nova/series/nova-universe-revealed/
December 2021 – One major dimension of the WHIMC project is to leverage Artificial Intelligence (AI) techniques to create pedagogical agents. Their job is to scaffold student experiences and enhance interactivity in the hope of boosting engagement and our main goal of raising STEM interest. Although these agents are not fully completed and implemented on our main server, we have made important strides for creating them.
Our design for the agents’ ability to assess student observations have been centered around Dr. Shute’s 4-process adaptive cycle. For the first process (capture), our agents record student observations. Student observations can be made using one of our four predetermined types (analogy, comparative, descriptive, and inference) with templates and suggestions or uncategorized without any support. In the next process (analyze), the agents analyze the quality of the structure of the student’s observation. If there is a mismatch of the student’s observation type with the agent’s predicted observation type, the observation is deemed incorrect. When choosing the machine learning algorithm to apply to our project, we compared eight different popular text categorization algorithms with different vectorization approaches. We trained and validated each possibility on a 80/20 split of 2020 WHIMC camp data that was coded by researchers on the project. Our results showed that for our dataset a multilayered perceptron using term frequency-inverse document frequency was the best choice with an overall accuracy on the validation set of about 73%. We also wanted our agent to intelligently estimate student mastery of the observation types. Thus, we implemented Baker, Corbett, and Aleven’s (2008) approach for Bayesian Knowledge Tracing (BKT). This algorithm was trained on 2019 WHIMC camp data also coded by researchers, and estimates hyperparameters to estimate student knowledge growth. For the third process (select), we use a data-driven approach to select feedback components. We started by clustering all observations on the server and creating word clouds on each planet to identify topics of interest to comment on. We then spell check using edit distance from a dictionary of words and vectorize using a pre-trained language representation model, Bidirectional Encoder Representation from Transformers (BERT), the student’s observation. The representation is compared using cosine similarity between the student’s observation and researcher created knowledge components to generate researcher-designed goal-oriented feedback to give to the student. For the last process (present), we show the intelligently chosen feedback in the player’s in-game chat window along with an open-learner model to graphically show the student’s progress calculated using the BKT algorithm mentioned before. We hope these additions to the agent will greatly enhance student engagement, learning, and interest in STEM at our future WHIMC camps.
November 2021 – H Chad Lane and Neil Comins recently presented our project at the Advanced Informal STEM Learning annual PI conference.
September 2021 – The WHIMC team is excited to announce a new “What If” world (Two Moons) as well as two additional Exoplanets (Kepler 186f and Brown Dwarf CWW 89Ab). Look out for additional updates and new quests to all of our worlds throughout the server and new solar system exhibits coming later this fall. Read more about them here.
August 2021 – What a busy summer! We once again ran camps with our partner the Urbana Neighborhood Connections Center. This year kids not only explored “What If” worlds and Exoplanets but they also imagined and drew their own worlds and survival habitats for Mars, in addition to PBS Nova Labs and simulations in Universe Sandbox. All of our curriculum, including lesson plans, slides, visual aides, worksheets and more can be found in the educator resources section. We had a record-breaking number of science observations this year and will have more data about what kids learned to share soon.
May 2021 – Dr. Sherry Yi, previously a graduate student on WHIMC, defended her dissertation in March and successfully deposited her dissertation with the University of Illinois. Dr. Yi made significant contributions to the project in her time at UIUC by elaborating on what interest triggering events look like in Minecraft, and what contextual aspects are most critical in promoting interest in STEM with video games. Dr. Yi’s research won first place at the Research Live! competition and she recently accepted a position as a Research Scientist at Osmo in the Bay area. We will miss Sherry greatly and wish her well in her new adventure!
April 2021 – The WHIMC Project was well-represented at the American Educational Research Association (AERA) Annual Meeting this year. We presented two posters:
One poster, Threads of Interest: Case Studies of Interest Development in a Game-Based STEM Summer Camp (Link to poster: Threads of Interest),focused on how the WHIMC Minecraft environment can be a useful for building on pre-existing interests in science for young learners, and generate new interests in science for learners who may not currently see themselves as interested.
Our other poster, Conducting a Video-Game Based Camp Intervention During the COVID-19 Pandemic (Link to poster: Camp Intervention) offers best practices for conducting a hybrid camp using a digital game, such as identifying novice players early in the camp and providing needed scaffolding and training. Using the hybrid in-person/online model for the camp resulted in more instances of interest development than previous years in-person only.
March 2021 – Development is well underway for the latest version of the WHIMC server! We’ve been focusing a lot on user experience to make sure learners who aren’t part of a camp or class can get the most out of their explorations. At the same time our team has also emphasized curricular scaffolding to help learners to make scientific observations that will also result in stronger data collection.
PBS Nova Labs has recently announced their Exoplanet Lab which will send interested learners on over to explore simulations of the same kinds of worlds they’ve just learned to detect!
All the excitement around the Perseverance Rover landing inspired us to create a Minecraft simulation of Mars that’s as close to the real thing as we can get! Learn more about our modeling and development process in Rendering the Red Planet in Minecraft.
July 2020 – We returned for the third straight year to the Urbana Neighborhood Connections Center (UNCC), but this time we did so virtually. Despite our team being remote, we had two exciting weeks exploring hypothetical versions of Earth and Exoplanets in Minecraft. We also recorded a record number of unique in-game observations by a single camper at 96 in one week. We will see if that number can be topped next year!
February 2020 – We presented our work and led a week-long professional development workshop series with Ateneo University in the Philippines. Topics covered included the use of Minecraft for informal learning in STEM, introductory programming with ComputerCraft, computational thinking applications in curriculum design, and 3D printing. Over 150 people came to our opening talk, we guest starred on a podcast and worked with over 15 representatives from schools, the Department of Education, and one non-government organization. Read more about it here. Many thanks to Didith Rodrigo and her many collaborators!
Sep 2019 – New articles on our work from the University of Illinois News Bureau and Associated Press. Fox Champaign visited campus to interview us last week, so we’ll be posting a link to that video in coming weeks!
Aug 2019 – We are very excited to share the news that our follow-on proposal has been funded by the National Science Foundation! Our new grant will allow us to create new STEM interest-triggering resources, tools for exoplanet exploration and creation, new online learning activities (in collaboration with PBS NOVA), and corresponding planetarium shows (in Boulder, CO). See the NSF summary for more information.
Apr 2019 – We are running a camp at the Next Generation School. What an entrance!
Mar 2019 – Catch us in June at the Foundations of Digital Games conference discussing our findings from our work with the Urbana Neighborhood Connections Center and FabLab.
Feb 2019 – Check out Dr. Lane & Dr. Comins at AISL!
Jan 2019 – Look out for our talk at ISTE this June. Learn more about our project and ways to implement games like Minecraft into your own classroom.
June 2018 – It was great working with enthusiastic campers and staff at Urbana Neighborhood Neighborhood Center and FabLab. Now to dive into data…
Week 1 of Minecraft camp at the Urbana Neighborhood Connections Center. They love building!!