Building a multiplayer walk-up-and-play experience to foster enviromental awareness.

Duration
Jan‍

Team Size
4

What I delivered

Background

As museums transition from rigid exhibition spaces to hubs of interactive engagement, we were tasked with designing a multiplayer walk-up-and-play experience intended for natural science museums.

1. Topic searching

From ideation,
to down-
selection

2. Ideas in motion

Evaluating the
ideas
through video
prototyping

3. Wizard of Oz

Play testing before technical implementation

4. Building an MVP

Computer vision and game algorithm.

5. Refining Interactions

Usability test and more iterations

6. Final Showcase

Public exhibit and reflection

1. Topic Searching and Ideation

With the goal of achieving team alignment on one topic space for our interactive experience, we generated 24 initial walk-up-and-play concepts that take on the different topics for environmental awareness.

After down-selecting to 12, we went through rounds of rapid peer review sessions to glean initial feedback. We eventually landed on the topic space of wildlife habitat fragmentation by highways and roads.

Design Principles

To make actionable what we've learned from the initial collection of concepts, I crafted design principles based on the peer review feedback. This framework primed the team for easier discussion and decision making later on. We then went through multiple round of crazy 8s to generate and refine ideas, using the design principles as a decision matrix to evaluate the impact of each new concept.

01.
Storytelling

Tie the core game mechanics to the narrative of the social impact.

02.
Go big

The scale of the controller and input design should fit a public setting.

03.
Collaboration

Emphasize teamwork and collective strategies for the core gameplay interaction.

04.
Call-to-action

Provide actionable steps or resources that players can use to make a difference.

2. Storyboarding and Video Prototyping

We wanted to test out the key interactions of the two selected ideas with a focus on core mechanics and social impact. We first storyboarded the ideas and used stop motion and green screen techniques to put them in motion. The video prototypes allowed us to better understand the ideas in a storytelling format.

The selected idea involves players placing physical bridges on the game board for the animals to travel across fragmented habitat spaces. Applying the design principles, the key interaction weaves the a potential real-life solution, the construction of wildlife crossings, into the core gameplay.

The unselected idea involves players forming a "human bridge" to reunite animals from two opposing walls.

The idea was scrapped because of the weak connection between the key interaction and the social impact, coupled with feasibility concerns.

Reflection on Video Prototyping:

The process of acting out the sequence of interactions proved to be more valuable for the team than the final video outputs. The videos primarily served as reference points to facilitate later discussions, while the act of role-playing the interactions provided richer insights.

Power of Roleplaying:

The process of storyboarding and filming the videos organically evolved into an internal experience prototyping session. By role-playing the interactions, we gained firsthand insights into the gameplay mechanics and potential technical implementations required to bring our vision to life.

3. Wizard of Oz Testing

Before moving onto implementation, we wanted to test out the interaction with Wizard-of-Oz techniques. The goal was to evaluate the learnability of the mechanics, observe how players collaborate and strategize, and compare the effectiveness of the two options for our input method.

We tested two different input methods. Option 1 involves players using a stylus to draw bridges between different plots of land, while option 2 requires players to place physical bridge blocks. Our assumption was that both methods embodied the tactility and engagement we envisioned for a public space.

I created this visualization to plan out the testing setup. The WoZ prototype was built on Figma and projected onto a whiteboard. To simulate the effect of the gameplay responding to controller input, we assigned two “wizards” working behind the scene, adding bridges and controling the movement of the animals.

Finding 01

The participants preferred placing physical bridge blocks for its interactivity and visual representation of wildlife crossings.

Finding 02

The participants expressed a desire for in-game animal dialogues and audio feedback for deeper emotional immersion.

Finding 03

Rather than collectively strategizing bridge placement, one participant typically took over the gameplay.

Finding 3 revealed a lack of intended collaboration, making it the highest priority to address. I advocated for a revised approach where Player A places bridge blocks while Player B manually steers the animal with a bear-shaped controller. This method also obviated the need for developing a pathfinding algorithm for the animal.

4. Implemention studies

Animal movement: CPX controller

A bear plushie functions as the custom controller for animal movement. Inside the bear we hid a CPX coded to detect roll and pitch, allowing the player to “drive the animal”.

Game MVP

I developed the base algorithm of the game with basic playability using mouse clicks. The map generation leverages a two-dimensional array that gives each plot of land a unique ID.

Bridge building: color detection

For the game to recognize the bridge blocks in real time, I built a color detection functon to analyze the placement and angle of bridges. This approach proved to ineffective in implementation.

Bridge building: ArUco markers

I realized color detection makes the input susceptible to lighting conditions, player’s outfit color, and the overall detection speed. I then pivoted into using ArUco markers for computer vision.

Bridge block update

We iterated through various versions of the bridge block design to ensure it was appropriately scaled for use in a public setting, while also prioritizing ergonomics for comfort and usability.

Developing graphic assets

To tie all the implemented elements together, we themed the game with map and character designs, as well as sounds and dialogue boxes to provide input feedback.

5. Usability Test & Refining the Experience

As we approached the date for the showcase, we conducted 4 final usability tests to identify points of friction. To address them, I aligned the team to stick to the 80/20 rule to concentrate our efforts - we focused on addressing the 20% of issues or pain points that were causing 80% of the friction or dissatisfaction in the player experience.

Finding 01

Although we provided an onboarding animation teaching the mechanics, participants expressed the need for real-time guidance during initial gameplay.

Finding 02

Participants had different instincts as to how the animal character responds to the bear controller, often confusing forward movement with backward.

Solution 01

We incorporated the onboarding tutorial into the first parts of the gameplay.

Solution 02

We updated the tutorial in the onboarding animation to clarify the input method.

Showcase

Impact

The walk-up-and-play experience showcased at the half-day event was an overall success, with no major hardware issues encountered during the showcase. Apart from camera realignment for computer vision calibration, the team remained hands-off.

The players reported becoming more aware of habitat fragmentation as a critical environmental crisis and expressed a willingness to learn more about the topic. The collaborative gameplay unfolded as intended by design, fostering an engaging educational experience. However, the trade-off between gameplay and educational content became evident, as the game contained only one level and was perhaps oversimplified, resulting in less gameplay and more emphasis on the educational aspects.

Reflection

As the technology lead and leading designer for this project, I had the unique opportunity to bridge the experiential and technical considerations that shaped our design approach. This dual role allowed me to align the team on the most feasible path forward, given the time and expertise constraints.

The main portion of our efforts was dedicated to building and testing the idea through various methods, guided by the principle of building as fast as possible to gain internal insights and external feedback. This approach enabled us to quickly identify potential issues and pivot away from less desirable directions, ensuring that our design remained grounded in both experiential and technical feasibility throughout the process.

  Next Project: Exhibit Design  ->