Beyond

The Challenge

With declining appeal to younger visitors, particularly Gen Z, the Carnegie Natural History Museum faces difficulties in modernizing its traditional exhibition approach as the largest museum in Pittsburgh.

The Solution

To revitalize the Carnegie Natural History Museum's appeal to younger visitors, particularly Gen Z, we developed immersive and multi-sensory experiences to engage GenZ that incorporates interactive technologies (such as voice, eye, and gesture interactions) while preserving the traditional charm of the museum as the largest in Pittsburgh.

Awards

Platinum Winner | Muse Design Award 2023↗

Silver Winner| NY Product Design Award 2023 ↗

Providing an engaging learning experience for Gen Z visitors

Our goal is to revitalize the Carnegie Natural History Museum's appeal to younger visitors, particularly Gen Z. To achieve this, we use mixed reality goggles to create an immersive experience. Visitors can go on a scavenger hunt to collect digital copies of animals or plants, complete tasks, and learn while having fun. They can also wear their collections as part of their personal style in a virtual setting. Visitors are encouraged to share what they learn and their collections as AR filters on social media using mobile-based AR filter creation tools.

Creating "Gamified museum experience” & “participatory social interactions”

Beyond has two parts - a "gamified museum experience" to engage Gen Z, and "participatory social interactions" to increase CMNH's digital influence, which increase CMHN's social influence in the digital community and is tailored for the lifestyle of Gen Z.

Feasible AR Prototype

We used C4D to create 3D models, and used Spark AR to create our own filters for Beyond.

Why The CMNH (Carnegie Museum of Natural History)?

"The museum has a dream vision to become the world’s most relevant history museum, by enabling visitors to discover greater stories, and increasing the connection between nature and people."
However, “Our internal data shows the base of visitors is shrinking year by year, especially for younger generation. We are now losing connection with younger audiences.”
--Nicole, Curator of CMNH

Interviewed with Nicole, Curator of CMNH, 11/08/2021

Why CMNH Is Losing Connection With GenZ?

GenZ is a group of people who were born between 1996 - 2010. Through secondary research Ubimo, we know that the current frequency of museum visits among GenZ is 23% lower than the national average. This is primarily because the traditional exhibition approach is not effective in attracting GenZ.
Though a quantitative research and qualitative research with GenZ, we found GenZ expect museums to be:

1. Interactive

2. Experiential

3. Social shareable, such as Instagrammable popups

The current exhibition is static and boring.

Why GenZ Is Important To CMNH?

As the next generation of visitors and donors, Gen Z will play a vital role in shaping the future of the museum. In the coming years, they will increasingly become a key target audience for the museum's programs, exhibitions, and educational initiatives.

Specific Pain-points

Combining with primary and secondary research, we found current CMNH has the following detail pain points:

1. Poor navigation experience

Visitors often get lost in the museum and cannot find the gallery they want to visit. Additionally, the current museum map is not clear and is difficult for visitors to read.

2. Lack of Interactivity

The current experience at CMNH is passive and lacks interaction with visitors. This makes it difficult for visitors to engage and learn from their museum visit.

3. Week post-museum experience

The museum cannot offer further opportunities for exploration beyond the scientific information presented during the museum visit.

4. Lack of channels to share thoughts

Currently, Gen Z does not have a channel to share their feelings, ask questions, or form their own opinions from different perspectives.

Design Principle

We decided to focus on Botany Hall as a case stud as it aligns with the latest vision of the museum, giving us a window to see from their perspectives and find design opportunities.

Why AR/MR Technology May Help?

Based on key insights and design principle, AR/MR (Augmented Reality/Mixed Reality) is a good way to provide an immersive, interactive, and engaging experience because it allows users to experience the world around them in new and exciting ways.

1. Have a range of interactive and engaging experiences

AR/MR can provide a range of interactive and engaging experiences, such as interactive exhibits, games, and simulations.

2. Have accessibility to mobile devices

Users can use their mobile devices or specialized AR/MR equipment to interact with virtual objects in real-time and engage with the content on a deeper level.

3. Provide multi-sensory experience

AR/MR can also provide a multi-sensory experience, including visual, audio, and haptic feedback, which can create a more immersive and engaging experience for users.

Explore Oculus, Varjo, Hololens at CMU AR/MR Lab.

Parking AR/MR Ideas In Storyboard

We drew storyboards followed by design principles.

Journey Map Of Beyond

We have created a new journey map to explore the potential locations for MR/AR experiences.

‍

Design Direction

An immersive and engaging AR/MR experience for GenZ that incorporates multi-sensory interactions, such as voice commands, gestures, and wearable devices.

‍

How to ensure multiple devices, such as goggles and phones, can collaborate seamlessly?

We explored and defined touch points in each phase of the user journey, enabling us to determine which technologies, interactions, and devices would be utilized at each stage.

‍

Automatically transfer real-time data over a museum's local area network.

Connect, upload, and share data using Bluetooth, Wi-Fi, and Cloud. The local area network ensures data privacy.

‍

How can we provide end-to-end engaging multi-sensory interactions during a museum visit?

We created a scenario in which Gen Z visits Botany Hall and considered which interactions would enhance their end-to-end immersive experience.

Through the use of wearable devices equipped with voice-activated and hand-gesture-controlled interfaces, as well as proximity-based technologies.

Interaction 01

Wearable Device & Voice Commands

How To Trigger Menu?

Unlike MRTK recommendations, we chose to use "raise the wrist with palm-down" for the menu. This gesture is more accessible because it borrows the watch affordance and is less confusing for first-time users, which is particularly useful in a museum where people may not be familiar with the commands.
Additionally, we redesigned the admission ticket to include anchor points (Beyond logo) to improve detection accuracy.

Hand-gesture-controlled interfaces

Interaction 03

Proximity-based technologies

When users are close to the diorama, it triggers the welcome voice and interaction.

How can we facilitate social interaction for GenZ that is participatory after the museum visit?

Social sharability is often considered a characteristic of GenZ.

Be an AR content creator through Beyond app.

With the Beyond app, Gen Z can create AR content using the items they collect in the museum.

Design System

We use green and orange as the main colors to keep a fresh, warm and welcoming atmosphere. For 3D, we go with a low-poly, faceted, and fun style tailored to the taste of Gen Z.

Hololens UI

To ensure legibility in MR settings, we follow Microsoft's guidelines for font size.

Mobile App

To ensure legibility in MR settings, we follow Microsoft's guidelines for font size.

How to assess the feasibility and utility of multi-sensory interactions?

Prototyping and testing ideas through Unity, Leap Motion and Spark AR.

We created a Unity prototype using MRTK Kit as a base, which allowed us to test grabbing a plant from a diorama using a pinch and drag scenario. We customized the kit to fit our design and tested it in Varjo, a see-through MR headset, to ensure the gestural commands and font size were effective, which proved successful in the prototype.

When defining gestures for a project, Spark AR's tracking systems are inspiring and enable us to design effective motion-based interactions in the future.

The Need For A Blurred Ground

During the critique of our final presentation, a question arose: is a blurred background necessary in the MR world to enhance contrast between objects and the real-world background, or could it create more problems such as safety concerns in public spaces or a detached visual experience? Exploring the balance between these factors would be an interesting topic to investigate.

‍

Group Experience

While BEYOND currently focuses on individual experiences, we believe extending it for groups of visitors could be meaningful, as many people begin their museum experiences on school or family trips.

‍

Beyond CMNH

At the Carnegie Museum of Natural History, visitors can expect an immersive and engaging learning experience that goes beyond what they might have experienced before. Through our product, Beyond not only connects this museum with others around the world via hashtags on social media but also bridges the gap between visitors and museums. By increasing exposure to museums and connecting them into one global network, Beyond motivates visitors to return and explore other museums in the future.

Accessibility is the top priority.

We aim to reduce the learning curve by designing culturally and technologically accessible gestures and interactions. To follow the watch affordance, we designed the trigger menu using "raise the wrist," and we use "palm down" to choose items. We avoid finger-point gestures due to technical issues with camera detection and palm-up gestures due to negative cultural connotations in some countries, particularly the United States, a country of immigrants.
‍

Technology limitations.

MR/AR technology is not universally accessible, especially for those with vertigo or disabilities. In public spaces such as museums, where people move their hands frequently, additional steps to prevent false activation are necessary. Considering hands-free scenarios is also beneficial since hands are valuable and often in use.
‍

Don't be shy to say "I don't know."

My first experience with AR and MR technology required me to learn a lot within a short timeframe of 5 weeks. Since it was impossible to know everything, I relied on my teammates for assistance. I learned that design is not only about appearance but also functionality, and as a team, we each have unique strengths and can seek help when needed by saying "I don't know."

Thank You To My Team

want to express my sincere gratitude to my amazing teammates on Project Beyond. Youngryun, thank you for sharing your incredible visual skills with me throughout this project. Your eye for detail and ability to bring our ideas to life were truly inspiring. Zhuoneng, thank you for teaching me technical AR/MR development skills that I never thought I would be able to learn. Your expertise and dedication to the project were invaluable. And to Guanbai, thank you for helping me improve my communication and presentation skills. Your thoughtful feedback and guidance were crucial to our success. Together, we created something truly special, and I am honored to have worked alongside each of you. Thank you for an unforgettable experience!
‍

Design Process