The Challenge
With declining appeal to younger visitors, particularly Gen Z, the Carnegie Natural History Museum faces difficulties in modernizing its traditional exhibition approach as the largest museum in Pittsburgh.
A mixed-reality museum experience for GenZ rooted in interactive experiential learning and participatory social interactions
Overview
The Challenge
With declining appeal to younger visitors, particularly Gen Z, the Carnegie Natural History Museum faces difficulties in modernizing its traditional exhibition approach as the largest museum in Pittsburgh.
The Solution
To revitalize the Carnegie Natural History Museum's appeal to younger visitors, particularly Gen Z, we developed immersive and multi-sensory experiences to engage GenZ that incorporates interactive technologies (such as voice, eye, and gesture interactions) while preserving the traditional charm of the museum as the largest in Pittsburgh.
Youngryun Cho;
Greg Chen;
Zhuoneng Wang
5 weeks
(Oct. to Dec. 2021)
User research
Interaction Design
AR Prototype
Mobile App Design
Video
Carnegie Museum of Natural History (CMNH)
Outcome
Providing an engaging learning experience for Gen Z visitors
Our goal is to revitalize the Carnegie Natural History Museum's appeal to younger visitors, particularly Gen Z. To achieve this, we use mixed reality goggles to create an immersive experience. Visitors can go on a scavenger hunt to collect digital copies of animals or plants, complete tasks, and learn while having fun. They can also wear their collections as part of their personal style in a virtual setting. Visitors are encouraged to share what they learn and their collections as AR filters on social media using mobile-based AR filter creation tools.
Creating "Gamified museum experience” & “participatory social interactions”
Beyond has two parts - a "gamified museum experience" to engage Gen Z, and "participatory social interactions" to increase CMNH's digital influence, which increase CMHN's social influence in the digital community and is tailored for the lifestyle of Gen Z.
Feasible AR Prototype
We used C4D to create 3D models, and used Spark AR to create our own filters for Beyond.
Research
Why The CMNH (Carnegie Museum of Natural History)?
"The museum has a dream vision to become the world’s most relevant history museum, by enabling visitors to discover greater stories, and increasing the connection between nature and people."
However, “Our internal data shows the base of visitors is shrinking year by year, especially for younger generation. We are now losing connection with younger audiences.”
--Nicole, Curator of CMNH
Why CMNH Is Losing Connection With GenZ?
GenZ is a group of people who were born between 1996 - 2010. Through secondary research Ubimo, we know that the current frequency of museum visits among GenZ is 23% lower than the national average. This is primarily because the traditional exhibition approach is not effective in attracting GenZ.
Though a quantitative research and qualitative research with GenZ, we found GenZ expect museums to be:
1. Interactive
2. Experiential
3. Social shareable, such as Instagrammable popups
Why GenZ Is Important To CMNH?
As the next generation of visitors and donors, Gen Z will play a vital role in shaping the future of the museum. In the coming years, they will increasingly become a key target audience for the museum's programs, exhibitions, and educational initiatives.
Specific Pain-points
Combining with primary and secondary research, we found current CMNH has the following detail pain points:
1. Poor navigation experience
Visitors often get lost in the museum and cannot find the gallery they want to visit. Additionally, the current museum map is not clear and is difficult for visitors to read.
2. Lack of Interactivity
The current experience at CMNH is passive and lacks interaction with visitors. This makes it difficult for visitors to engage and learn from their museum visit.
3. Week post-museum experience
The museum cannot offer further opportunities for exploration beyond the scientific information presented during the museum visit.
4. Lack of channels to share thoughts
Currently, Gen Z does not have a channel to share their feelings, ask questions, or form their own opinions from different perspectives.
Synthesis
Design Principle
We decided to focus on Botany Hall as a case stud as it aligns with the latest vision of the museum, giving us a window to see from their perspectives and find design opportunities.
Why AR/MR Technology May Help?
Based on key insights and design principle, AR/MR (Augmented Reality/Mixed Reality) is a good way to provide an immersive, interactive, and engaging experience because it allows users to experience the world around them in new and exciting ways.
1. Have a range of interactive and engaging experiences
AR/MR can provide a range of interactive and engaging experiences, such as interactive exhibits, games, and simulations.
2. Have accessibility to mobile devices
Users can use their mobile devices or specialized AR/MR equipment to interact with virtual objects in real-time and engage with the content on a deeper level.
3. Provide multi-sensory experience
AR/MR can also provide a multi-sensory experience, including visual, audio, and haptic feedback, which can create a more immersive and engaging experience for users.
Parking AR/MR Ideas In Storyboard
We drew storyboards followed by design principles.
Journey Map Of Beyond
We have created a new journey map to explore the potential locations for MR/AR experiences.
Generation
Design Direction
An immersive and engaging AR/MR experience for GenZ that incorporates multi-sensory interactions, such as voice commands, gestures, and wearable devices.
Design Challenge 01
How to ensure multiple devices, such as goggles and phones, can collaborate seamlessly?
We explored and defined touch points in each phase of the user journey, enabling us to determine which technologies, interactions, and devices would be utilized at each stage.
Design Solution 01
Automatically transfer real-time data over a museum's local area network.
Connect, upload, and share data using Bluetooth, Wi-Fi, and Cloud. The local area network ensures data privacy.
Design Challenge 02
Design Solution 02
Through the use of wearable devices equipped with voice-activated and hand-gesture-controlled interfaces, as well as proximity-based technologies.
Interaction 01
Wearable Device & Voice Commands
How To Trigger Menu?
Unlike MRTK recommendations, we chose to use "raise the wrist with palm-down" for the menu. This gesture is more accessible because it borrows the watch affordance and is less confusing for first-time users, which is particularly useful in a museum where people may not be familiar with the commands.
Additionally, we redesigned the admission ticket to include anchor points (Beyond logo) to improve detection accuracy.
Interaction 02
Hand Gestures
Goal
1. Lower the cost of the learning curve.
2. Reduce hands movement range
Hover & Pinch
We explored different ways to perform 'hover and pinch' gestures and found that the third one, 'palm pointing,' is the most accurate and accessible.
Drag & Collect
We came up with this idea for 'drag to collect' from the 'swipe to confirm' gesture on the iPhone, giving users a natural feeling of confirmation.
All Gestures
Interaction 03
Proximity-based technologies
When users are close to the diorama, it triggers the welcome voice and interaction.
Design Challenge 03
Design Solution 03
Be an AR content creator through Beyond app.
With the Beyond app, Gen Z can create AR content using the items they collect in the museum.
Design System
We use green and orange as the main colors to keep a fresh, warm and welcoming atmosphere. For 3D, we go with a low-poly, faceted, and fun style tailored to the taste of Gen Z.
Design Challenge 04
How to assess the feasibility and utility of multi-sensory interactions?
Design Solution 04
Prototyping and testing ideas through Unity, Leap Motion and Spark AR.
We created a Unity prototype using MRTK Kit as a base, which allowed us to test grabbing a plant from a diorama using a pinch and drag scenario. We customized the kit to fit our design and tested it in Varjo, a see-through MR headset, to ensure the gestural commands and font size were effective, which proved successful in the prototype.
When defining gestures for a project, Spark AR's tracking systems are inspiring and enable us to design effective motion-based interactions in the future.
Next Step
The Need For A Blurred Ground
During the critique of our final presentation, a question arose: is a blurred background necessary in the MR world to enhance contrast between objects and the real-world background, or could it create more problems such as safety concerns in public spaces or a detached visual experience? Exploring the balance between these factors would be an interesting topic to investigate.
Group Experience
While BEYOND currently focuses on individual experiences, we believe extending it for groups of visitors could be meaningful, as many people begin their museum experiences on school or family trips.
Beyond CMNH
At the Carnegie Museum of Natural History, visitors can expect an immersive and engaging learning experience that goes beyond what they might have experienced before. Through our product, Beyond not only connects this museum with others around the world via hashtags on social media but also bridges the gap between visitors and museums. By increasing exposure to museums and connecting them into one global network, Beyond motivates visitors to return and explore other museums in the future.
Reflection
Accessibility is the top priority.
We aim to reduce the learning curve by designing culturally and technologically accessible gestures and interactions. To follow the watch affordance, we designed the trigger menu using "raise the wrist," and we use "palm down" to choose items. We avoid finger-point gestures due to technical issues with camera detection and palm-up gestures due to negative cultural connotations in some countries, particularly the United States, a country of immigrants.
Technology limitations.
MR/AR technology is not universally accessible, especially for those with vertigo or disabilities. In public spaces such as museums, where people move their hands frequently, additional steps to prevent false activation are necessary. Considering hands-free scenarios is also beneficial since hands are valuable and often in use.
Don't be shy to say "I don't know."
My first experience with AR and MR technology required me to learn a lot within a short timeframe of 5 weeks. Since it was impossible to know everything, I relied on my teammates for assistance. I learned that design is not only about appearance but also functionality, and as a team, we each have unique strengths and can seek help when needed by saying "I don't know."
Appendix
Thank You To My Team
want to express my sincere gratitude to my amazing teammates on Project Beyond. Youngryun, thank you for sharing your incredible visual skills with me throughout this project. Your eye for detail and ability to bring our ideas to life were truly inspiring. Zhuoneng, thank you for teaching me technical AR/MR development skills that I never thought I would be able to learn. Your expertise and dedication to the project were invaluable. And to Guanbai, thank you for helping me improve my communication and presentation skills. Your thoughtful feedback and guidance were crucial to our success. Together, we created something truly special, and I am honored to have worked alongside each of you. Thank you for an unforgettable experience!
Design Process