A title image of a museum visitor looking at the wall and seeing augmented reality content.

Artifact: An Immersive Visitor Experience

Client
Carnegie Museum of Art
Collaborators
Jamie Choi, Yeonjin Park, and Matthew Muenzer
Project duration
Six weeks
Role
Motion design lead, UI design co-lead, concept development, interaction design
Technical Skills
Figma, Adobe Premiere, Adobe After Effects, and Unity

Overview

Improve the visitor experience at the Carnegie Museums in Pittsburgh, which have remained relatively static for the past decade. Consider areas including wayfinding, information browsing, experiential learning, and social interaction.

Outcome

Artifact is an immersive visitor experience for The Carnegie Museum of Art. Visitors choose their own paths through the museum based on what is most meaningful to them, learn about the art they find most interesting with interactive and immersive content specific to each work, and share their own ways of experiencing art with their community.
Concept Video
Artifact Visitor Journey

Ticketing

Arriving at the museum, visitors are introduced to the Artifact experience by the ticketing personnel on site. After purchasing their passes, the visitor is directed to the Artifact experience desk located in front of the Forum Gallery off the main entryway to pick up headsets.

Onboarding

At the Artifact desk, Visitors are given a Hololens 2 device and assisted by staff to get it fitted and calibrated.

Once set up, visitors go through a brief AR tutorial to learn Artifacts key gestures and artwork interactions.

Experience Selection

Once onboarding is complete, visitors choose from (1) a fully guided thematic tour created by the museum’s curatorial staff; (2) a community tour published by other museum-goers; (3) a custom tour generated for the visitor based on criteria such as color, time period, or medium.

They may also browse the museum freely.

Getting to the Gallery

Lightweight AR wayfinding guide visitors to stops along their tour.

Discovering Interactions

Artifact’s cube logo signals where visitors can interact with artwork in AR. When the cube logo is selected, Artifact's on-object user interface appears.

Artwork Interactions

From the on-object UI, visitors can browse four categories of artwork information: (1) general information; (2) artist background; (3) process and craft; (4) historical context.

Within each category, visitors may read text-based information, or engage with interactive audiovisual content.

Adapting your Experience

At any time, visitors may call up a control panel by touching their wrist. From this panel they can change or exit their tour, access settings, view bookmarked objects, or review the gesture interface.

Contributing your Experience

When following customized tours or browsing independently, visitors can assemble and submit their own tour to Artifact’s online community.

Museum staff moderate Artifact’s community tours and publish them. At that point, they can be viewed and selected by future visitors in the experience selection phase.

Remember your experience

When visitors are done with the Artifact experience, they are directed to return their Hololens 2 to the check-in desk. When they do, they’ll receive a memento card showing a list of the artworks they saw that day.

A QR code links to a GIF which rotates through the artworks that the visitor saw that day.
PROCESS
Our team began by outlining our design process, including our major project milestones, which we completed as a team, and smaller tasks, which we assigned to individual team members.
I: Understanding the Problem Space

The 21st Century Museum

There are immense challenges facing the traditional art museum. We narrowed in on three: (1) A pressure to reorganize collections; (2) the problem of the "white cube;" (2) institutional and physical barriers to change.
1
Pressure to reorganize
For centuries museums told one kind of story: A linear march through art history that ignored the contributions of women, nonwestern artists, and other marginalized and vulnerable populations. Museums are now under tremendous pressure to redress this historic bias by reorganizing their collections in order to tell more nuanced, inclusive, and diverse histories.
2
The problem of the "white cube"
By placing an artwork in a museum, it is immediately decontextualized. The protest art of the 1980s was designed to be hung up on the street, not in the white walls of a neoclassical museum. The challenge for museums is to provide visitors with a way to reunite objects with their original place and context.
3
Institutional and physical barriers to change
At the same time that there is pressure for reform, the barriers to making change are high. Curators we spoke to highlighted the logistical, bureaucratic, and budgetary difficulties of mounting an exhibition or rehanging the gallery.

Problem Statement

How might we empower curators to create rich learning experiences for visitors? Specifically, how might we encourage experiences that both highlight the diverse histories behind the art and visitors’ individual curiosities?

II: Exploratory Research

Augmented Reality in Museums

Augmented Reality is an exciting emergent technology, but it’s not the solution to every problem. We interrogated the benefits and limits of AR and determined it was the right fit for our problem statement for four major reasons
Expand Toolset
Museums can only communicate so much with wall text. Augmented Reality can provide additional, seamlessly connected channels for what you can learn about art. Opt-in interactivity and user choice afford museumgoers a wealth of new information without overloading them.
Focus on Art
Connecting with physical art objects will always be the most important interaction for art museum visitors, AR does not have to interfere with this interaction. The amount of AR information can range from peripheral to immersive, and curators can decide how much information they want presented.
Nimble Curation
AR can empower museums to create new ways of experiencing their collections, and reach out to new and changing audiences. In contrast to spending years building a physical exhibition, AR allows curators to launch rich visitor experiences relatively quickly.
Responsive Museums
Museums collect data on attendance, but outside of surveys are limited in what they can learn about visitors. AR can provide rigorous data on what artworks visitors see and how they navigate galleries. A museum could leverage this data to responsively improve curation of not only virtual, but also real-life content.

AR Interface Research

Since our team decided to build Artifact for the Hololens 2, we tested out the device ourselves and dug into Microsoft’s Mixed Reality Toolkit (MRTK) to understand its UI and UX patterns.
Unlike mobile UI, AR extends into three-dimensional space. This meant that we had to define two sets of gestures, one to interact with nearby objects and one for those farther away from the user. The 3D possibilities afforded by AR also mean that UI objects tend to mimic real physical objects.

Finally, Hololens 2 utilizes persistent AR content, where UI elements are placed in a three-dimensional location around the user and do not change their position if when the user’s field of vision changes. In order to change the position of UI elements, users must move it using gestures.

Visitor Journey Map

At the conclusion of our research phase, we created a map of how visitors would navigate the Artifact experience. It highlighted key actions, touchpoints, museum locations, and the level of AR information presented in the visitor's field of vision.

III: Concept Development

Systems Diagrams

Systems diagrams outlined the technology and devices involved in our solution, as well as the flow of data between these devices.

Visual Language and Branding

Typefaces, colors, and visual treatments were cut down to create a sense of clarity and consistency for the user
We used CMOA’s existing visual identity as a point of departure, utilizing many of its brand elements in Artifacts interface but cutting down on the typefaces, colors, and visual treatments to create a sense of clarity and consistency for the user.

For the Artifact logo, we found inspiration in the sharp geometry of the two angle brackets in the letter “c” from the existing logo.

Our chosen typeface, Helvetica Neue, is used across Artifact’s materials for display, header, and body text in order to maximize readability and complement the geometric branding elements.

User Interface Design

We let four principles guide our UI design system. These principles kept the possibilities and limitations of AR in mind, especially for users who are still getting acquainted with the technology.
Readable
High contrast colors, type, and layout
Reachable
Fewer, more obvious interaction targets
Recognizable
Repeated patterns to identify interactions
Minimal Input
Reducing clicks for users adjusting to the technology
Respectfully Unobtrusive
Strategically select, place, and distribute content
IV: Deliver

Video Storyboard

We wanted our concept video to be a highlight reel of Artifact's features, not an end-to-end visitor journey. To determine which key features and interactions we wanted to include, we created shot-by-shot video storyboards.

Video Production

Video production involved carefully choreographed collaboration as we added visual effects in Adobe After Effects and edited the final video in Premier.
Click here to return to our final concept video

Key Performance Indicators

Artifacts KPIs include metrics to measure customer acquisition, engagement, and task completion.

We defined three key KPIs to measure Artifact's success if it were to be launched at CMOA: (1) 2,000 customers by the end of the Artifact’s first year after launch, as measured by the CMOA’s tick desk; (3) 25% or more of customers publishing their own tour to the Artifact community, as a way to measure engagement; (3) 80% of tour customers completing at least 80% of their tour.

Further Development

Rounding out the Service
The systems designed for Artifact present many opportunities to build a complete service, particularly leveraging data on how visitors experience CMOA. We prototyped a video wall for the atrium showing community tours and visualizing the most popular art in the galleries. Importantly, this screen lets museumgoers without the AR experience connect with Artifact-generated content.
Scalability
Artifact is a scalable product. Although we designed our solution around specific challenges at CMOA, the product’s framework can be easily applied to reinvigorate other public art collections.
Group Interactions
Museums are a place for shared experiences. We see opportunities for future Artifact features to be collaborative multiplayer experiences, such as virtually recreating an artwork together.
CMU x CMOA
CMOA proximity and relationship with Carnegie Mellon University gives it a unique position to bring AR into the visitor experience. We can foresee a relationship between designers and developers at CMU with curators/educators at CMOA.
PROJECT TAKEAWAYS
There are certain irreplaceable physical experiences. How can AR technology add to these experiences without disturbing them?

As resources are poured into mixed reality products and services, design leaders need to keep in mind there are certain irreplaceable physical experiences. There will never be a digital product that can replace the emotional and spiritual power of seeing an original art object in person. Curators and other art professionals made this point repeatedly in our interviews.

In designing Artifact, we challenged ourselves to create an AR experience that enhances museum experiential learning and wayfinding without disturbing the interaction between viewers and art objects. This consideration informed our work throughout the project. Beyond Artifact, however, I hope designers in the mixed reality space challenge themselves to deliver the greatest possible benefit with the least possible disturbance to meaningful physical experiences.