Brocca

Immersing users in language-learning

Spring 2019 // 8 weeks
Carnegie Mellon University
Team Project // 5 people

Problem

Foreign languages are best learned in an immersive context, where users can observe and interact with native speakers in real-life scenarios. However, most in-school programs and language-learning apps emphasize repetition and memorization of vocabulary and grammar, instead of immersive, contextual input.

Solution

Brocca is a collaborative language-learning game in VR that helps people immerse themselves in a foreign language. Users can explore virtual cities where their target language is spoken and learn how to apply words and phrases in real-life scenarios. To encourage true cultural immersion, learners can interact with content uploaded by native speakers, such as photos, stories, music, and popular phrases. This enables them to understand the intricacies of language, such as slang and humor, that apps and classroom learning just can’t teach.

Methods

Lit Review, Interviews, Speed-Dating, Paper Prototyping, Experience Prototyping

Tools

Figma, Principle, Final Cut

Role

I contributed mainly to concept definition and animation. I also directed the concept video.

PROJECT CONTEXT

How might we improve the language-learning experience to make it more immersive?

As none of my teammates had prior experience teaching foreign languages, we began the project with secondary background research. This helped us understand scientifically-proven methods of language-learning, as well as the current competitive landscape.

LITERATURE REVIEW

The first step we took was reviewing language-learning literature, so we could discover the evidence-based methods for teaching a foreign language. The literature review continued throughout the first half of the project as we found more and more papers that could be applied to our final solution. We investigated several topics, shown below.

Lit Review Topics

Insights

Reviewing the scientific literature helped us glean the following key insights. We incorporated many these insights into our refined solution, as you will see in the final design section.

COMPETITIVE ANALYSIS

We also conducted a competitive analysis to determine how we could differentiate ourselves from existing language-learning solutions. We looked at popular language learning apps like Babel, DuoLingo, and Rosetta Stone, as well as social language-learning platforms like Interpals and Conversation Exchange.

Findings: No Modern Social Language-Learning Solutions

The competitive analysis showed us that most popular language learning tools (DuoLingo, Babel, Rosetta Stone) emphasize repetitive vocabulary practice instead of immersion and social conversation. This is helpful for beginners but doesn’t bring a learner to fluency. It also doesn’t teach them about cultural intricacies like slang and humor.

The social language-learning platforms that do exist have outdated UI and poor UX. Thus, there are not many modern solutions to help people learn language in a more immersive, social context.

INITIAL CONCEPT

Our insights from the competitive analysis and lit review suggested that talking with native speakers in-context is key for successful language learning, and that gamification can be a great way to encourage users to stick to their learning goals. Based on these findings, we created our initial design concept:

A Social Language Learning “Pen Pal” App.

We wanted to build a social, gamified language learning app were users were matched with “pen pals” who speak a different language. This app would simulate immersion in user’s target language by enabling them to interact with native speakers and learn about their experiences.

SEMI-STRUCTURED INTERVIEWS

Once we had a well-informed direction for our initial concept, our next phase involved interviewing users about their language learning experience to get firsthand information about their needs and pain points. We interviewed 6 people who were currently learning a foreign language or had tried to learn a foreign language in the past. Much of what we found in the interviews supported insights from the literature review.

Insights

IDEATION AND SPEED-DATING

From these insights, we went back to the drawing board to ideate on our initial idea of a pen pal app. We created several storyboards exploring a range of potential applications of the “pen pal” idea, including a VR social language-learning experience. Interviewees had emphasized the importance of talking with native speakers and cultural immersion, so we incorporated these principles into many of our ideas.

We then tested storyboards with users to get feedback on potential solutions in a low-effort way.

Sample Storyboards

PIVOTING TO VIRTUAL REALITY

The results from speed-dating helped us pivot from our initial pen pal app idea. We discovered that users were most excited about a virtual reality language-learning experience that was truly immersive. They expressed concern about privacy and commitment issues with many of the “pen pal” ideas.

For this reason, we threw out the idea of a formal pen pal and updated our concept idea to the following:

A VR Social Language-Learning Game

We wanted to create a truly immersive VR experience where users could travel to different countries and learn language in “real-life situations.” We envisioned that this would be a massive multiplayer online game where users could be paired with people from different countries to learn conversational skills in new languages.

SKETCHING GAMEPLAY

After refining our initial concept, we began sketching out screen ideas user flows through the VR game. This helped my team sync on the final concept and move forward to experience prototyping.

EXPERIENCE PROTOTYPING

We decided that in order to test our concept, experience prototyping would be a great method to use. We felt that experience prototyping was the best way to simulate the interactive nature of VR.

We went through 2 rounds of experience prototyping, testing different levels of language-learning. At “level 2”, we incorporated conversations with native speakers.

Action Shots

REFINING UX

Feedback from experience prototyping encouraged us to revise our concept. While users deeply enjoyed the gamified and immersive nature of our solution, they felt some elements were confusing. Further, they expressed that many of the social interactions were unnecessary and uncomfortable.

Moving forward, we made the following changes based on user-feedback:

1. Narrowed our scope to focus on novice learners.

   We originally envisioned building out several levels of gameplay in our final prototype, catering to both beginner and intermediate learners. However, user-testing helped reveal that our scope was too broad; we were trying to do too many things. We decided to focus only on novice learners to simplify the project and encourage more detailed refinement of prototype components.

2. Eliminated real-time conversations.

   We originally envisioned Brocca as a massive multiplayer online game; users could talk to other players in the VR world and engage in “conversation challenges” with native speakers of their target language. However, test subjects said this was unnecessary; they didn’t care about interacting with other learners. Further, they stated that being forced to have a conversation with a native speaker was scary and difficult as a beginner, a sentiment supported by the “silent period” concept found in the literature review. This lead us to eliminate real-time conversations and interactions.

3. Made social component asynchronous, allowing players to generate language-learning content for each other.

   Though we eliminated real-time conversations, we still wanted the app to have social component to encourage cultural immersion. So, instead of including direct social interactions, we decided to make the interactions asynchronous, having users generate content that could later be seen by other users. This lead us to our final concept: a collaborative (rather than social) language-learning game where users could share cultural artifacts (photos, stories, music) in their native language to help each other learn.

4. Combined the “guidebook” and the “passport” into one tool.

   Originally users had two artifacts they could use to view their language-learning progress; a passport and a guidebook. Users were confused by these artifacts because they more or less served the same purpose. Thus, we combined them into one for the final prototype.

OVERVIEW

Brocca: A Collaborative Language-Learning Game in VR

Most language-learning classes and apps aren’t effective in helping people become fluent because they lack an immersive component; users learn the correct words, but not the intricacies necessary for proficiency.

Brocca the solution to this problem, immersing users in a foreign language. With Brocca, users can explore virtual cities where their target language is spoken. They learn how to apply words and phrases in real-life scenarios, whether it be ordering items from a restaurant menu or exploring popular landmarks. To encourage true cultural immersion, learners can interact with content uploaded by native speakers, such as photos, stories, music, and popular phrases. This enables users to learn the intricacies of language, such as slang and humor, that apps and schools just can’t match.

Brocca helps people learn both the words of a new language and culture that come with them, until they truly understand that language to its core. Including gamification makes language-learning even more engaging and immersive.

Software: We created this game for a VR headset with the capabilities of the Oculus Go. However, the concept could be subtly modified to work for many VR devices.

AN IMMERSIVE EXPERIENCE: Users get to “travel” to different cities around the world where their target language is spoken. They visit different landmarks and complete language-learning challenges in each of these cities. They can track their progress in their passport.

RATIONALE: We wanted to make the language-learning process as immersive as possible; both the literature review and user interviews suggested this helps people retain new language. By traveling to virtual cities in VR, users can feel as though they are really there, immersed in the culture.

CULTURAL EXPLORATION: Users can explore popular landmarks within virtual cities, reading/listening to information about them in their target language. If they do not understand what is being said, they can translate the content into their native language.

Although the app would auto-populate some of the landmark data, users native to these cities can also submit their favorite places to be seen as landmarks by other users, making the learning experience collaborative.

RATIONALE: Again, we wanted the experience to be truly immersive. Users can simultaneously experience the joy of visiting cultural destinations and learn the language used to describe them.

CHALLENGES SIMULATING REAL-LIFE SCENARIOS: Each landmark has language-learning challenges associated with it. Users earn XP for completing these challenges. These challenges require users to both recognize and produce language, including verbal language.

RATIONALE: Here, we emphasize the important of contextual learning. The literature review indicated that people retain language better when they learn it in real-life situations.

COLLABORATIVE LEARNING WITH NATIVE SPEAKERS: Users can upload content in their native language to share with other users, including photos, music and stories. Users can even contribute content used in learning challenges, such as photos and descriptions of food for the menu items in the restaurant challenge above. This makes the language learning process collaborative and enables people to learn about each other’s culture.

RATIONALE: Our interviews, as well as the literature review, stressed the importance of learning language from native speakers, who can teach the colloquial aspects of a language that textbooks and apps cannot.

LISTENING IN: At certain landmarks, users can observe a prerecorded session of two native speakers having a conversation. There is both verbal and text documentation of this conversation.

RATIONALE: Users told us that some of their best language learning came from just listening to native speakers talk (in videos and in person). Similarly, the lit review demonstrated that having a “silent period” where learners don’t produce any speech but instead listen to speech and absorb patterns is crucial for producing fluency. 

LEVELING UP: Once users complete all the language-learning challenges at a particular city, they level up and receive a stamp for that city on their passport. A new city then becomes unlocked for them to explore.

RATIONALE: The literature review suggested that gamification, when paired with an already compelling idea, will increase compliance. Gamified, positive feedback is especially important for novice learners to encourage continued progress towards a goal. Further, the competitive analysis indicated that many popular language-learning apps already use gamification.

NEW STYLES: Users can unlock avatar customizations as they generate cultural content (uploading a photo, music, etc.) for other players. This is meant to incentivize users to contribute. We also decided to “lock” music and photos posted by other users so people cannot see this content until they themselves contribute.

RATIONALE: We needed to think of a way to encourage users to contribute language-learning content to Brocca. After all, much of the platform’s richness comes from the collaborative elements. The incentivizing technique of avatar customization was a good choice because it is well documented in the scientific literature and popular games. Almost every digital game offers increasingly diverse and personalized avatar customizations as users level up.

REFLECTION

If I had more time, I would…

1. Build out more levels of gameplay. Currently, the Brocca experience is designed for beginner language-learners. Many of the learning challenges focus on teaching basic phrases and vocabulary. If I had more time, I would have liked to build out higher levels of language-learning.

2. Incorporate real-time conversations. Something we experimented with but did not end up including in our final prototype is the ability for users to have real-time conversations with native speakers. Practicing conversational skills is crucial for gaining fluency, and it would be good for higher-level language learners to have access to this feature. I would love to explore how we could integrate real-time conversations into Brocca.

3. Explore more accessible software implementations. Although we designed Brocca with the Oculus Go in mind, I would like to explore other platforms our solution could integrate with. Designing it for something like Google Cardboard would make it more accessible, as not everyone can afford to buy an Oculus Go.