Unified Field collaborated closely with Mala Gaonkar, David Byrne, and the Todo Mundo team to turn their ideas into technology that enables an engaging and participatory performance. The design process was a very iterative one; as scripts evolved, we worked together on interface designs, functionality and programming that all support an immersive theatrical production staged at Pace Gallery’s Art + Technology site in Menlo Park.
We developed intuitive interfaces and control / presentation systems for two rooms within the experience. Our collective team learned, changed and improved upon our components until opening day and beyond.
As Pace Gallery states, The Institute Presents: Neurosociety can be described as “a series of interactive environments that Byrne and Goankar have created to present the emerging work of fifteen cognitive neuroscience labs around the world.” On davidbyrne.com, the team writes:
We assume that to experience an experiment provides a richer understanding of its concepts than being told about it. The word ‘experiment’ derives from the root word ‘experience’. So, to us, this series of immersive rooms in which one would have experiences seemed a logical way to engage with the work they were doing. Information is available everywhere, but experience is unique and has a growing value in a digitizing world. We decided each lab’s experiments should be translated into physical experiences.
We began by reading an in-process script. To turn their intentions and interactions into interfaces and presentation materials, we sat it on script readings and discussions to ensure that we remain in sync with the Neurosociety team’s iterations. The team describes the resulting activities as three acts:
Act I (Game Show: Moral Dilemmas) demonstrates how the vocabulary and grammar in our brains affects our moral behavior and our connections with others. Act II breaks these decisions down, and shows how our pre-conceived ideas build the grammar of our decisions—even electoral decisions (Classroom:Predicting Politics). Act III delves further into our sensory vocabulary (White Room: Equiluminance, Being Britta, and Rhodopsin).
We worked with the Neurosociety team on Act I (Game Show) and Act II (Classroom). By the time we began our collaboration, the team had already tested variations of these experiences within various venues over the last few years. They talked us through what worked and didn’t work within their previous iterations. They asked us to create a system that would allow them to perform “experiments” with attendees, record their responses and tie their responses to anonymous demographic information. The system was developed in a way that would allow it to become a mobile app in future generations of the immersive performance.
The system we created includes two rooms, 22 tablets locally running a custom mobile app (10 participants and one facilitator for each room), 2 servers, 10 Arduino-controlled coin receipt/distribution systems (for Game Show), projected content in one room, content presented on multiple large screens in the other room, and an anonymous demographics registration tablet that connects to a wristband printer within the entry. The facilitator tablets for each room control the movement through the steps of the experiments, and the participants log in to their individual tablets upon entering each room, so that their answers are tied to their user code within the master database. We also designed interfaces for all of the devices and visualization methods for the presentation of activities and reflection of responses.
Upon entering the experience, participants either anonymously enter demographic information or may decline to do so. Each participant receives a wristband that includes a unique user code; they wear this for the duration of the experience. Visitors to Neurosociety are first guided to Game Show, where they participate in a series of “moral dilemmas,” answering questions posed by their guide. Their responses are immediately registered and reflected back to the group, and they discuss their answers juxtaposed to other participants’ answers given over time.
Within the Classroom, participants respond to pairs of photographs of political candidates, guessing which person won based solely on appearance. As with Game Show, the participants’ decisions are reflected back to them. In this case, they’re shown the accuracy of their answers and their responses in relation to those logged by all participants over time. This data serves as points for discussions, led by the guides, about personal bias.
In addition to designing appropriate interfaces and implementing the overall control system, we worked with the Neurosociety team to address challenges that they’d encountered in past iterations. One of their previous challenges was with the “fairness games” used within their Game Show room. These games involved “money,” and tracking the amounts given and received by each participant became cumbersome. They asked if we could make something analogous to an Automatic Teller Machine that would allow them to track the physical, metaphoric funds more smoothly.
Our team proposed and built an Arduino-driven coin exchange system. Individual Arduinos placed at each participant’s Game Show podium controlled hardware elements sourced from slot machine providers. Todo Mundo produced Neurosociety coins, and these coins were delivered or received from participants, counted by the setup we created, and reflected both within the participants’ individual devices and overall metrics for the group.
In addition, our team enthusiastically looked for opportunities for both improvement and iteration. One of the Game Show “moral dilemmas” uses a brief clip from the TV show Star Trek to embody an idea. During our design phase, we discovered that one of our designers, a massive Star Trek fan, owns an elaborate collection of action figures for the exact episode’s characters. We staged a photo shoot to enliven the content for that section of the performance; in addition, we used those photos as the basis for icons we created for use in the answers reflection visualization.
We also built a robust system of potential pathways within Game Show, which allowed guides to make choices on the sequence of activities depending on the proceedings of each individual group. Certain activities were performed by all groups, whereas the paths of others were determined either automatically by the results of proceeding activities or decided by the guides at key points.
This multi-path system allowed the Neurosociety team to iterate and determine what they felt were the best sequences of events within the time allotted for each performance. Through user testing and performance run-throughs, the system allowed the team to revise their approach through multiple configurations to land on a final iteration that included only a portion of what we originally built.
As a highly interactive live performance, the team sometimes learns new things about both user interactions and the impact of the site. Beyond installation and initial performance run-throughs, all of which informed refinements, we continue to work with the Neurosociety team to revise both the interface of the control system and items within the technical build that need to address the real-world conditions of a non-traditional space (in this case, the renovation of a former Tesla dealership). We continue to both learn and troubleshoot alongside the Institute team, working to make the system responsive, invisible, and as seamless as possible to enable the magic of the immersive theatrical experience.