Rock, Paper, Awesome (RPA) was Encore Lab’s initial foray into developing the means for tangible and embodied interactions that would connect to our S3 technology framework. The goal for RPA was simple; individual labs could create their own unique tangible or embodied interactions through which they played rock, paper, scissors with other labs that were physically distributed around the world.

The Theory

We chose rock, paper, and scissors as our test-bed because it provided us with not only a well-defined set of semantics (i.e., win, lose, draw, player ready, player gone), but also a very loose coupling in how we enacted those semantics. For instance how a player chose “rock” in one space could be entirely different from how a lab chose it in another (e.g., standing in a particular spot in the room, versus pushing a button). This allowed us to think deeply about what it meant to convey the same message through various tangible and embodied interactions, and to begin building an understanding of how these different interactions affected the meaning making of the participants. In essence we built a “reverse tower of babel” where multiple languages could all be interpreted through S3, allowing recipients at both ends to effectively communicate through their own designs.

Rock Paper Awesome

In this way, RPA is more than just a game of rock, paper, scissors – it is an avenue for us to begin investigating novel ways for users to interact with the world, and for connecting these investigations within a broader knowledge community. We aim to not only connect these communities, but also to add a layer of user-contributed design to their interactions, where community members engage in creative fabrication and exchange of tangible, interactive media that reflect their ideas, workflow or presence, bridging the distances and connecting the community.

Three critical questions guided our development of RPA and this component of S3 in general:

  • How can we bring distributed communities together through tangible and embodied interactions?
  • What are the possible roles for tangible and physical computing, and ambient or interactive media that are deeply connected to the semantics, workflow, physical presence, ideas, activities, and interests of the distributed communities?
  • How does the temporality of the interactions (synchronous versus asynchronous) determine the selection of appropriate kinds of interactions and representations?

We are currently sending out kits, first versions of the code, and design documents to labs at the Learning Technologies Group at the University of Chicago, and Intermedia at the University of Oslo. We are excited to see how they will develop and contribute new interactive designs that represent their own representations of space and meaning within the game.

The Technology

The physical interactions and ambient feedback is handled by an Arduino microcontroller. The Arduino allows users to develop a wide array of inputs (e.g. proximity, light, and sound sensors, buttons and levers), and outputs (e.g. sound, light, movement). Using the S3 framework, RPA facilitates different game “events” (e.g., joining the game, choosing Rock) by sending messages over an XMPP chatroom (conference). We originally attempted to implement these messages over the XMPP server only using the Arduino  – however, given the relatively limited amount of RAM on the Arduino board (2KB) this turned out to be overly restrictive and we started looking at other solutions.

As a solution to this issue, we made a simplified set of event messages (i.e., single text characters) that were sent over the Arduino’s serial port to a connected computer. For testing purposes we used a laptop. However, in permanent installations, we envision RPA having a more compact and flexible setup. In order to achieve this, we connected the Arduino board to a Raspberry Pi. The benefits of the Raspberry Pi is that it is small and cheap, allowing us to dedicate a Pi for each game installation, and to have the “brains” of RPA be as unobtrusive as possible.

In order to connect the various RPA installations we use node.js as an intermediary between the XMPP chatroom and RaspberryPI. Messages that are posted to the XMPP chatroom are picked up by the node.js server and sent over serial port to the Arduino, which then executes the user-designed action, such as turning on a light or playing a chime. Respectively, any event trigger on the Arduino (e.g. a button is pressed), is sent over the serial port to node.js and translated into a XMPP message.

Sample Arduino code for RPA and the node.js setup code can all be freely downloaded, tinkered with and customized from github.

The Run

We set up two “stations” at OISE, one on the third floor and one on the 11th floor. Players challenged each other to a game of rock, paper, scissors (see the video below).

Each location had different tangible, audible, and visual inputs and outputs providing players unique multi-modal experiences that conveyed the same message. At the third floor location, a “servo motor” swung a dial to let the player know a challenger was waiting to play. At the eleventh floor location, an LED flashed to convey the challenge. We have tested other designs (not shown here) that used proximity sensors to detect where players were within a room, using their location to trigger an event (such as choosing rock). In another instance, a light sensor conveyed one player’s availability to other players (in remote locations) when the lights in the original player’s room were on.

Going Live! RPA at TEI 2013

We submitted RPA to TEI 2013′s student design challenge. The conference was held in Barcelona Spain and provided an ideal opportunity for us to try out RPA (and S3) in a live setting with users who had no experience with it. We had stations running at the site and at labs site running in Toronto allowing us to observe a wide range of interactions and gain feedback from participants. We also added a new layer to RPA which connected a real-time visualization of win/lose/draw results to the game – although this visualization duplicated some of the functionality of the tangible RPA elements it did represent a significant step in merging the tangible elements of S3 with a key element of the existing architecture.