Jolly Roger is a puzzle hunt team management and puzzle solving productivity application developed by members of the MIT Mystery Hunt team Death and Mayhem. It is a volunteer-run Meteor/React app and has been in continuous development since 2014. I started consulting on the user interface (suggesting and implementing UX improvements) in 2020.

One of my biggest contributions was the audio call UI, which was designed and developed in time for the January 2021 Mystery Hunt (the first remote MIT hunt, due to the pandemic).

Goal

The goal was to attractively and coherently fit useful audio call participant information into a 40x40 box, with which to build a compact audio call UI.

Constraints

The Jolly Roger puzzle page UI has to be as compact as possible to leave enough room for the embedded Google Sheet (where almost all puzzle solving collaboration happens) within the limited real estate of a single laptop screen. (Most years, when the Mystery Hunt is on-site, most users will be working from a laptop rather than from home where they may have larger external monitors.) The audio call box would have to share the left sidebar with the existing text chat section, so it could neither be any wider than the chat UI nor excessively tall.

Given its vintage, Jolly Roger uses React Bootstrap as its UI library. Obviously there's no built-in audiovisual UI elements in that library, but whatever design I made would have to look congruous with the rest of Bootstrap.

Background research

Video call support was not in scope for this project, but I took inspiration from the (now defunct) Google Chat's video call UI (and anti-inspiration from Zoom's). I liked how compact Google Chat made its viewer boxes in shared-screen mode, and how it shows who is talking/making sound with a volume bar-esque overlay (rather than a subtle outline in Zoom's Brady-Bunch mode, which also cannot appear on more than one user at a time). Google Meet's colorful, edge-to-edge initials-box for users with video turned off was also much more attractive than the equivalent Zoom placeholder (a tiny avatar or full name floating in an awkwardly-proportioned black rectangle).

Discord's audio chat UI was also something I looked at, but I decided not to go that route. Although each user-item in Discord's version is very compact vertically, because it's one user per row the total size is typically taller than a tileable interface for a typical number of participants solving a given puzzle. A Discord-style list wasn't going to leave enough space for the text chat box below unless we added scrolling, which we wanted to avoid if possible so that you wouldn't miss people talking if they were off the edge of the visible part of the list.

UI design

In the background of each user box: either a solid color or the user's Discord avatar (if they had connected their Discord account to their Jolly Roger account).

On top of that: a volume bar visualization. We got the frequency distribution of the sound for free with the WebRTC implementation the first version of the audio UI used, but I frobbed it a little by divvying it up into bar-buckets, then setting a minimum decibel level before a given bar would appear at all. The latter was meant to reduce the amount of extraneous, nearly-soundless flickering from random microphone noise which was distracting and visually noisy. I made the volume bars "Bootstrap blue" with a slight gradient at high volumes to subtly emphasize louder (talking volume) sounds.

On top of the volume bars in the first version of the audio UI was a three-pixel line at the bottom of the box, whose color indicated what state that user's WebRTC connection was in. This was handy for debugging (especially as users helped us discovered more ways that a browser or network setup could break calls) and provided an additional buffer against excessive volume flickering. (In a later version of the software, the audio call backend changed such that the connection statuses stopped being relevant concepts, so the bottom lines are no longer present.)

On top of that: a layer for a muted icon, if applicable, absolutely positioned in the corner. Users could mute themselves as needed, and in a later version users could mute other users (if e.g. they'd stepped away from the computer and were causing lots of background noise).

Finally: the user's initial, either from their name (if they'd entered it into their Jolly Roger profile) or their account handle. Hovering over a person-box would show their full name/handle as alt text along with their status (if they were muted).

For the box that represented "me" (the viewer's user account), the box would have a narrow dark blue outline and a slight box-shadow.

This was also a chance to expand on a previous feature where we could track who was viewing a given puzzle. Before, we only used this to show a count of how many people were viewing (to give a sense of whether this puzzle already had a lot of people working on it or if it needed more attention). Now we could use the same "person-boxes" from the audio UI to show who was viewing the puzzle page, to help solvers identify and form relationships with their teammates even if they were solving remotely.

At this size, the person-boxes were easy to tile, and even when wrapping they avoided crowding out the text chat part of the sidebar below them.

UX design

In the initial version of the audio call box, the developers put three buttons at the top of the box after joining: "Deafen", "Mute", and "Leave". "Deafen" was a concept borrowed from Discord (for occasions where you wanted to watch or stream to a voice channel, but didn't want to be distracted by the actual talking in the channel). The developers added "deafen" mode here because it made it easier to test audio calls with multiple speakers (you'd open different accounts in different browsers locally, then have them all join the same call while muted and deafened so that you wouldn't get audio feedback). However, this feature was irrelevant for our users--if you didn't want to listen in, you could leave the call at any time without losing any other context (we didn't support streaming or video or anything of the sort).

Eventually I convinced the developers to let me hide the "deafen" button behind a dev-mode flag, so that it wouldn't appear the rest of the time. This saved a lot of space for the other two buttons and let us have more descriptive button labels.