Network Music Performance During COVID-19 and Beyond: A Quick Review of Available Software

During this COVID-19 period with an unprecedented global lockdown, I’ve been asked for advice about suitable software to use for online concert collaboration. This blog post is an illustrative list of available software, which might be useful to others not only for this period, which hopefully will end up soon, but also for the next future where we might need the keep a scenario of physical distance as the new norm.

What makes a good software for network music performance or telematic performance?

When selecting the software for network music performance (NMP),¹ it is worth mentioning two criteria that you might check before making a decision, based on what it is commonly expected from a computational system that supports collaboration, particularly collaborative performance:²

• Low latency:³ In computational systems, latency refers to any delay during the transmission of information from the sender to the receiver. It is therefore a temporal measure. The ideal low latency is often compared to the speed of light, which is the fastest speed at which information can travel in a network. Network latency is an example of communication system’s latency, where the medium determines the velocity of transmission. This can be measured either one-way (from sender to receiver) or round-trip (from sender to receiver and back). Ways of measuring it include sending a ping back to the sender once the received packet is received.
  Low latency is essentially the Holy Grail of network music, and more generally, real-time audio software. For the purpose of collaborative music making, a consensus of accepted latency is about less than 25ms, also known as the ensemble performance threshold (Renaud et al., 2007), which is also expected in co-located experiences due to other reasons such as the acoustics of the room and the time need to reach a coordinated stable tempo (Chafe and Gurevich, 2004). However, when introducing the Internet network as a component of the system, it is difficult to keep it that low due to the sum of several latencies that affect the total latency, including not only the Internet network but also the audio hardware and the NMP software used, among others.
  Some of the software establishes network delays of more than 25ms (e.g. 200ms) as part of the aesthetics of the NMP, which is termed by Renaud et al. (2007) as latency accepting approach (LAA), which contrasts with a realistic jam approach (RJA).

• Workspace Awareness (individual and group): Defined as “the up-to-the-moment understanding of another person’s interaction with a shared workspace” (Gutwin & Greenberg, 2002, p. 412), in music it is also important to simultaneously distinguish between the individual’s voice and other participants’ voices emanating from the shared workspace (Hancock et al., 2005). A set of mechanisms have been suggested in the computer-supported cooperative work literature to be successful with collaborative applications, namely supporting (1) group coordination, (2) data sharing, and (3) the provision of information about group and individual actions (Dourish & Bellotti, 1992).

Other software-related aspects that are worth analysing are whether the software is open source, whether it is multi-platform, the cost, the audio (and video) quality, the level of privacy, and any other relevant technical details. The type of ensemble (e.g. electronic music, purely acoustic, or a combination) will also determine the technical needs, with software more suitable than other. It is out of the scope of this blog post to discuss other related criteria more inherent to digital music performance, such as music expressivity, tangibility, performativity and technicality.

Relevant software for network music

This list proposes three major applications that can be used in NMP: (1) generic videoconference systems, (2) rehearsal / jamming systems, and (3) live streaming systems. Generic videoconference systems refer to systems designed for video conferencing that eventually can be used for music rehearsal or jamming, but that have not been designed for this purpose and therefore might miss some desirable properties for a successful NMP. Music rehearsal / jamming systems refer to systems specifically designed as audio (and visual) solutions for NMP, typically focusing on the participants of the ensemble. Live streaming systems refer to platforms that are designed for broadcasting multimedia content to a large audience.

These categories are not mutually exclusive, because (1) and (2) are generally used by the group of performers, and (3) is a means of making the event public to a large audience or for archiving purposes. However, in order to avoid repeating the same software in different groups, each is classified by its major application.

The examples are illustrative, as there are many more solutions out there, as well as audio applications that support collaboration in their own environments (e.g. collaborative live coding in live coding environments, jamming apps for iOS / Android, and so on) and new software that is emerging from this worldwide critical situation, which would deserve another dedicated blog post.⁴

Generic Video Conferencing

• Zoom - Zoom is a video conferencing and instant messaging multi-platform solution that has been used for jamming. Lately from its worldwide use during COVID-19 some issues have been raised about the need to implement end-to-end encryption. At the time of writing this blog post, the free solution only allows 1:1 meetings whilst the paid solutions support 100 participants and above (up to 1,000 depending on the plan).
• Jitsi - Jitsi is a free and multi-platform video conferencing and instant messaging solution. Information is encrypted. Desktop streaming is also possible. At the time of writing this blog post, it is reported on the forum of the Jitsi community that more than 35 participants can start to be a problem, with a hard limit at 75 participants. Livestreaming the event is recommended for the potential audience instead, which supports millions of participants.

Music Rehearsal / Jamming

• Jacktrip - Jacktrip is a free, multi-platform, high-quality audio solution that supports bidirectional audio streaming, where it is possible to send as many channels as possible (depending on the computer and network). JackTrip is based on JACK as its audio server. It is possible to control the packet size or frames/period (latency) and the sampling rate (bandwidth), which gives a fine grain of control of the audio quality. This indicates that this software is designed to support an RJA approach.
• Soundjack - Soundjack is a free (accepting donations), cross-platform, real-time communication software that provides high-quality (uncompressed) audio that is configurable and at the time of writing this blog post, video at a beta stage. Audio servers supported include JACK, Soundflower, and Loopback, among others, with preference for JACK. Latency is configurable at a fine grain level (audio buffer, network buffer, jitter buffer). This software is designed to support an RJA approach.
• eJamming - eJamming is a paid software with one month free trial. It works for Windows or Mac. It supports up to 4 musicians, depending on the bandwidth. The musicians are expected to connect with their musical instrument and a microphone plugged to their computer’s sound card or external audio interface. There are plans to support live streaming. It is possible to record the sessions (16 audio and 16 MIDI tracks available). There is a chat. This software is designed to support an RJA approach.
• LoLa - LoLa is a low latency, high quality audio and video transmission system designed for remote rehearsal and performance. It also supports distributed audience. LoLa is available for free in academic and education non-profit environnments. In the other cases, it has a shareware license. LoLa has specific hardware and network requirements. This software is based on RJA.
• NINJAM - NINJAM is an open source (GPL) and cross-platform software. The Reaper DAW includes NINJAM support, as well. It uses compressed audio and it is possible to work with one or multiple instruments. This environment allows users to set the latency to higher values, and accept that the Internet latency simply cannot provide real-time synchronisation, thus providing a new way of jamming based on playing previously recorded buffers. Latency is measured in measures, which is a more musical approach than other solutions. This exemplifies a LAA. NINJAM uses a central server. There is a chat. It is unclear what is the limit of participants.
• Quintet.net - Quintet.net is a multimedia performance platform developed on Max/MSP that supports up to 5 musicians under the supervision of a conductor role (server) and an additional visualist can join as well. The musicians can send MIDI control data via a microphone (by means of a pitch tracker) or a (MIDI) controller. Using a computer keyboard is also possible. The software uses a sampler or MIDI for instrumental playback and brings Max/MSP features such as granular synthesis, plug-ins and visuals. It has a built-in chat feature that allows the audience or listeners to participate (unlimited number). This software also follows the LAA.
• Locus Sonus - Locus Sonus is a platform designed to support collective listening experiences and audio mobility. The Locus Sonus Stream Project offers a global network where participants become “open mikes” who can stream local soundscapes to a central server. The related software Locuscast is free and designed for App iOS and Android. It requires to create an account and the stream can be located on the Locus Sonus sound map. An available Pure Data patch can also be used. This software aligns with the LAA.

Live Streaming

• OBS - OBS is a free, open-source, cross-platform broadcasting software that is built with Qt. It is possible to stream to Twitch and YouTube, as well as other providers. OBS is used to capture and record your screen while also capturing audio. It is possible to assemble different visual elements on the screen as well as create transitions to different scenes that can be launched in real time.
• Icecast - Icecast is a free software for streaming media. It is basically a streaming media (audio/video) server that supports different types of streams (e.g. Ogg, Opus, WepM, MP3). It is typically used to create Internet radio stations, among different applications. The software requires “source clients” to start the streams. Icecast latency tends to be a lot larger than the rest of programs described in this blog post, but it provides an easy start.

Final words

NMP is a broad topic, as well as latency and related concepts. The aim of this blog post was to provide a quick summary of available software. This should be a starting point for those interested in beginning to explore the possibilities. There are a number of tutorials online about how to measure latency using different software, such as the latency test with Audacity. A combination of various platforms can also be interesting, as we explored with Female Laptop Orchestra (FLO) & Women in Music Tech (WiMT) @ Audio Mostly 2017, with WoNoMute and at the NTNU/UiO’s MCT master program. Although the current conditions are extreme and unwanted, we can take it from the positive side of finding new ways of making music and collaborate.

Editorial note. Amendment on December 27, 2021. It is also worth checking Jamulus, a free multi-platform client-server based software written in C++ that transfers compressed audio with configurable latency at a professional level and with support for a synchronized metronome. Documentation is provided in English, Spanish, German, Italian, French and Portuguese. SÁGORA, another free multi-platform software for transferring high-quality audio with low latency with documentation both in English and Spanish. Both seem to support an RJA approach.

Acknowledgements

Thanks to Gerard Roma for feedback during the writing of this blog post. Thank you to Thor Magnusson for suggesting new software reflected on the editorial note.

References

• Chafe, Chris, and Michael Gurevich. “Network time delay and ensemble accuracy: Effects of latency, asymmetry.” Audio Engineering Society Convention 117. Audio Engineering Society, 2004.
• Dourish, Paul, and Victoria Bellotti. “Awareness and Coordination in Shared Workspaces.” Proceedings of the 1992 ACM Conference on Computer-Supported Cooperative Work. 1992.
• Gutwin, Carl, and Saul Greenberg. “A Descriptive Framework of Workspace Awareness for Real-time Groupware.” Computer Supported Cooperative Work (CSCW) 11.3-4 (2002): 411-446.
• Hancock, Mark S., et al. “Exploring Non-speech Auditory Feedback at an Interactive Multi-user Tabletop.” Proceedings of Graphics Interface 2005. Canadian Human-Computer Communications Society, 2005.
• Renaud, Alain, Alexander Carôt, and Pedro Rebelo. “Networked Music Performance: State of the Art”. Proceedings of the AES 30th International Conference. Saariselkä: Finland, 2007.