Asystole by Mirabelle Jones


Asystole is a 3D-printed interactive sonic book capable of reading your heart rate from the blood flow in your face using computer vision and turning it into music. Each EKG pattern printed in the book’s surface corresponds to a sound sample which is triggered to play via light sensors as the pages are turned. As you read Asystole, the sound of your heartbeat will play against the waveform of the EKG pages to form a duet between your body and the book. The book is dedicated to heart health awareness and making the unseen consequences of our decisions in regards to our biological well-being visible and audible.

Each page of Asystole, presenting as a sculptural “slice,” is a 3D printed waveform visualizing different EKG patterns: ventricular fibrillation, atrial fibrillation, atrial flutter, tachycardia, torsades de pointes, nodal rhythm and sinus rhythm (a regular heartbeat). To ensure the accuracy of these 3D printed forms and to learn more about their implications, a combination of medical students, cardiatric surgeons, and registered ER nurses were consulted. Each page rotates around a cylindrical spine with an embedded speaker. Underneath each page is a photo cell light sensor capable of sensing when the page has been turned, triggering a sound sample of that waveform to play. When all the pages have been turned, an asystole (or flat-line) can be heard. The book uses computer vision to determine the heart rate of the reader. While the reader is reading the book, their face remains mostly still. This allows a program written in C and R which uses a process known as Eularian Video Magnification elaborating on research by MIT and Dwight Goins of Microsoft, to determine the reader’s heart rate based on subtle changes in the RGB values, IR values, and amplified motion of their face using the Kinect’s sensor array. Once determined, the heartbeat is played as a sound alongside the current EKG pattern of the book creating a duet between book and reader. This project is dedicated to my cousin Kimmie who died prematurely of a heart attack. In the months of research that followed, I would learn that heart disease is the number one cause of death in America. It is my hope that Asystole will encourage us to think about the unseen consequences of our decisions to our biological health at a time when we are often disassociated from our corporeality through our immersion in digital technology.


Artist Bio

Mirabelle Jones is a queer, non-binary creative technologist, interdisciplinary artist, and researcher based in Copenhagen investigating critical, creative practices in technology. Their work explores the immersive storytelling potential of sensors, spatialized sound, LEDs, animatronics, XR, wearables, artificial intelligence, natural language processing and computer vision. They are a PhD candidate at The University of Copenhagen in the Department of Computer Science (DIKU) within the Human-Centered Computing section and possess an M.F.A. in Book Art & Creative Writing from Mills College and a BA in Literature from UC Santa Cruz. A proponent of STEAM education and interdisciplinary development, they are a frequent speaker on the subject of accessible, ethical, and feminist design practices. Their works have most recently been featured at Catch: Center for Art, Design, and Technology, ATA Gallery, the Museum Meermanno and the Center for Performance Research and appear in several collections including the One National Gay & Lesbian Archives and the Center on Contemporary Art’s historic Hear Our Voice collection. Their performances and visual works have been heralded by the Huffington Post, ArtNet, Ms. Magazine, Ingeniøren, Bustle, ATTN, Refinery29, Inquisitr, Mic., Sleek Magazine, Feminist Magazine, Deutsche Welle, Google News, Yahoo News, PBS, Roundtable Journal, Tip, Berliner Zeitung and elsewhere.