Asystole by Mirabelle Jones

Asystole is a 3D-printed interactive sonic book capable of reading your heart rate from the blood flow in your face using computer vision and turning it into music. Each EKG pattern printed in the book’s surface corresponds to a sound sample which is triggered to play via light sensors as the pages are turned. As you read Asystole, the sound of your heartbeat will play against the waveform of the EKG pages to form a duet between your body and the book. The book is dedicated to heart health awareness and making the unseen consequences of our decisions in regards to our biological well-being visible and audible.

Each page of Asystole, presenting as a sculptural “slice,” is a 3D printed waveform visualizing different EKG patterns: ventricular fibrillation, atrial fibrillation, atrial flutter, tachycardia, torsades de pointes, nodal rhythm and sinus rhythm (a regular heartbeat). To ensure the accuracy of these 3D printed forms and to learn more about their implications, a combination of medical students, cardiatric surgeons, and registered ER nurses were consulted. Each page rotates around a cylindrical spine with an embedded speaker. Underneath each page is a photo cell light sensor capable of sensing when the page has been turned, triggering a sound sample of that waveform to play. When all the pages have been turned, an asystole (or flat-line) can be heard. The book uses computer vision to determine the heart rate of the reader. While the reader is reading the book, their face remains mostly still. This allows a program written in C and R which uses a process known as Eularian Video Magnification elaborating on research by MIT and Dwight Goins of Microsoft, to determine the reader’s heart rate based on subtle changes in the RGB values, IR values, and amplified motion of their face using the Kinect’s sensor array. Once determined, the heartbeat is played as a sound alongside the current EKG pattern of the book creating a duet between book and reader. This project is dedicated to my cousin Kimmie who died prematurely of a heart attack. In the months of research that followed, I would learn that heart disease is the number one cause of death in America. It is my hope that Asystole will encourage us to think about the unseen consequences of our decisions to our biological health at a time when we are often disassociated from our corporeality through our immersion in digital technology.