Search your favorite song right now

1. Viral Counterpoint of the Coronavirus Spike Protein (2019-nCoV)

Viral Counterpoint of the Coronavirus Spike Protein (2019-nCoV)

While we cannot see small nanoscopic objects like proteins or other molecules that make up virtually all living matter including our cells, tissues, as well as pathogens such as viruses, our computational algorithm allows us to make its material manifestation audible. This piece is a musical representation of the amino acid sequence and structure of the spike protein of the pathogen of COVID-19, 2019-nCoV (protein data bank identifier 6VSB [1]). MUSICAL DECEIT A virus’ genome hijacks the host cell’s protein manufacturing machinery, and forces it to replicate the viral genome and produce viral proteins to make new viruses from it. This musical art teaches us something about the fine line between beauty of life and death as an opposite pole. As you listen to the protein you will find that the intricate design results in incredibly interesting and actually pleasing, relaxing sounds. This doesn’t really convey the deadly impacts this particularly protein is having on the world. This aspect of the music shows the deceiving nature of the virus, how it hijacks our body to replicate, and hurt us along the way. So, the music is a metaphor for this nature of the virus to deceive the host and exploit it for its own multiplication. HOW IT IS DONE: What you hear is a multi-layered algorithmic composition featuring both the vibrational spectrum of the entire protein (expressed in sound and rhythmic elements), the sequence and folding of amino acids that compose the virus spike structure, as well as interwoven melodies - forming counterpoint music - reflecting the complex hierarchical intersecting geometry of the protein [2]. Scientific references: [1] Wrapp et al., "Cryo-EM structure of the 2019-nCoV spike in the prefusion conformation," Science, 2020, DOI: 10.1126/science.abb2507 [2] Buehler et al., "A Self-Consistent Sonification Method to Translate Amino Acid Sequences into Musical Compositions and Application in Protein Design Using Artificial Intelligence," ACS Nano, 2019, DOI: pubs.acs.org/doi/10.1021/acsnano.9b02180

nothing at of , which is


2. Koto Ensemble of Protein 1akg (Venom of Predatory Marine Snails)

Koto Ensemble of Protein 1akg (Venom of Predatory Marine Snails)

What you hear is the sonified expression of the 3D structure of the protein with PDB ID 1akg (ALPHA-CONOTOXIN PNIB FROM CONUS PENNACEUS, see: http://dx.doi.org/10.2210/pdb1AKG/pdb), translated into an orchestral setting (koto [a Japanese string instrument], cello, double bass, viola, flute, and fluegelhorn). The koto is used to express the primary, secondary and tertiary structure of the protein. The piece reflects a classical period model of the hierarchical mapping to describe the 3D folded structure of a protein in musical space. The musical mapping begins from the primary amino acid sequence (GCCSLPPCALSNPDYCX) and encodes multiple hierarchical structural levels, and adds musical experimentation. We use the concept proposed by Yu, Buehler et al. (ACS Nano) to map amino acid sequences to music, and augment the mapping by adding more data. The 3D folded structure of a protein is coded by inserting strummed chords and interwoven melodies that code for the geometric arrangement of amino acids, the protein's tertiary structure. Rhythm and note volume code for secondary structure. [1] The protein used for this example is 6cjc (CRYSTAL STRUCTURE OF A FC FRAGMENT LALA MUTANT (L234A, L235A) OF HUMAN IGG1 (CRYSTAL FORM 3)) [2] https://doi.org/10.1021/acsnano.9b02180

nothing at of , which is


3. Last Move 84: Silk Protein Sonification

Last Move 84: Silk Protein Sonification

Sonification of a silk protein, rich in beta-sheet secondary structure. Reference: [1] C.H. Yu, Z. Qin, F. Martinez, M.J. Buehler, A Self-Consistent Sonification Method to Translate Amino Acid Sequences into Musical Compositions and Application in Protein Design using Artificial Intelligence, ACS Nano, in press, 2019

nothing at of , which is


4. Orchestra Of Amino Acids

Orchestra Of Amino Acids

This piece is constructed entirely from the sounds generated from amino acids, the building blocks of proteins. Encoded by the DNA sequence, amino acids form chains that assembly into hierarchical materials that serve diverse functions in biology (as enzymes, structural materials, signal transmitters, and many others). All sounds used to compose this piece are generated from the sounds of amino acids, computed from their quantum mechanical vibrations, reflecting their elementary molecular features. For further details on the scientific foundation, see [1]. The main melody is derived from the amino acid sequence of three proteins: 1) 6cjc (adrenodoxin reductase, a flavoenzyme that is involved in the biosynthesis of steroid hormones; https://www.rcsb.org/structure/1CJC) 2) A de novo protein synthesized using a machine learning algorithm as reported in [1] 3) 194l (lysozyme enzyme; found in tears of humans and animals and egg white; https://www.rcsb.org/structure/194L) The melodic backbone is composed of a progression from proteins 1) to 2) to 3). Motifs derived from these proteins are used throughout to generate repetitive elements and to conjoin melodic concepts (e.g., segments extracted from 6cjc are used as transitional elements between 1cjc and the de novo protein and again during the transition to 194l. All rhythmic elements (bass drum, snare-like, hi-hat, and others) are generated from soundings of amino acids. Both melodic and rhythmic patterns used are derived from the three proteins cited above. No other synthetic or sampled sounds have been used in the composition. What you hear is a composition in the natural 20-tone amino acid scale. BACKGROUND: Materials and music have been intimately connected throughout centuries of human evolution and civilization. Indeed, materials such as wood, animal skin or metals are the basis for most musical instruments used throughout history. Today, we are able to use advanced computing algorithms to blur the boundary between material and sound and use hierarchical representations of materials in distinct spaces such as sound or language to advance design objectives. The approach used in this work is that the translation of protein materials representations into music not only allows us to create musical instruments, but also enables us to exploit deep neural network models to represent and manipulate protein designs in the audio space. Thereby we take advantage of longer-range structure that is important in music, and which is equivalently important in protein design (in connecting amino acid sequence to secondary structure and folding). This paradigm goes beyond proteins but rather enables us to connect nanostructures and music in a reversible way, providing an approach to design nanomaterials, DNA, proteins, or other molecular architectures from the nanoscale upwards. Reference: [1] C.H. Yu, Z. Qin, F. Martinez, M.J. Buehler, A Self-Consistent Sonification Method to Translate Amino Acid Sequences into Musical Compositions and Application in Protein Design using Artificial Intelligence, ACS Nano, in press, 2019

nothing at of , which is


5. Coronavirus spike protein spectra including interaction with human ACE2 receptor

Coronavirus spike protein spectra including interaction with human ACE2 receptor

What you hear is a comparison between two coronavirus protein spikes, here focusing on their overall frequency spectrum using the method we published in EML in 2019. The left image, heard first, is from HKU1 (a human betacoronavirus that causes mild yet prevalent respiratory disease). The right image is from COVID-19, and heard second. You can see and hear that the novel coronavirus spike protein has a much lower frequency spectrum, reflecting slower motions. Listen to this piece for a fuller musical assessment of the entire spike protein in the virus: https://soundcloud.com/user-275864738/viral-counterpoint-of-the-coronavirus-spike-protein-2019-ncov

nothing at of , which is


6. Counterpoint Through Fracture: Part II - Stress Tensor Variations

Counterpoint Through Fracture: Part II - Stress Tensor Variations

For piano. We are researching the use of algorithmic composition to generate counterpoint through fracture. Driven by the universal physical forces near a singularity, captured in the so-called Cauchy stress tensor (read more about it here: https://en.wikipedia.org/wiki/Cauchy_stress_tensor), we are creating interdependent polyphony through independent patterning in time and pitch. FRACTURES ARE UNIVERSAL A fracture is a crack in a material (often seen in glass, but also represented in earthquakes when tectonic plates fracture, or in the unfolding of a protein). Fracturing is the process of destruction of structure, and hence is the division between function and failure in material and has correspondences in philosophy. The tipping point of the crack, the locality of greatest action, and where bond breaking occurs, is the singular crack tip, the sharp edge. All fractures feature a universal distribution of stresses – the forces that act inside the material and which stretch/compress chemical bonds. No matter what material, as you get closer to the singularity – the tip of the crack – the distribution of forces are identical. In this study we explore these universal features to generate algorithmically composed music. We try to explore whether the universal features of fractures have the capacity to generate counterpoint – universal features of music, to better understand the analogy across manifestations. Read more about the physics of fracture in [1]. HOW THE MUSIC IS CREATED The piece is constructed from multiple movers in the singular stress field, and using the full stress tensor to derive musical structure. The observes move in circular fashion around the singularity in an oscillatory motion. Each individual note played is generated based on the stress tensor, a physical quantity that describes the forces inside a material. Large stresses pull notes into high pitch, lower stresses compress it to lower pitches. The violent nature of forces experienced by the observer circling around the crack tip generate complex but highly organized melodic forms. As the stress field features universal, respective patterning, the various melodic lines generated have relationships rooted in the nanoscale physics surrounding the singularity. Fracture-generated counterpoint hence may reflect the universal features of fractures and provides a model thereof. Owing to the universal significance of fracturing as a process in material and amaterial manifestations, the musical effects generated hereby can be an interesting subject of further analysis and inspiration. [1] M. Buehler, Atomistic Modeling of Materials Failure, 2008 www.springer.com/gp/book/9780387764252

nothing at of , which is


7. Sonification of the Coronavirus Spike Protein (Amino Acid Scale)

Sonification of the Coronavirus Spike Protein (Amino Acid Scale)

This piece is a sonification of the spike protein of the pathogen of COVID-19, 2019-nCoV (protein data bank identifier 6VSB [1]). What you hear is a multi-layered sonification approach featuring both the vibrational spectrum of the entire protein (expressed in sound and rhythmic elements), as well as the sequence and folding of amino acids that compose the virus spike structure [2]. References [1] Wrapp et al., "Cryo-EM structure of the 2019-nCoV spike in the prefusion conformation," Science, 2020, DOI: 10.1126/science.abb2507 [2] Buehler et al., "A Self-Consistent Sonification Method to Translate Amino Acid Sequences into Musical Compositions and Application in Protein Design Using Artificial Intelligence," ACS Nano, 2019, DOI: https://pubs.acs.org/doi/10.1021/acsnano.9b02180

nothing at of , which is


8. Spider Playing With Proteins

Spider Playing With Proteins

Can a spider, a water-borne parasite of humans, a protein from rabbit, and a protein from a virus play together in a sort of virtual orchestra and link species and scales? Yes! This piece consists solely of AI-generated sounds generated from the spider web construction recordings and sounds of various protein vibrations (including the drums used to generate rhythmic elements, it is all derived directly from protein vibrations, the melodic part, and so on – they are all created from various proteins). The vibrational spectra heard here do not follow any of the known musical scales but rather, define their own, and interact with the sounds generated by the spider AI. Music allows us to achieve a sort of normalization of the various patterns from the nanoscale to macroscale, and make the beauty of nature accessible to us, and to allow for interspecies and interscale communication in a musical sense. This work is a collaboration with Tomas Saraceno Studio in Berlin, Germany. Credits for original spider sounds used in the composition: Tomas Saraceno Studios, Berlin, Germany. The spider sounds recorded during web construction by Studio Saraceno were used to train an artificial neural network - the "spider AI". The spider AI is then used to generate new web construction sounds that are used in the composition of the music, and which interplay with the sounds of proteins. The music is composed of the sounds generated from three proteins with Protein Data Bank (PDB) identifiers 5iom, 1sve and 4rga, representing the transcendence from material to sound, and across species. In addition to the AI evolved spider sound (the star!), the three proteins used to generate sound are: 1)5iom protein: Nucleoside Diphosphate Kinase from Schistosoma mansoni (a water-borne parasite of humans, and belongs to the group of blood flukes), learn more here: en.wikipedia.org/wiki/Schistosoma_mansoni (image of track from that website) 2)1sve protein: Protein Kinase A in Oryctolagus cuniculus, Bos Taurus (rabbit). The enzyme catalyzes the transfer of a phosphate group from adenosine triphosphate (ATP) to a specified molecule. ATP is a complex organic chemical that provides energy to drive many processes in living cells. Learn more here: en.wikipedia.org/wiki/Kinase 3)4rga protein: Phage 1358 receptor binding protein (rare group of phages infecting Lactococcus lactis). Learn more about phages here: en.wikipedia.org/wiki/Bacteriophage

nothing at of , which is


9. Collagen Etude

Collagen Etude

Collagen constitutes one-third of the human proteome, providing mechanical stability, elasticity, and strength to organisms and is the prime construction material in biology. Collagen is also the dominating material in the extracellular matrix and its stiffness controls cell differentiation, growth, and pathology. As a molecule, it forms a rope-like structure with three interwoven amino acid chains, resembling a powerful string, or triple helix. As one of the most abundant proteins, it plays a foundational role in all of biology. Read more about collagen in the references below [1-3]. Creating its unique triple helical shape is a particular sequence pattern of amino acids, which is reflected in the regular, periodic realization of the musical score presented here. The protein sonified in this composition is a short collagen segment with Protein Data Bank ID 1cag, with identical chains, each of which has this sequence: PPGPPGPPGP PGPPAPPGPP GPPGPPGPPG [1] https://en.wikipedia.org/wiki/Collagen [2] A. Gautieri, M.J. Buehler, et al., Nano Letters, 2012, https://pubs.acs.org/doi/10.1021/nl103943u [3] M.J. Buehler, PNAS, 2006,https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1567872/

nothing at of , which is


10. Concerto of a De Novo Protein Designed using Artificial Intelligence

Concerto of a De Novo Protein Designed using Artificial Intelligence

Musical representation of de novo protein designed using artificial intelligence, and folded and equilibrated using molecular dynamics, as reported in [1]. The musical composition renders the 3D structure of this alpha-helix rich protein in an audible space, presenting complex melodic and rhythmic patterns that reflect the hierarchical design of this protein. This is a de novo protein, in other words, a protein that does not exist in nature, but which was designed through our AI algorithm that generated new music, then translated back into a protein amino acid sequence. [1] C.H. Yu, M.J. Buehler, "Sonification based de novo protein design using artificial intelligence, structure prediction and analysis using molecular modeling," APL Bioengineering, DOI: 10.1063/1.5133026, 2020.

nothing at of , which is


11. Honeybee Silk

Honeybee Silk

This is a sonification of a protein extracted from honeybees, and folded using a deep learning method described in [1]. BACKGROUND: The development of rational techniques to discover new mechanically relevant proteins for use in variety of applications ranging from mechanics, agriculture to biotechnology remains an outstanding nanomechanical design problem. The key barrier is to design a sequence to fold into a predictable structure to achieve a certain material function. Focused on alpha-helical proteins (as found in skin, hair, and many other mechanically relevant protein materials), we report a Multi-scale Neighborhood-based Neural Network (MNNN) model to learn how a specific amino acid sequence folds into a protein structure. The algorithm predicts the protein structure without using a template or co-evolutional information at a maximum error of 2.1 Å. We find that the prediction accuracy is higher than other models and the prediction consumes less than six orders of magnitude time than ab initio folding methods. We demonstrate that MNNN can predict the structure of an unknown protein that agrees with experiments, and our model hence shows a great advantage in the rational design of de novo proteins. Reference: Z. Qin, B. Marelli, M.J. Buehler, et al., "Artificial intelligence method to design and fold alpha-helical structural proteins from the primary amino acid sequence," Extreme Mechanics Letters, Vol. 36, p. 100652, 2020 https://doi.org/10.1016/j.eml.2020.100652

nothing at of , which is


12. Spider Variations

Spider Variations

Going deeper inside a spider web. Multiple spider web variations interact across a spectrum of sonic textures. The sounds are based on a sonification method of a spider web as described in the references below [1-2], part of a larger collaboration with Tomás Saraceno, Saraceno Studios, Ally Bisshop, Thomas Muehletahler, and Evan Ziporyn, and the MIT Center for Art, Science and Technology (CAST). Several individual webs constructed by Cyrtophora citricola are played in this piece, creating an overlaying soundscape of different shades. The piece includes a rhythm track for added structure, but all other sounds you hear are generated from the sonification of the spider web. The image [3] shows a 3D spider web, lit up using a green laser during scanning. The spider is visible in the top middle as she constructs the web. References: [1] Su, I.; Qin, Z.; Saraceno, T.; Krell, A.; Mühlethaler, R.; Bisshop, A.; Buehler, M. J. Imaging and Analysis of a Three-Dimensional Spider Web Architecture. J. R. Soc. Interface 2018, 15 (146), 20180193. doi.org/10.1098/rsif.2018.0193. [2] Su, I.; Qin, Z.; Bisshop, A.; Muehlethaler, R.; Ziporyn, E.; Buehler, M. J. Sonification of a 3D Spider Web and Reconstitution into Musical Composition Using Granular Synthesis. in submission [3] Image by I. Su, MIT and M. Buehler, of a tidarren sisyphoides spider building a web, in collaboration with Tomas Saraceno Studio (Berlin)

nothing at of , which is


13. Proteins, Fractures, Piano and Voice

Proteins, Fractures, Piano and Voice

This experimental piece reflects the construction of healed fractures from separated, fragmented pieces. Self-healing is a widely seen property at the nanoscale, and can be utilized as a means to explore structure formation, dissociation and creation. The illustration of this concept, realized in this recording, features a combination of sounds generated from proteins, fractures, and piano. The sounds of fractures are generated based on a sonification method to turn the singularity near a atomically sharp crack tip into audible vibrations. The protein vibrations used here reflect a distinct scale to express a unique impression of nanoscale vibrations scaled up to a comprehensible level. The human voice integrates fractured elements at the beginning, turning into more nuanced and complex vibrational patterns. At 4:45 onwards both rhythm and scale is dominated by the sonification of the protein 6CJC, which transcends back into the original Western scaling. It is taken up again at 5:55 for a short while before being overshadowed by other repeats of organ, piano and others. A prominence of the singularity is seen at 6:55 reflecting the randomization of structure caused by singular forces near the crack. As the piece ends, at 8:45 onwards the underpinnings of vibrational spectra of the 1SVF protein is examined and promoted.

nothing at of , which is


14. Counterpoint Through Fracture: Part III - Spiral Roosters

Counterpoint Through Fracture: Part III - Spiral Roosters

This piece is a reprise of the motifs presented in Part I, now set into a more complex musical arrangement, and with more rhythmic guidance. As the other compositions in this series, this part is created based on the universal stress field of a crack, exploiting the unique features of the singular stress tensor to derive melodic and counterpunctual content through overlapping stress fields. GENERAL FEATURES: FRACTURES ARE UNIVERSAL A fracture is a crack in a material (often seen in glass, but also represented in earthquakes when tectonic plates fracture, or in the unfolding of a protein). Fracturing is the process of destruction of structure, and hence is the division between function and failure in material and has correspondences in philosophy. The tipping point of the crack, the locality of greatest action, and where bond breaking occurs, is the singular crack tip, the sharp edge. All fractures feature a universal distribution of stresses – the forces that act inside the material and which stretch/compress chemical bonds. No matter what material, as you get closer to the singularity – the tip of the crack – the distribution of forces are identical. Here we explore these universal features to generate algorithmically composed music. We try to explore whether the universal features of fractures have the capacity to generate counterpoint – universal features of music, to better understand the analogy across manifestations. Read more about the physics of fracture in [1]. HOW THE MUSIC IS CREATED A set of Euclidean rhythmic motifs (with sparsity: 1/3, 5/12, and 7/12 to reflect different atomic organizations in crystalline forms) are acted upon based on the principal stresses near a crack under tensile loading. We use the singular principal stress field to drive melodic content formation, whereas several independent moving observers assess the local principal stresses. The observes move in circular fashion, towards and away from the singularity in a spiral, oscillatory motion (see track image). Large stresses pull notes into high pitch, lower stresses compress it to lower pitches. These motions provide the basis for the music you hear. The violent nature of forces experienced by the observer circling around the crack tip generate complex but highly organized melodic forms. As the stress field features universal, respective patterning, the various melodic lines generated have relationships rooted in the nanoscale physics surrounding the singularity. Fracture-generated counterpoint serves as a means to explore the universal features of fractures, and provides a model thereof. Owing to the universal significance of fracturing as a process in material and amaterial manifestations, the musical effects generated hereby can be an interesting subject of further analysis and inspiration. [1] M. Buehler, Atomistic Modeling of Materials Failure, 2008 https://www.springer.com/gp/book/9780387764252

nothing at of , which is


15. Fire Dance (extended version)

Fire Dance (extended version)

Experimental throwback to melodic and rhythmic opportunities at the interface of organized, hierarchical structure and melody, inspired by fire. Follows a classical A-B-A structure with a couple of breaks. Focuses on the use of simple analog synthesis to explore melodic ideas across the piece (e.g. using a 303 to realize lead melodies). Materiomusic explores new ways to design music based on natural phenomena, and here we designed a piece to overlay eruptive forces of nature into sound. This is an older piece on the path to understand the significance of fracture singularities in driving the design of musical counterpoint through the action of physical stresses. In this piece, you hear a variety of conventional synthesizer instruments, paired with rhythmic and melodic elements extracted from protein vibrations. They form the backbone to some of the transitions heard at 6:30.

nothing at of , which is


16. Concert of Enzyme Lysozyme

Concert of Enzyme Lysozyme

This piece is constructed entirely from the sounds generated from amino acids, the building blocks of proteins. Encoded by the DNA sequence, amino acids form chains that assembly into hierarchical materials that serve diverse functions in biology (as enzymes, structural materials, signal transmitters, and many others). All sounds used to compose this piece are generated from the sounds of amino acids, computed from their quantum mechanical vibrations, reflecting their elementary molecular features. For further details on the scientific foundation, see [1]. The main melody is derived from the amino acid sequence of a the lysozyme enzyme with Protein Data Bank identifier 194l. Motifs derived from this proteins are used throughout to generate repetitive elements and to conjoin melodic concepts. Learn more about lysozyme here: https://en.wikipedia.org/wiki/Lysozyme All rhythmic elements (bass drum, snare-like, hi-hat, and others) are generated from soundings of amino acids. Both melodic and rhythmic patterns used are derived from the protein cited above. No other synthetic or sampled sounds have been used in the composition. What you hear is a composition in the natural 20-tone amino acid scale. BACKGROUND: Materials and music have been intimately connected throughout centuries of human evolution and civilization. Indeed, materials such as wood, animal skin or metals are the basis for most musical instruments used throughout history. Today, we are able to use advanced computing algorithms to blur the boundary between material and sound and use hierarchical representations of materials in distinct spaces such as sound or language to advance design objectives. The approach used in this work is that the translation of protein materials representations into music not only allows us to create musical instruments, but also enables us to exploit deep neural network models to represent and manipulate protein designs in the audio space. Thereby we take advantage of longer-range structure that is important in music, and which is equivalently important in protein design (in connecting amino acid sequence to secondary structure and folding). This paradigm goes beyond proteins but rather enables us to connect nanostructures and music in a reversible way, providing an approach to design nanomaterials, DNA, proteins, or other molecular architectures from the nanoscale upwards. Reference: [1] C.H. Yu, Z. Qin, F. Martinez, M.J. Buehler, A Self-Consistent Sonification Method to Translate Amino Acid Sequences into Musical Compositions and Application in Protein Design using Artificial Intelligence, ACS Nano, in press, 2019

nothing at of , which is


17. Amyloid protein sonification

Amyloid protein sonification

This piece reflects the sonification of an amyloid protein. Amyloids are a type of folding of a protein into highly stable, sticky fibrils, which are associated with around 50 diseases. Learn more about amyloids: https://en.wikipedia.org/wiki/Amyloid Reference: [1] C.H. Yu, Z. Qin, F. Martinez, M.J. Buehler, A Self-Consistent Sonification Method to Translate Amino Acid Sequences into Musical Compositions and Application in Protein Design using Artificial Intelligence, ACS Nano, in press, 2019

nothing at of , which is