Mimi Yin’s Work with Algorithmic Dance Improvisation at NYU’s Center for Ballet and the Arts

It begins with a dark stage. Performers and algorithm-controlled lights respond to each other, seeming as if they’re dancing together. The sharp squares of light projected onto the floor fade in and out while the dancers move around the stage in time with the music. They speed up, slow down, and move certain body parts to make the lights move around the stage in an unplanned and totally improvised manner.

This is the culmination of Mimi Yin’s experiment in combining technology and dance. During the spring 2019 semester, Yin, a professor in NYU Tisch’s Interactive Telecommunications Program (ITP), joined NYU’s Center for Ballet and the Arts (CBA) as a resident fellow to research the relationship between interactive media and improvised choreography. CBA aims to combine the art and culture of ballet to create new ideas and dances inspired by the past and future. Yin based this project on her existing knowledge of choreographic intervention, which she teaches in her course, and previous experience with interactivity and dance.

Applying Interactivity and Dance to Choreographic Intervention

Yin is well-versed in programming, visual art, and choreography, which inspired her to make visual art and dance interact more completely as visual mediums in this project. Yin said that her “research is a response to the use of interactive media on set design, meaning that [she] felt like too often the collaboration between dance artists and interactive media artists was not enough.” Yin has explored similar ideas in the past with smaller-scale projects, but this is the biggest project she’s undertaken so far. She created her CBA project with collaborators NiNi Dongnier and Tiriree Kananuruk.

For Yin, making the dancers the focus of the piece was the biggest hurdle. In an earlier version, the light was projected onto a back wall with performers dancing in front of it. This proved to be a problem because in our screen-centered society everyone’s eyes automatically went to the light and turned the dancers into tiny shadows. Instead, Yin decided that moving the projected lights from the back wall to the floor prevented a distracting backdrop. With the lights on the floor, Yin created the illusion of the dancers inhabiting an interactive space and “ended up intensifying faces and added to the legibility of the choreography.” Now viewers’ eyes are drawn to the boundaries of the shapes, rather than the shapes themselves, which emphasizes the piece’s speed.

How Technology Influences Dance

A series of lights and depth-cameras were mounted in the ceiling and controlled by an algorithm. Yin explained that the depth-cameras track the dancers as blobs of objects that are standing out from the ground and draw boundaries around them to create shapes. Every time a dancer pauses, speeds up, slows down, or makes a quick jutting motion with an arm or leg, the program changes the light shape.

Typical user-focused tech requires automatic responses to a person’s inputs so they can see the effect their actions caused right away. Yin’s algorithm functions opposite to this, with a purposefully slower response time so that the dancer can interact with it as improv. Yin found that to inspire improvisation as art, the program couldn’t provide instantaneous reactions. It had to be more than interactive and let the human dancers cue responses. In layman’s terms, Yin said, “You can’t have a conversation with someone who’s repeating what you’re saying all the time.”

Like a conversation, the algorithm is coded so that dancers make real-time improvised decisions in response to a new light formation. There are 15 to 20 logical responses,allowing the dancers to improvise to the light formations — through split-second decisions and reasonable predictions. This improvisation aspect keeps the dancers from timing their responses, resulting in a slightly different performance every time.

Intentional but Unplanned

Previously, Yin had conducted a similar project that was focused around dancers responding to sound. Before either project, Yin thought that working with sound would be easier for interactive media performers. Now, after having worked with both sound and light, Yin feels it is easier to interact artistically with light than with sound. The reason being, sound does not take precedence over our other senses the way that light does. An unsuccessful sound project results in an unclear performance and a best-case scenario would just seem like a perfectly choreographed score.

After working with this project, Yin determined that the most important thing when creating a relationship between the dancers and the media is not motion sensing; it is the intention of the dancer. This relationship is created when the dancer beats the algorithm’s timing through intentional but unplanned actions, such as intending to raise one’s arm while the light is fading in. Ultimately, Yin concluded that the actual art-making does not take place in the algorithm but in the mind of the performers and the audience.