Turning Emotions into Art with AI

Alexander Reben SM ’10 creates paintings based on his reaction to photos, all without picking up a paint brush. The project, called amalGAN, starts with a machine that can read his body's signals—brainwaves, electrodermal activity, pulse, eye gaze—while showing him images on a screen. An AI algorithm repeatedly adjusts and combines the images, attempting to zero in on the elements he likes the most.

As Reben explains, “It takes those images and renders them into an abstract fractal. There is another AI algorithm which then looks back at the image that is created, tries to figure out what's in it, and then takes what it thinks it sees and makes a title for it.”

The image is then either made into a print or sent overseas to anonymous painters who render it with oil paint on canvas, says Reben, who became a full-time artist in 2013 and also consults on various projects, from engineering product prototypes to creating experiences for brands. “The people doing the paintings are copying what the AI system comes up with to the best of their ability and the best of their understanding of what it is. To me, that chain of human-machine collaboration—even though the last links are humans who don't even know they're collaborating with a machine—is really interesting and it connects the two together really strongly.”

In all, the process has 10 steps, says Reben. And amalGAN is just one of the many projects he has worked on over the years that fuse his love of technology and art.

“I would say my art is mostly about how humans and technologies are evolving together,” says Reben, who became particularly fascinated with AI and robotics during his time at the Media Lab. For his master’s thesis, he worked on a social robot and autonomous filmmaker called Boxie. That project led to BlabDroids, robots designed by Reben that have traveled the world interviewing people and creating documentaries using the heartfelt responses they receive.

Watch the video above to hear more of Reben’s story and see his art on display.