Baby Babble-to-Language Mystery Illuminated

When Deb Roy SM ’95, PhD ’99 became a dad in 2006, he did more than vow to document his son’s life. The Media Lab associate professor started an academic project to record his baby’s waking hours and use this data to illuminate the mystery of how humans naturally acquire language. This effort, the Human Speechome Project (speech + home), was designed to yield some 400,000 hours of audio and video data over three years.

The logistics were impressive. He installed 11 overhead, omni-directional fisheye video cameras and 14 ceiling-mounted microphones that could record all activity in his home—and the cameras rolled. They recorded an average of 12-14 hours a day, generating about 300 gigabytes per day, with some provisions for privacy.

Fast forward three years. The BBC recently reported Roy’s progress, including audio and video samples. You can hear the baby learn to say “water” and see a software visualization of parent-child interactions. Although he is still assessing data, he has some initial results including identifying what he calls "word births,” when a baby first begins to use a word.

Roy, who directs the Media Lab's Cognitive Machines group and Center for Future Banking, is exploring new uses for hardware and software tools—such as a video-analysis algorithm—developed to record, view, and assess the data. A Speechome Recorder, which looks like a floor lamp and contains an overhead microphone and camera, will soon be deployed in pilot studies of children with autism.