Creating 3D with Algorithms, Not Optics

Lumii, a company founded by MIT alumni, is making 3D display surfaces and printing holographic-style images with ordinary inkjet printers using a new technique: they are replacing "traditional optics with algorithms, to create high-quality, glasses-free 3D experiences."

The company has deep MIT roots: it was founded in Boston by Thomas A. Baran SM '07, PhD '12; Daniel Leithinger SM '10, PhD '15; and Matthew Hirsch SM ’09, PhD ’14.

A recent article explains the printing process, which was on display last week at Siggraph 2016, the signature conference on computer graphics and interactive techniques.

This Q&A with Baran provides a glimpse of how Lumii grew out of work at MIT and where the founders want to take this new technology:

How did so many alumni come together to start this company?

Matt's PhD thesis was in Ramesh Raskar's group in the Media Lab and provided a key insight allowing glasses-free 3D displays to be created using conventional display layers and optimization. Daniel's PhD, in Hiroshi Ishii's group, focused on new tangible 3D interaction, design and visualization techniques, and my PhD thesis, in Al Oppenheim's group in RLE, uncovered theoretical links between signal processing algorithms and large-scale mathematical optimization.

Lumii came out of a natural combination of our interests. To make large-format hologram-like prints using unmodified inkjet printers, you need to take advantage of the foundational work on multilayer displays in Matt's thesis. And you also need to be able to crunch the numbers, by designing new classes of large-scale algorithms capable of processing up to a teraray (trillion light rays) per print. As light field prints are a fundamentally new design medium, it necessitates the development of a new design language codifying how individuals interact with the medium, both in terms of content creation and consumption.

Were any of the ideas or technology started at MIT?

Matt's PhD thesis, and collaborations with former MIT postdocs Gordon Wetzstein and Doug Lanman (now respectively at Stanford and Oculus Research), originated much of the foundational work behind what we're commercializing. Now, at Lumii, we're leveraging signal processing advances to take the theory and algorithm work to the next level, extending the achievable image depth (pop out) by a factor of 20x beyond the MIT results.

Can you describe what a “light field” print is?

"Light field" in light field print refers to the bundle of light rays being emitted from the surface of the print, whose intensities vary both in angle and location. This allows the print to steer distinct images in many directions, and when your eye catches two of these images your brain interprets it as being 3D.

A light field print is different from a traditional hologram in that it uses a ray-based description of the display surface as opposed to a wave-based description. Wave-based math would require the use of very small surface features to create a 3D effect. We can use light fields to reproduce virtual depth using features that are many orders of magnitude larger than those in holograms, letting us make hologram-like prints using off-the-shelf printers.

How might industries or individuals use this advance in the future?

What we're developing is a fundamentally new print medium, and we've gotten lots of interest across a variety of industries as well as from makers and artists who want to print these at home. Medical device companies are interested in visualizing 3D medical images, for example MRI data. There are also a lot of applications in the architecture and construction space, related to on-site visualization of building plans. And there are big opportunities in advertising, using our prints to produce highly-engaging 3D messaging for the backlit ad boxes already in place in airports, malls and transit systems.

Want to experience it first hand? Sign up for Lumii’s Alpha version.