Avegant's light-field tech gives hope to a mixed-reality future

The startup is Avegant, a company you might already know. A few years ago, it created the Glyph, a personal entertainment center that looks (and acts) like a pair of headphones. At the same time as it was making Glyph, however, the swirl and excitement around VR and AR was in the air, and it was hard to ignore. “Everything was moving forward to more wearable computing devices,” said Edward Tang, Avegant’s CTO. “So we looked into it.”

The team soon found that there was a fundamental problem with much of the transparent, mixed-reality displays out there — and it’s that all of them have a fixed focal point. You could pin virtual things to a wall and manipulate them remotely with controllers, but you couldn’t get up close. “The real experience I want to have is, I want to be able to just walk up to something and hold it or touch it, and have something feel like it was right in front of me,” says Tang. “At the end of the day, if you want to display something within about a meter, the focus needs to be correct.”

The solution, as Avegant announced last month, was to look into light-field technology. The technology displays multiple focal points at the same time, meaning that objects appear blurry and sharp depending on your focus. In short, it mimics the way you actually see in the real world.

Light-field-display technologies already exist — it’s the same tech that Magic Leap allegedly uses — but Avegant says that those aren’t really feasible in the very near term. “They require crazy computers and a lot of mechanical active optics,” says Tang. “There are some pragmatic problems between building a product like that and actually shipping it in the next few years.”

So Avegant invented something. It’s a brand-new optical component that the company says is a whole new way of generating light fields. The key difference, according to Tang, is that Avegant’s tech would be able to use existing manufacturing techniques and the existing supply chain. “This allows us to scale in volume,” he says.

The company wouldn’t go into detail as to what the magic component actually is, but I did get to try out a demo headset showcasing the tech. I was brought to a dimly lit room in Avegant’s office in Belmont, California, that was outfitted with several tracking sensors along the walls and ceiling. Tang then handed me the headset, a pretty rough engineering prototype with tape along the side and thick cables connecting it to a nearby PC.

My first demo took place in the solar system. Almost immediately, I was struck by how clear everything was. The asteroids floating through space looked pin-sharp — certainly sharper than any HD display I’ve ever seen. Also, unlike that of Microsoft’s Hololens, the field of view on Avegant’s prototype was enormous. It felt like I had a 100-inch TV in front of my face. One of the reasons for this clarity is Avegant’s previous experience with retinal imaging. The image of that solar system was projected directly into my eyes with an array of tiny micromirrors.

As I started to walk around, I found that at certain angles I could actually shift my focus from one planet to another, blurring out the surrounding environment. For example, when I was standing next to the Earth, the moon orbited right in front of me. When I focused directly on the moon, the Earth faded into the background. And when I then shifted my gaze back to the Earth, the moon blurred into the foreground. I was a little shocked and surprised by it, and I remember exclaiming out loud, “This is so weird!” But, of course, it shouldn’t be weird at all, because this is how we normally see. It’s just that I’m not used to looking at virtual objects this way.

I had the same experience in the next demo, which simulated the feeling of being in an ocean, with fish and turtles floating around me. What’s more, a school of fish would swim in and around a chair, as if they were using it as a hiding spot. Ripples of water would cascade around chairs and tables. It all felt so real, as if I were really under the sea. Avegant tells me that this was done with the help of those tracking sensors and the fact that they mapped the physical objects ahead of time.

Perhaps the most surreal experience I had was when I interacted with a virtual human being. She had a bit of an uncanny Valley vibe; she looked real, but not quite real enough. As I walked toward her, she smiled, and I could actually see the freckles on her face and the details of her lashes. As I walked away, her facial expression changed to one of sadness and confusion. Tang says this is an example of how you could have social experiences in AR. The proof-of-concept is certainly there, though I have to admit the experience was a little too strange for me.

Of course, the headset shown here is still in its prototype stage, but Tang tells me the technology is actually pretty mature. There is no reason that it can’t be made available to consumers today. Indeed, the computer that the prototype was attached to was just a standard gaming PC. The headset can even be outfitted and adapted to mobile chipsets, like that of NVidia’s Tegra lineup or the latest Snapdragon processor.

“We’re no strangers to building cutting-edge display technologies and actually bringing them to market in a manufacturable and affordable way,” says Tang. “We’re not a science company, we don’t just do research. We actually want to take these concepts and turn them into manufacturable things.”

Still, Avegant is a small fish in an increasingly big pond. It simply doesn’t have the mindshare of bigger, more-established entrants in AR. Magic Leap has millions more in investment money and a whole lot more hype, and even though Microsoft’s Hololens doesn’t offer as good an experience, it’s at least an actual product that developers can get their hands on. What’s more, Facebook has announced that it’s working on AR glasses of its own, which could very well rival anything Avegant plans to do.

Yet, Avegant’s light-field technology is still incredibly promising. Tang even showed me a mockup of what the final Avegant light-field headset might look like, and it’s a slim and lightweight affair that’s miles more stylish than what Hololens looks like right now. Magic Leap is perhaps the closest to what Avegant is proposing, but hardly anyone has seen it in action and it doesn’t look like it’ll be a reality any time soon.

Yes, Avegant doesn’t have the mindshare of the bigger companies mentioned here, but at least it’s attempting to push AR forward. Even if the company doesn’t eventually make it to the finish line, it could very well have offered a glimpse into what affordable, consumer-ready AR will actually look like.