Keeping an Eye on the Metaverse

With the recent unveiling of Facebook becoming Meta, advances in AR/VR technology seem to be on the horizon. As Meta takes the fore in the development of AR/VR technology, and other tech giants not far behind, we can expect the way we view content—not only the amount but the way that information is shown to us—will change forever.

For one thing, immersion will be the name of the game as, presumably, full dive VR is the ultimate objective. Near-to-eye (NTE) technology means the screens are less intrusive to the experience, and thus an integral part of current HMD-based AR/VR systems. When it comes to normal, day-to-day, display technology, graphics are judged based on a certain expected viewing distance (within an arms-length for laptops and phones or the length of a room, as with a television), but as a screen gets closer to the eye, the requirements for resolution increase.

Given growing requirements of visual displays, the question becomes how technology can meet those demands and how human-computer interactions need to change to simulate reality effectively. Analysts have projected, according to the technologies patented by Meta, that the company aims to achieve hyper-realistic graphics as a prerequisite to a full dive experience. Yet, the low resolution of displays remains among one of most pivotal challenges in current AR and VR technologies.

Can Foveated Displays Make Immersive VR/AR Possible?

In fact, full resolution throughout the entire field-of-view is not even necessary since human eye itself does not perceive visual information on the periphery with high acuity. This means foveated displays are a viable solution to the resolution problem. A foveated display uses the center of the eye’s retina to determine the area of the display where images should show the highest resolution, while everything outside of that is displayed with lower resolution—mimicking the foveal focus and deterioration of acuity toward the periphery as in the human eye. The advantages are twofold: first, this reduces the computing power needed to process the images by eliminating the necessity for high resolution rendering outside of the foveal area, and second, because it simulates the reduction of acuity toward the periphery as in the human eye, acclimatization should be minimal. Of course, eye-tracking is a keystone technology in implementing a foveated display. 

Eye-tracking: Necessary to Immersion

For the hardware to render the correct zones with accuracy, it must first know where to render. This is where eye-tracking becomes a necessity; by tracking the retina and the depth of focus, a foveated display can render responsively to the viewer’s visual focus. Most eye tracking systems use invisible infra-red light sources to illuminate the retina to determine what the viewer is looking at.  The use of holographic optical elements that only reflects that infra-red wavelength but passes all other wavelengths is an excellent fit (see below).