1.

Introduction

Augmented reality (AR) is widely recognized as the next-generation computing platform. It may replace smart phones and computers as the most important human and machine interface. In AR, information is presented to viewers with virtual objects such as graphics or captions that immerse them in real environments without interfering with their vision. Clearly, the most essential component of AR is the near-eye display (NED), which is worn by the viewer and is used to combine real and virtual imageries together. Although AR NEDs offers a replacement for smartphones and computer monitors and provide a next-level visual experience to viewers, designs for AR NEDs involve trade-offs between several different metrics, including resolution, eye box, form factor, field of view (FOV), eye relief, brightness, full color, battery life, etc. All of the mentioned metrics are actually correlated. Therefore, the greatest challenge in AR NEDs is not in optimizing any individual metric, but instead simultaneously providing a wide FOV, high resolution, wide eye box, and slim form factor device.

A small form factor AR NED is usually composed of three main parts: (1) a transparent waveguide plate, (2) a collimating optics module, and (3) a micro display unit.

The collimating optics module acts as a bridge between the transparent waveguide and the micro displays. The transparent waveguide is capable of in-coupling the virtual image into the waveguide, transmitting light waves with a total internal reflection (TIR), and outcoupling the virtual images into the user’s eyes. Based on the coupling components used, waveguides can be categorized into two main types: reflective and diffractive.

In reflective waveguiding AR NEDs such as Epson’s Moverio, the molded plastic substrate acts as the waveguiding for the virtual image, and one single semi-reflective mirror is placed in front of the human eye to reflect the virtual image to the viewer. To increase the eye-box and keep the AR NEDs as compact as ordinary glasses at the same time, multiple semi-reflective reflectors can be deployed. Several reflective waveguides with a one-dimensional (1D) exit pupil expander (EPE) have been reported.¹ Amitai² proposed a reflective waveguiding two-dimensional (2D) EPE as well.

The diffractive waveguiding AR solution makes use of the diffractive optical element.^3,4 To mitigate the chromatic aberration caused by the diffractive grating, diffractive waveguiding has to combine two or three layers of waveguiding structures targeting red, green, and blue lights.⁵

Each of these technologies requires different micro display characteristics. For example, a single semi-reflective mirror and 1D EPE waveguide require a larger display panel and a large emission cone for large FOV AR NEDs. On the other hand, AR NEDs with a 2D EPE waveguide require a very high brightness and compact display panel.

In this paper, we propose a color-filter front-lit liquid-crystal-on silicon (CFFLCOS)⁶ to meet the requirements of a single semi-reflective mirror and 1D EPE waveguide AR NEDs. In the second section, we demonstrate our new development of color sequential front-lit LCOS (CSFLLCOS). The CSFLLCOS is $<0.47\mathrm{cc}$ with a resolution of $1024\times 1024$. It is capable of delivering over 100,000 nits for 2D EPE waveguide AR NEDs.

2.

Front-lit LCOS

In a front-lit device, a 0.7 mm thick polarized waveguide replaces the conventional polarization beam splitter (PBS) cube. Therefore, the full display system can be as small as $<0.5\mathrm{cc}$. Figure 1 shows the comparison of a conventional 2.5cc illumination system of the projector (right red box) and a front-lit LCOS device (left yellow box). The conventional illumination projector is composed of a PBS, LED, illumination optics, and color filter LCOS (CFLCOS). The color filter front-lit LCOS (CFFLLCOS yellow box) is shown on the left side. The volume of CFFLLCOS is less than 0.25cc, including the front-lit and color filter LCOS. Clearly, a compact and light weight CFFLLCOS is more favorable to AR glasses designs.

Fig. 1

(a) Size comparison between CFFLLCOS (left yellow box) and conventional CFLCOS projector (right red box) and (b) a real image of color filter front-lit LCOS.

3.

Color Filter Front-lit LCOS

The structure of the color filter front-lit LCOS is shown in Fig. 2. It consists of six parts: (1) LED light source, (2) coupling lens, (3) input polarizer [(i.e., wire grid film (WGF)], (4) polarized waveguide, (5) LCOS panel (i.e., color filter LCOS or color sequential LCOS), and (6) analyzer (i.e., absorptive polarizer). The white light, beam $\mathbf{S}$, emitted from the LED is collected by the coupling lens. The Lambertian light beam $\mathbf{S}$ from the LED is shaped to a profile with two intensity peaks by the coupling lens. We call the two intensity peaks beam $\mathbf{A}$ and beam ${\mathbf{A}}^{\prime }$. Both beam $\mathbf{A}$ and beam ${\mathbf{A}}^{\prime }$ are polarized by the WGF, which is located between the polarized waveguide and coupling lens. The reflected polarized ray, which is bounced back by the WGF, is recycled in the coupling lens system with a suitable conversion surface. Beam $\mathbf{A}\text{-}\mathbf{s}$ and beam $\mathbf{A}\text{-}{\mathbf{s}}^{\prime }$ are then injected into the waveguide plate. The $\mathbf{A}\text{-}\mathbf{s}$ beam hits the micro mirror array inside the polarized waveguide without any further optical bouncing within the waveguide. It is then steered toward the color filter LCOS CFLCOS by the micro mirror array. The ${\mathbf{A}}^{\prime }\text{-}\mathbf{s}$ beam propagates across the polarized waveguide by one additional TIR. Finally, beam ${\mathbf{A}}^{\prime }\text{-}\mathbf{s}$ hits the micro mirror array, where it is located further away from the WGF, and is reflected to the CFLCOS similar to beam $\mathbf{A}\text{-}\mathbf{s}$. Because beam ${\mathbf{A}}^{\prime }\text{-}\mathbf{s}$ propagates further away from the LED, it allows front-lit to illuminate a larger display area. Both $\mathbf{A}\text{-}\mathbf{s}$ and ${\mathbf{A}}^{\prime }\text{-}\mathbf{s}$ are converted into p-polarized light ($\mathbf{B}\text{-}\mathbf{p}$ and ${\mathbf{B}}^{\prime }\text{-}\mathbf{p}$) by the liquid crystal layer in the CFLCOS.

Fig. 2

Structure of color filter front-lit LCOS.

The CFLCOS is equipped with a pixelized red (R), green (G), and blue (B) color filter and mixed twisted nematic (MTN) liquid crystal configuration.^7,8 The white light input source is converted into the RGB color image by the pixelized red, green, and blue color filter and the $2.5\mu \mathrm{m}$ thick liquid crystal layer. The reflectance $R(\lambda )$ of the MTN liquid crystal mode is optimized by selecting the proper twist angle $\phi$, retardation $\mathrm{\Gamma }$, and beta angle $\beta$.^9,10 The average reflectance $R_{\mathrm{avg}}$ of the CFLCOS is defined as

Fig. 3

Reflectance of CFLCOS with full white image.

The reflected beams $\mathbf{B}\text{-}\mathbf{p}$ and ${\mathbf{B}}^{\prime }\text{-}\mathbf{p}$ then pass through the polarized waveguide because the micro mirror array is transparent to p-polarized light. The analyzer on top of the polarized waveguide acts as a clean up polarizer, which improves the contrast ratio of the device. Such a system enables the observer (Fig. 2) to see a color image as the usual direct view display [Fig. 1(b)].

Owing to the high efficiency of white LEDs, the brightness of CFFLLCOS can be as high as 30,000 nits under 175 mW LED power. Figure 4(a) shows the brightness versus LED power. Figure 4(b) shows the corresponding emission cone of the CFFLLCOS. The full wave half maximum (FWHM) of the emission cone is $\pm 15\mathrm{deg}$. Furthermore, by adjusting the optical element in the coupling lens, the emission cone of the CFFLLCOS can be adjusted to cope with different AR-NEDs waveguide requirements. For example, we demonstrate an asymmetric emission cone with FWHM (vertical $\pm 30\mathrm{deg}$, horizontal $\pm 12\mathrm{deg}$). It is particularly useful for a large FOV, i.e., 35 deg, 1D EPE waveguiding AR NEDs.

Fig. 4

(a) The brightness to white LED power. (b) The emission cone of the front-lit CFLCOS with FWHM ($\pm 15$). (c) An asymmetric emission cone with FWHM (vertical $\pm 30\mathrm{deg}$, horizontal $\pm 12\mathrm{deg}$).

4.

Color Sequential Front-lit LCOS

As shown in previous section, the color filter front-lit LCOS is compact (i.e., pixel size is $7.5\mu \mathrm{m}$) and bright (30,000 nits) enough to power the AR NEDs using a single semi-reflective mirror or 1D EPE waveguide. However, the 2D EPE waveguide AR NEDs require microdisplays with an even higher brightness (i.e., $>\mathrm{10,0000}\mathrm{nits}$) and smaller pixel size (i.e., $<5\mu \mathrm{m}$). The optical performance of color filter front-lit LCOS has difficulty meeting these requirements.

It is found that the pixel size of the color sequential LCOS (CSLCOS) can be as small as $4.25\mu \mathrm{m}$. Furthermore, the reflectance of the CSLCOS is about $R_{\mathrm{avg}}>70\%$, which is about six times higher than the reflectance of the CFLCOS (Fig. 5). Therefore, once the CSLCOS is equipped with front-lit, it will become an ideal image source for 2D EPE waveguide AR NEDs.

Fig. 5

Reflectance of CSLCOS with $4.25\mu \mathrm{m}$ pixel.

To prevent unfavorable color mixing of the color sequential display, the response time of the color sequential LCOS has to be shorter than the frame time of 2.7 ms (i.e., 1 s/360 fps). In Fig. 6, we demonstrate the gray to gray response time of the thin cell gap of the $1.6\mu \mathrm{m}$ analog color sequential LCOS (CSLCOS). All gray to gray transitions shown in Fig. 6 are defined as graylevel 1 to graylevel 2. It can be found that, having the thin cell gap color sequential LCOS, all of the gray to gray response times are shorter than 2.5 ms.

Fig. 6

Gray-to-gray response time of an analog CSLCOS.

Figure 7 shows the structure of CSFLLCOS. The working principle is similar to CFFLLCOS as discussed in previous section. Nevertheless, the white LED and CFLCOS are replaced by an RGB mini LED light bar and CSLCOS, respectively. The RGB mini LED light bar is composed of three red, four green, and four blue mini LED chips.

Fig. 7

Structure of color sequential front-lit LCOS CSFLLCOS.

A CSFLLCOS is built successfully. The real working sample is shown in Fig. 8(a). A clear full $1024\times 1024$ color sequential display image can be captured from that working CSFLLCOS, as shown in Fig. 8(b). The outermost dimension of the full module is shown in Fig. 8(c), with length $L=13.7\mathrm{mm}$, width $W=9\mathrm{mm}$, and total thickness being less than $H=3.8\mathrm{mm}$. The overall volume of the module is about 0.47cc. Because the width of the device is less than 9 mm, it can be easily installed in the AR glasses arm. The full on and full off contrast is 500:1. Owing to color sequential lighting mechanism and high contrast device, the color gamut is over 140% of sRGB standard.

Fig. 8

(a) The color sequential front-lit LCOS with resolution $1024\times 1024$. (b) A real display image captured from the working sample. (c) The dimensions of the display module are $L=13.7\mathrm{mm}$, $W=9\mathrm{mm}$, and $H=3.8\mathrm{mm}$.

The brightness of the color sequential front-lit LCOS can be adjusted by the RGB LED power. Figure 9(a) shows the brightness to RGB LED power curve. The maximum brightness is about 150,000 nits with an LED power of 500 mW. The white balance can be maintained at 6800K in full brightness adjustment range (15,000 nits to 150,000 nits). The emission cone of the color sequential front-lit LCOS is shown in Fig. 9(b). The FWHM of the emission cone is $>\pm 12\mathrm{deg}$.

Fig. 9

(a) The brightness emission of the color sequential front-lit LCOS under different RGB LED power levels. The white balance stays at 6800K. (b) The emission cone of the color sequential front-lit LCOS.

5.

Conclusions

The main parameters of CSFLLCOS, CFFLLCOS, and conventional LCOS projector (illumination part) are summarized in Table 1. The PBS in the conventional LCOS projector is replaced by the polarized waveguide in the front-lit device, so the module size of front-lit device is much smaller than the conventional LCOS projector. The CFFLLCOS is suitable for AR NEDs using a single semi-reflective mirror or 1D EPE waveguide. Moreover, the color sequential front-lit LCOS, which can deliver very high brightness $>\mathrm{100,000}\mathrm{nits}$, meets the display module requirements of 2D EPE waveguide-based AR-NEDs.

Table 1

Main features of CSFLLCOS, CFFLLCOS, and conventional LCOS projector.

6.

Discussion and Future Works

There are several front-lit design considerations that are important in real applications. Because the micro-mirror array is located between the LCOS image plane and the collimating optics, the image quality is very sensitive to the presence of artifacts in the micro-mirror array. To avoid such issues, the micro-mirror array is positioned far away (1 to 2 mm) from the image plane (liquid crystal layer) of the LCOS. Therefore, the artifacts on the micro mirror array will not form the image by the collimating optics. On the other hand, the steering properties of the micro mirror array must adjust to having good transparency to p-polarized light. It is also crucial to avoid ghosting images. Furthermore, the illumination cone on the LCOS panel is controlled by the coupling lens. The coupling lens has to compensate for the illumination cone angle variation across the LCOS. Currently, the nine points uniformity of the front-lit device is about 70% (Table 1). The polarization recycling will be further optimized in the future, and it is expected that the optical efficiency of future devices can be 1.5 times higher.