1.

Introduction

Volumetric displays render three-dimensional (3-D) images onto real-volume space.^1–4 Viewers can observe the 3-D images from any surrounding viewpoint without requiring additional devices. Therefore, volumetric displays could be applied for 3-D visualization in many fields.

We had previously proposed an algorithm that utilizes the 3-D architectures of volumetric displays, for developing an information display system exhibiting multiple two-dimensional (2-D) images in different directions with directional characteristics.^5–8 As shown in Fig. 1(a), viewers can view different images depending on the viewpoints of the volumetric display designed by the algorithm. The potential uses of the system include digital signage, security systems, art, and amusement because it can provide different 2-D information to multiple people simultaneously.

Fig. 1

Three-dimensional structure exhibiting multiple images: (a) glass cube exhibiting three images. (b) LED-based volumetric display exhibiting two multicolor dynamic images: alphabet in the front view and numbers in the side view.

The volumetric display shown in Fig. 1(a) was made with a glass cube in which many small cracks were induced by a laser. The display represents three monochromatic static images. Moreover, in an earlier study,⁵ we had provided a brief introduction to a volumetric display composed of $8\times 8\times 8$ light-emitting diodes (LEDs) as an application of the proposed algorithm and shown an outline of a system exhibiting dynamic color images (alphabet and numbers) in two directions, as shown in Fig. 1(b). In this paper, we describe the hardware design of the volumetric display system in more detail.

However, only eight colors (red, green, blue, cyan, magenta, yellow, white, and black) are available in the aforementioned system, which uses a microcomputer to control the lighting pattern of the 3-D LED array. Therefore, in addition to describing the multicolor system, we aim to achieve full-color representation by controlling the emission color of each volume element (voxel) of the LED array. For full-color representation, we used pulse-width modulation (PWM).^9,10 In PWM, the light intensity gradation is represented by just controlling the on/off ratio of LEDs in a short period. That is, adjusting the lighting times of red, green, and blue LEDs enables full-color representation. Note that high-speed signal processing is necessary for achieving PWM. Therefore, we designed and developed a special-purpose control circuit of the LED array using a field-programmable gate array (FPGA), which operates at a higher frequency and is more suitable for parallel computing than microcomputers.

Moreover, we verify the system’s operation and demonstrate that it exhibits different full-color dynamic images in two orthogonal directions. This system represents a prototype for a directional display based on our previously proposed algorithm,^5–8 which allows multiple viewers to receive 2-D images independently and simultaneously.

2.

Hardware Design

In this section, we describe the hardware design of the proposed systems: a multicolor volumetric display and a full-color volumetric display.

2.1.

Multicolor Volumetric Display System

The LED-based volumetric display consists of two units: a display unit and a control unit. The LED array used as the display unit is composed of eight LED boards, on which $8\times 8$ full-color LEDs are mounted, as shown in Fig. 2(a). These boards were obtained by deconstructing a commercially available product (3-D LED Cube MB8X, LEDGEND Technology Inc.). On each of the boards, 12 serial-in/parallel-out LED drivers (SCT2024¹¹), each of which has 16 parallel outputs, are connected in cascade as shown in Fig. 2(b). For each LED board, 192 elements (Els) ($64\text{voxels}\times 3\text{channels}$) can be controlled independently using only one serial input. Channels 1, 2, and 3 represent red (R), green (G), and blue (B), respectively. Here, the maximum frequency of the LED drivers is 25 MHz.

Fig. 2

Display unit. (a) Schematic diagram of the LED boards. The volumetric display comprises eight LED boards. (b) Simplified block diagram of the LED boards consisting of the serial-in/parallel-out LED drivers.

These LED drivers are controlled by four 1-bit signals: clock (CLK), serial data input (SDI), latch (LA/), and output enable (OE/) in the following steps. First, the data are latched when LA/ is low. LA/ should be low while the data of SDI are input to the drivers. Next, the data of SDI are input serially from the red channel of voxel 1 to the blue channel of voxel 64 through the shift registers, as shown in Figs. 2(a) and 2(b). SDI is sampled at the rising edge of CLK. When LA/ is driven high after the data of SDI are input, the data on the shift register go through. When OE/ is forced to low value, the outputs of the LED drivers are enabled and some of the LEDs turn on according to the input SDI. By doing so, arbitrary lighting patterns can be represented by only four 1-bit signals per one LED board, namely SDI, CLK, LA/, and OE/. We will explain how to control four signals in detail with a specific example below (shown in Fig. 3).

Fig. 3

Specific example of the timing chart of processing in the multicolor volumetric display system.

As a control unit, we used the microcomputer board Arduino Mega 2560 (Arduino, LLC), which has 54 digital I/O pins and can be easily controlled by the Arduino programming language based on C/C++. The microcomputer generates control signals according to the source code written into the flash ROM of the board and sends them to the display unit to render 3-D images. The operation frequency of the microcomputer is 16 MHz.

Figure 3 shows a specific example of the timing chart of processing. First, we describe the four control signals for an LED board. In the timing chart, Els 1, 2, and 3 correspond to R, G, and B channels of voxel 1, respectively, as described in Fig. 2(b). When OE/ is high, all the LEDs on the board emit no light. When OE/ is driven low, some of the LEDs emit light according to the signals of SDI; thus, all the voxels of the board have respective colors (red, green, blue, cyan, magenta, yellow, white, or black). We refer to the duration of light emission as displaying period. SDI is high only when it corresponds to El 1; thus, voxel 1 turns red during the displaying period. Similarly, voxels 2, 3, and 64 turn green, magenta (red + blue), and white (red + green + blue), respectively. It is difficult to control all eight boards simultaneously with one microcomputer because it is not suitable for parallel processing. Therefore, the microcomputer controls the eight boards sequentially.

2.2.

Full-Color Volumetric Display System

In the multicolor system, which is the first prototype of the LED-based volumetric display, we used the microcomputer board as the control unit and succeeded in demonstrating that the display unit operates as intended. However, the display unit could represent only on/off states (1-bit) for each color channel due to the circuit structure of the LED array, as described in Sec. 2.1. To represent full-color 3-D images, a control unit that realizes PWM is required. The performance of the display unit depends on the maximum operation frequency of the LED driver (25 MHz) and is sufficient for PWM. However, the microcomputer is not suitable for PWM because of its limited maximum operating frequency (16 MHz) and processing property (serial processing). Therefore, we designed a control unit with FPGA to achieve full-color 3-D image representation with PWM, as shown in Fig. 4.

Fig. 4

LED-based volumetric display that can represent full-color dynamic 3-D images.

The host PC sends voxel data of the 3-D image to the control unit via a serial communication interface. The voxel data are color information comprising red, green, and blue components, each of which has 8-bit depth. The control unit generates control signals from the voxel data with PWM and sends them to the display unit. The display unit renders arbitrary 3-D images based on the received control signals. We developed software for controlling the host PC using Visual studio 2015 as a Windows Forms application.

The control unit was implemented on an Atlys board (Digilent Inc.)¹² on which an FPGA chip Spartan-6 LX45¹³ operating at 100 MHz is mounted. The control unit controls the display unit with digital I/O pins, which are enabled by an add-on breadboard (VmodBB, Digilent Inc.) attached on the Atlys board.

Figure 5 shows the block diagram of the control unit. Each block is described in detail as follows:

• 1. The serial port controller block receives voxel data from the host PC via a serial port mounted on the Atlys board. The 1-bit serial data are stored in memory and sent to the gamma correction block $8\text{-}\text{bit}\times 8\text{-}\text{bit}$. The total number of data required for displaying a frame is 12,288 bits ($=8\text{bits}\times 3\text{channels}\times 512\text{voxels}$). The baud rate of serial communication is flexible and can be raised to 12 Mbps.¹⁴ In this system, we set the baud rate to 1.8 Mbps. This communication speed is sufficiently high for realizing a real-time display system (e.g., a system with a 60-Hz refresh rate).

• 2. Typically, the output (light intensity) of a general 2-D display is proportional to the $\gamma$’th power of the input, with $\gamma$ adjusted such that the color gradation appears natural to human eyes. Here, $\gamma$ is called the gamma value and typically takes values from 1.8 through 3.0. The gamma correction block in our circuit raises the input to the 2.2th power. The input of this block is an 8-bit signal, which represents the gradation value of a channel (red, green, or blue). The block was designed to have a 9-bit output. The bit length of the output is 1 bit longer than that of the input to prevent information loss. The process is implemented as an 8-bit-in/9-bit-out look-up table. Here, each of the input and output of this block is a flexed-point number.

• 3. The gamma-corrected gradation values are stored in the RAM in the PWM block. Here, eight RAMs are needed to control eight LED boards in parallel. The PWM block creates 1-bit signal (SDI) from the 9-bit gradation values according to the principle of PWM and sends SDI in response to the demand from the display controller block. The PWM block sends eight signals of SDI to the display controller block.

• 4. The display controller block requests the data of SDI from the PWM block at the right time and outputs three signals (CLK, LA/, and OE/) in addition to the received data of SDI. Here, the frequency of CLK is 25 MHz, which is $1/4$ of the operation CLK of the Atlys board, because the LED drivers operate at a maximum frequency of 25 MHz. The pulse width (the duration of OE/) is set to the same amount of time as the time of sending 192 data of SDI ($7.68\mu \mathrm{s}$). To control eight LED boards in parallel, four control signals (CLK, SDI, LA/, and OE/) are required per LED board.

Fig. 5

Block diagram of the control unit implemented on the Atlys board.

Using the specific example shown in Fig. 6(a), the processing flow is described as follows. We consider the case where the RGB channels of gamma-corrected voxel 1 are 511, 255, and 127, respectively. The sequence of processes, which are detailed in the description of the multicolor system, should be repeated 511 times to display the frame of a full-color 3-D image. In the PWM block, the 9-bit counter (CNT) incremented at intervals of pulse width was implemented. By turning on the correspondent LEDs only when the 9-bit voxel data input are higher than CNT, the desired luminance proportional to the voxel data input can be achieved.

Fig. 6

Specific example of the processing flow. (a) Description of the full-color representation using PWM and (b) timing chart of processing.

Figure 6(b) shows the timing chart. In the case shown in Fig. 6(a), when CNT is between 1 and 127, all channels (RGB) of voxel 1 should turn on while OE/ is low. When CNT is between 128 and 255, the R and G channels of voxel 1 should turn on but the B channel should not. When CNT is between 256 and 511, only the R channel of voxel 1 should turn on. By doing so, the LED of voxel 1 emits light of the desired color (orange in this case).

3.

Design Algorithm

This section describes the algorithm⁵ used to determine the lighting pattern for the LED array exhibiting multiple images. Here, we describe the algorithm in the case where the array exhibits two images in orthogonal directions, as shown in Fig. 7. The volumetric display comprises $P\times Q\times R$ volume Els (voxels), which correspond to $8\times 8\times 8$ full-color LEDs in this study. The voxel value $V_{\text{ijk}}$ indicates the brightness of the LED at $(i,j,k)$ and can be determined as follows:

Now, we consider the images exhibited by the volumetric display with the determined voxels. We assume that the pixel values of the exhibited images are given by summations of the voxel values along the projection directions when we look at the display from a distance. Therefore, $a_{ij}^{\prime }$ and $b_{ij}^{\prime }$, the pixel values of the exhibited images A and B, are represented as shown in Eqs. (2) and (3), respectively

Note that the exhibited images are given by multiplying the original image and a background noise corresponding to the interference from the other image. The original image components in Eqs. (2) and (3) tend to be dominant over the background noise; thus, the exhibited images are recognized as the original images.⁵ On the other hand, this recognition occurs only when we look at the volumetric display from an appropriate viewpoint, i.e., the exhibited images have directional characteristics.

Fig. 7

Algorithm of the LED lighting scheme.

By applying the above calculation procedure to each channel, the volumetric display exhibiting two full-color images can be designed. The full-color expandability of the algorithm has been reported in previous work using inkjet-printing technology.¹⁵ This study experimentally demonstrated that the algorithm can be applied to the case where complicated images (i.e., full-color photographs) were used as the originals.

4.

Results

4.1.

Evaluation of Light Emission

To evaluate the full-color volumetric display system, we measured the luminance of a voxel ($Y$) with the input ($X$) from the host PC using a laser power meter (LP1, Sanwa Electric Instrument Co., Ltd.), as shown in Fig. 8(a). A regulated DC power supply provided constant voltage of 4.0 V to the display unit. The blue graph in Fig. 8(b) shows the normalized luminance when the voxel color is red. The result is in agreement with the theoretical value of the normalized $Y=X^{2.2}$, where $X$ is a digital value sent from the host PC to the control unit and $Y$ is a normalized luminance value. The root-mean-squared error between the experimental result and the theoretical value is almost zero ($1\times {10}^{-3}$). We obtained almost identical results for the other voxel colors (green and blue). From these results, we verified the output of the developed system.

Fig. 8

Measurement of the luminance of a voxel: (a) experimental setup and (b) measured luminance of a voxel. $X$ is a digital value send from the host PC to the control unit and $Y$ is a normalized luminance value.

4.2.

Volumetric Display Exhibiting Two Full-Color Dynamic Patterns

Figure 9 shows a volumetric display exhibiting different full-color dynamic images in two orthogonal directions. As shown in Figs. 9(a) and 9(c), one image is a string of alphabet (A to L) observed from the front of the display, and the other is a string of numbers (0 to 9) observed from a side. Different images are observed from different viewpoints, as shown in Figs. 9(b) and 9(d). See also Video 1. When the volumetric display was observed from viewpoints other than the front and side views, no meaningful images could be obtained. In particular, the viewing zones of the exhibited images were narrow. This shows that the developed system has directional characteristics.

Fig. 9

Volumetric display exhibiting two full-color dynamic patterns (see Video 1). (a) Original images of the front view (A to L), (b) projected patterns of the front view, (c) original images of the side view (0 to 9), and (d) projected patterns of the side view (MPEG, 1.7 MB [URL: http://dx.doi.org/10.1117/1.OE.56.7.073108.1]).

To demonstrate the full-color representation of the system, each frame was given a hue value 5 deg greater than that of the previous frame, as shown in Fig. 9, in which $1/3$ of all frames is displayed. Here, we set the saturation and brightness of the images to the maximum values. The hue–saturation–brightness color coordinates are converted to red–green–blue (RGB) values before communication to the system because the developed system is based on the RGB color model. In Fig. 9, the hue of the image increases from left to the right: $H=15\mathrm{deg}$, 30 deg, 45 deg, …, 360 deg. The multicolor volumetric display system could represent only eight colors, whereas the developed system can represent colors that could not be represented by the multicolor system, for example, orange and purple.

5.

Discussion

First, we discuss the image exhibited by the developed volumetric display. In the images shown in Figs. 9(b) and 9(d), the brightness differs according to the locations of the pixels. This difference was caused by the interference from the other image, as described in Sec. 3. We believe that such cross talk could be reduced using the iteration method proposed in a previous study.⁸ Moreover, the developed system, which has $8\times 8\times 8\text{voxels}$, is smaller in scale than the glass prototype shown in Fig. 1(a), which has $64\times 64\times 64\text{voxels}$. Therefore, the developed system could exhibit only simple images, e.g., a character. In future work, we will develop a large system to exhibit complicated images such as photographs. Moreover, we found a problem in the system that some voxels are hidden by the front black circuit board. A transparent circuit board will solve this problem.

Next, we discuss the frame rate of the display. The number of Els per LED board is 192 ($64\text{voxels}\times 3\text{channels}$). Therefore, it takes $7.68\mu \mathrm{s}$ for the control unit to send 192 data of SDI to the display unit with 25-MHz CLK ($192/25\mathrm{MHz}=7.68\mu \mathrm{s}$). As mentioned in Sec. 2.2, the pulse width (the duration of OE/) is set to the same amount of time as the time of sending the data of SDI ($7.68\mu \mathrm{s}$). Thus, it takes 7.85 ms in total to represent a frame because the cycle of sending the data and enabling output is repeated 511 times ($2\times 7.68\mu \mathrm{s}\times 511=7.85\mathrm{ms}$). When the baud rate of serial communication is 1.8 Mbps, the total communication time between the host PC and the control unit for all voxel data of a frame is $\sim 6.83\mathrm{ms}$ ($=8\text{bits}\times 3\text{channels}\times 512\text{voxels}/1.8\mathrm{Mbps}$). Because the communication time per frame is shorter than the computation time, the communication between the host PC and the control unit can be completed while representing the previous frame. Therefore, the communication time does not need to be considered to determine the frame rate of the display. As the result, the frame rate of the developed system is determined only by the computation time and is $\sim 127\mathrm{Hz}$ ($=1/7.85\mathrm{ms}$). Here, we discuss the limitation in the number of the voxels on the basis of the frame rate of general television (30 Hz). If the display operates at 30 Hz, the control unit developed in this study can control up to four times as many voxels as in the current system ($127/30\approx 4$), for example, $16\times 16\text{voxels}$ per four control signals.

Finally, we discuss the designed control circuit. In this study, we succeeded in designing a simple and easy-to-design circuit of a full-color volumetric display as a first prototype. The present circuit could not realize a real-time display system when the number of voxels increases (more than $16\times 16\text{voxels}$ per four control signals). This issue is more prominent in the field of volumetric displays than in the field of conventional 2-D displays, because the number of voxels increases on the cubic order. We will develop a higher-resolution system by increasing the number of control signals and implementing parallel processing. To this end, we will design and develop an LED-array circuit that can be controlled by a more effective operating scheme. For example, the use of full-color LEDs comprising control chips seems to be a good approach to realize an effective operating scheme.

6.

Conclusion

In this study, we developed a multicolor volumetric display system as a first prototype. In addition, we designed and developed the control circuit of an LED array for realizing a full-color dynamic volumetric display. The developed control circuit, which was implemented on FPGA, is able to control the lighting pattern of the LED array in parallel and at a high speed. Thus, we achieved the representation of full-color dynamic images with a simple circuit structure. Moreover, we experimentally evaluated the system by measuring the luminance of a voxel with varying input and succeeded in demonstrating that the volumetric display exhibits two full-color dynamic images. This demonstration shows the future expandability of the algorithm proposed in the previous study.⁵