JP7684716B2 - Image display device and image display method - Google Patents

Description

The present invention relates to an image display device and an image display method.
This application claims priority based on Japanese Patent Application No. 2020-200263, filed on December 2, 2020, the contents of which are incorporated herein by reference.

In recent years, research and development has been conducted on image display devices that allow a user to see a stereoscopic image without using glasses. In binocular stereoscopic vision, vergence-accommodation conflict is known as one of the causes of eye fatigue. When viewing an object in the real world, the position at which the focus is set coincides with the position at which the lines of sight of both eyes intersect due to binocular convergence. On the other hand, in the technology of naked eye stereoscopic vision, the focus is set to the image on the display screen, whereas the lines of sight of both eyes due to binocular convergence are adjusted to face the position of a stereoscopic image away from the display screen. The vergence-accommodation conflict is a deviation between the focus position and the position at which the lines of sight of both eyes intersect due to binocular convergence.

In order to alleviate the convergence accommodation conflict, a technique is known that stimulates the focus accommodation response by displaying images based on time division (Non-Patent Documents 1 and 2). In the technique described in Non-Patent Documents 1 and 2, light rays are incident on multiple viewpoint positions of one eye. At this time, the focus position is adjusted to match the intersection of the multiple light rays by the focus accommodation response that avoids the object being seen double.

T. Ueno, Y. Takaki, “Super multi-view near-eye display to solve vergence-accommodation conflict”, Optics Express, Optical Society of America, November 7, 2018, Volume 26, Issue 23, p. 30703-30715 Y. Watanabe, H. Kakeya, "A super-Multiview Display with Horizontal and Vertical Parallel by Time Division and Color Multiplexing", SID Digest, John Wiley & Sons, Inc., September 25, 2020, Vol. 51, No. 1, pp. 1017-1020

However, the technology described in Non-Patent Document 1 was only able to display a single color. The technology described in Non-Patent Document 2 was able to display some color, but the stimulation for inducing a focus accommodation response was insufficient. The technologies described in Non-Patent Documents 1 and 2 did not realize natural stereoscopic vision.
Thus, there is a need for more natural stereoscopic vision in naked eye stereoscopic vision, which includes, for example, stimulating a sufficient accommodative response to alleviate convergence accommodation conflict and increasing the sense of depth.

The present invention has been made in consideration of the above-mentioned points, and provides an image display device and an image display method that enable more natural stereoscopic viewing with naked eyes.

The present invention has been made to solve the above-mentioned problems, and one aspect of the present invention is an image display device for displaying a stereoscopic image, comprising: a plurality of display elements arranged in a matrix on a transmissive image display surface; a plurality of illumination arrangement control elements arranged in a matrix on an illumination arrangement surface for variably controlling an arrangement of illumination light, which is irradiated onto the image display surface from a rear side thereof, on the image display surface; a lens for generating a real image on the illumination arrangement surface; and a lens for controlling the display elements based on parallax image data which is image data generated based on parallax, and controlling the plurality of illumination arrangement control elements to output three-color illumination light from any of the plurality of illumination arrangement control elements based on a predetermined time division with a matrix of a predetermined size as a unit in the matrix arrangement. and a control unit for irradiating one of the three colors onto the image display surface, wherein in the time division, the three colors are respectively assigned to three of the lighting arrangement control elements included in the matrix in each frame of the time division as the colors of the illumination light irradiated by the lighting arrangement control elements, and the three colors are respectively assigned once to each of the lighting arrangement control elements included in the matrix for each period of the time division, and the parallax image data is an image display device comprising three types of images based on each of the imaging positions by the lens of the matrix of three lighting arrangement control elements that irradiate the illumination light among the lighting arrangement control elements included in the matrix in each frame of the time division, and each of the colors assigned to the three lighting arrangement control elements.

In one aspect of the present invention, in the image display device, the stereoscopic image is displayed as a high-resolution, full-color, super multi-view stereoscopic image based on parallax in the left-right and up-down directions.

In one aspect of the present invention, in the image display device described above, the lens includes an achromatic lens.

In one aspect of the present invention, in the image display device, the image display device is a display device that is worn on the head of an observer of the image display device.

In addition, according to one aspect of the present invention, in the image display device, the image display device is a stationary three-dimensional display device.

Another aspect of the present invention is an image display device that displays a stereoscopic image, the image display device comprising: a plurality of display elements arranged in a matrix on a transmissive image display surface; a plurality of illumination arrangement control elements arranged in a matrix on an illumination arrangement surface that variably controls an arrangement of illumination light, which is irradiated onto the image display surface from a rear surface thereof, on the image display surface; a lens that generates a real image on the illumination arrangement surface; and a control unit that controls the display elements based on parallax image data that is image data generated based on parallax, and causes any of the plurality of illumination arrangement control elements to irradiate the image display surface with any of three colors of illumination light from any of the plurality of illumination arrangement control elements based on a predetermined time division with a matrix of a predetermined size as a unit in the matrix arrangement. an illumination light arrangement step in which, in the time division, three colors are respectively assigned to three of the illumination arrangement control elements included in the matrix in each frame of the time division as colors of illumination light irradiated by the illumination arrangement control elements, and the three colors are respectively assigned once to each of the illumination arrangement control elements included in the matrix for each period of the time division; and a parallax image generation step in which, as the parallax image data, three types of images are generated based on each of imaging positions by the lens of the matrix of three illumination arrangement control elements that irradiate the illumination light among the illumination arrangement control elements included in the matrix in each frame of the time division, and on each of the colors assigned to the three illumination arrangement control elements.

One aspect of the present invention is an image display device for displaying a stereoscopic image, comprising: a plurality of display elements arranged in a matrix on a transmissive image display surface; a plurality of illumination arrangement control elements arranged in a matrix on an illumination arrangement surface for variably controlling an arrangement of illumination light, which is irradiated from a rear surface of the image display surface onto the image display surface, on the image display surface; a lens for generating a real image on the illumination arrangement surface; and a lens for controlling the display elements based on parallax image data, which is image data generated based on parallax, and controlling the plurality of illumination arrangement control elements to irradiate illumination light of four or more colors onto the image display surface from any of the plurality of illumination arrangement control elements based on a predetermined time division with a matrix of a predetermined size as a unit in the matrix arrangement. and a control unit for irradiating light to the lighting arrangement control elements, wherein in the time division, the four or more colors are respectively assigned to four or more of the lighting arrangement control elements included in the matrix in each frame of the time division as colors of illumination light irradiated by the lighting arrangement control elements, and the four or more colors are respectively assigned once to each of the lighting arrangement control elements included in the matrix for each period of the time division, and the parallax image data is composed of four or more types of images based on each of imaging positions by the lens of the matrix of four or more lighting arrangement control elements that irradiate the illumination light among the lighting arrangement control elements included in the matrix in each frame of the time division and each of the colors assigned to the four or more lighting arrangement control elements.

According to the present invention, more natural stereoscopic vision can be achieved with the naked eye.

1 is a diagram showing an example of a configuration of an image display device according to an embodiment of the present invention. FIG. 4 is a diagram showing an example of a color pattern according to an embodiment of the present invention. 5A and 5B are diagrams illustrating an example of a relationship between a color pattern and an incident position of illumination light on a pupil according to an embodiment of the present invention. FIG. 4 is a diagram showing an example of a color pattern according to an embodiment of the present invention. FIG. 13 is a diagram showing an example of a color pattern according to a comparative example of the embodiment of the present invention. FIG. 13 is a diagram showing an example of a color pattern according to a comparative example of the embodiment of the present invention. FIG. 13 is a diagram showing an example of a color pattern according to a comparative example of the embodiment of the present invention. FIG. 1 is a diagram illustrating an example of a functional configuration of an image display device according to an embodiment of the present invention. 1 is a diagram showing an example of a configuration of an image display device according to a first embodiment of the present invention. FIG. 4 is a diagram showing an example of a measurement result of a focus accommodation response in the first embodiment of the present invention. FIG. 11 is a diagram showing an example of a measurement result of a focus accommodation response according to a comparative example of the first embodiment of the present invention. FIG. 11 is a diagram showing an example of a measurement result of a focus accommodation response according to a comparative example of the first embodiment of the present invention. FIG. 11 is a diagram showing an example of a measurement result of a focus accommodation response according to a comparative example of the first embodiment of the present invention. FIG. 11 is a diagram showing an example of the configuration of an image display device according to a second embodiment of the present invention. FIG. 11 is a diagram showing an example of the configuration of an image display device according to a third embodiment of the present invention.

(Embodiment)
Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. For convenience of explanation, an XYZ orthogonal coordinate system, which is a three-dimensional orthogonal coordinate system, is shown in the drawings. In the XYZ orthogonal coordinate system, the direction of the Y axis is vertically upward. In the following explanation, the direction parallel to the Y axis is also referred to as the up-down direction. The direction of the Y axis is also referred to as the upward direction. The direction opposite to the direction of the Y axis is also referred to as the downward direction. The positive side of the direction of the Y axis is also referred to as the upper side, and the negative side of the direction of the Y axis is also referred to as the lower side. The direction parallel to the Z axis is also referred to as the depth direction. The positive side of the direction of the Z axis is also referred to as the front side, and the negative side of the direction of the Z axis is also referred to as the back side. The direction parallel to the X axis is also referred to as the left-right direction. The positive side of the direction of the X axis is also referred to as the right side, and the negative side of the direction of the X axis is also referred to as the left side.

[Configuration of image display device]
1 is a diagram showing an example of the configuration of an image display device 1 according to this embodiment. The image display device 1 is an image display device that displays a high-resolution, full-color, super multi-view display stereoscopic image based on parallax in the left-right and up-down directions. Super multi-view display is a technology that displays an image by simultaneously directing multiple light beams to multiple points within the pupil.

The image display device 1 includes a first display unit 11, an illumination unit 12, a lens L1, and a lens L2. The first display unit 11, the illumination unit 12, the lens L1, and the lens L2 are provided in the image display device 1 from the back side to the front side in the order of the illumination unit 12, the lens L1, the lens L2, and the first display unit 11. An observer observes the first display unit 11 from the front side. An eye E1 shown in FIG. 1 is either the left eye or the right eye of the observer.

The illumination unit 12 includes a second display surface D2 that variably controls illumination light irradiated from the rear surface of the first display surface D1 through the first display surface D1 so that the illumination light is incident on the pupil P1. A plurality of illumination arrangement control elements are arranged in a matrix on the second display surface D2. The illumination unit 12 includes, as an example, a transmissive LCD.

Here, the image display device 1 includes a light source unit (not shown in FIG. 1) on the rear side of the illumination unit 12. The illumination unit 12 and the light source unit are collectively referred to as an irradiation unit. The light source unit irradiates illumination light to the pupil P1 via the back surface of the second display surface D2. The light source unit is, for example, a backlight using an LED (Light Emitting Diode). The light source unit irradiates illumination light of one of three colors, red, green, and blue, to each of a plurality of illumination arrangement control elements arranged in a matrix on the second display surface D2. Each of the plurality of illumination arrangement control elements irradiates illumination light of the color of the irradiated illumination light to the pupil P1 via the first display surface D1.
The light source unit may be a backlight using other light sources instead of an LED. The illumination unit 12 may be a self-luminous LCD having a function of emitting illumination light, instead of the transmissive LCD and the light source unit.

The illumination unit 12 displays a viewpoint generated image IM2 on the second display surface D2 by controlling a plurality of illumination arrangement control elements. The viewpoint generated image IM2 is composed of pixels arranged in a matrix of a predetermined size in the matrix arrangement of the plurality of illumination arrangement control elements on the second display surface D2. As shown in Fig. 1, as an example in this embodiment, the viewpoint generated image IM2 is composed of pixels arranged in a matrix of 3 rows and 3 columns.
1, an xy Cartesian coordinate system, which is a two-dimensional Cartesian coordinate system, is shown for the viewpoint generated image IM2 in addition to the XYZ Cartesian coordinate system. In the xy Cartesian coordinate system, the directions of the x-axis and the y-axis respectively correspond to the directions of the X-axis and the Y-axis of the XYZ Cartesian coordinate system.

In the viewpoint generated image IM2, each pixel (i.e., illumination arrangement control element) corresponds to a plurality of viewpoints of the eye E1. Therefore, the pixels constituting the viewpoint generated image IM2 are also called viewpoints. Each pixel constituting the viewpoint generated image IM2 is illuminated with illumination light of one of three colors, red, green, and blue. The pattern of the arrangement of the colors of the illumination light illuminated on each pixel constituting the viewpoint generated image IM2 is also called a color pattern. In the color pattern, colors are arranged based on a predetermined time-division pattern. The color pattern will be described in detail later.

The illumination light emitted from each of the multiple illumination arrangement control elements that compose the viewpoint generation image IM2 is incident on multiple positions on the pupil P1, resulting in the generation of multiple viewpoints. The multiple viewpoints are generated with parallax in the up-down and left-right directions. The parallax in the up-down direction and the parallax in the left-right direction induce a focusing response in the eye E1 that attempts to focus in the up-down and left-right directions, respectively.

Up-down parallax has no effect on focusing on an image of lines in the up-down direction because it tries to focus on the up-down direction. Similarly, left-right parallax has no effect on focusing on an image of lines in the left-right direction because it tries to focus on the left-right direction. In the image display device 1, up-down and left-right parallax are generated by the multiple viewpoints included in the viewpoint generation image IM2, so it is possible to focus on the up-down and left-right directions.

The first display unit 11 includes a transmissive first display surface D1 on which a plurality of display elements are arranged in a matrix. The first display unit 11 includes, for example, a transmissive liquid crystal display (LCD). The first display unit 11 displays a parallax image IM1 on the first display surface D1 by controlling the plurality of display elements. The parallax image IM1 is image data generated based on parallax.

The parallax image IM1 is composed of three types of images (referred to as viewpoint images IV1) corresponding to each viewpoint (colors displayed in pixels) near the pupil P1 where the real image of the viewpoint generated image IM2 displayed on the second display surface D2 is generated in each frame. The viewpoint images IV1 are images observed from each viewpoint near the pupil P1 where the real image of the viewpoint generated image IM2 is generated. The viewpoint image IV1 is an image when the first display surface D1 is viewed from a predetermined direction based on the parallax, and is drawn using the colors displayed in the pixels in the viewpoint generated image IM2 to which the viewpoint image IV1 corresponds.

In the example shown in FIG. 1, the blue, green, and red pixels in the parallax image IM1 are arranged at the bottom, center, and top in the vertical direction, respectively. In this case, the viewpoint image IV1-1 corresponding to the blue pixel (viewpoint) is an image drawn using blue when looking down on the first display surface D1 from a predetermined angle. Similarly, the viewpoint image IV1-3 corresponding to the red pixel (viewpoint) is an image drawn using red when looking up on the first display surface D1 from a predetermined angle. The viewpoint image IV1-2 corresponding to the green pixel (viewpoint) is an image drawn using green when looking at the first display surface D1 from the front.

In the color pattern of the viewpoint generated image IM2, the color displayed in each pixel is changed based on a predetermined time-division pattern. In synchronization with this, the viewpoint image IV1 corresponding to each viewpoint (color displayed in each pixel) of the viewpoint generated image IM2 is updated for each frame.
As described above, the viewpoint image IV1 is drawn corresponding to the colors displayed in the pixels of the viewpoint generated image IM2, so the viewpoint image IV1 itself is a monochromatic image. Since multiple light rays of blue, green, and red are projected onto the retina of the eye E1, the viewpoint generated image IM2, which includes multiple viewpoint images IV1 corresponding to each color, is observed as a color image.

As described above, the parallax image IM1 is composed of three types of viewpoint images IV1 (viewpoint image IV1-1, viewpoint image IV1-2, viewpoint image IV1-3) based on the positions in a matrix of a predetermined size in each frame of the time division of three lighting arrangement control elements that emit illumination light, and the colors assigned to the three lighting arrangement control elements.

The lenses L1 and L2 are plano-convex lenses, for example. The focal lengths of the lenses L1 and L2 are respectively f1 and f2. The distance from the second display surface D2 to the lens L1 is equal to the focal length f1. The distance from the lens L2 to the first display surface D1 is equal to the distance d1. The distance from the first display surface D1 to the eye E1 is equal to the distance d2. In this case, the sum of the distances d1 and d2 is equal to the focal length f2. The image displayed on the second display surface D2 is projected onto the pupil of the eye E1 at a magnification equal to the focal length f2 divided by the focal length f1.
In addition, the lenses L1 and L2 may be lenses other than plano-convex lenses as long as they satisfy the above-mentioned conditions for the focal length. For example, when manufacturing a device having a large display screen, it is preferable that the lenses L1 and L2 are Fresnel lenses in order to reduce the thickness of the image display device 1 in the depth direction.

[Color Pattern]
Here, a color pattern C1 according to this embodiment will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of the color pattern C1 according to this embodiment. In the color pattern C1, the three colors red, green, and blue are respectively assigned to three pixels included in a matrix of a predetermined size in each frame of the time division, and the three colors red, green, and blue are each assigned once to each pixel included in the matrix for each period of the time division.

In color pattern C1, the number of divisions in time division is nine, and nine viewpoints are included. Color patterns C1-1 to C1-9 indicate color patterns in each frame of the time division of color pattern C1, respectively. As an example, color pattern C1 changes in the chronological order of color patterns C1-1 to C1-9. In each of color patterns C1-1 to C1-9, three colors, red, green, and blue, are arranged side by side in the x-axis direction.

As described above, the color pattern C1 is realized by a plurality of illumination arrangement control elements arranged on the second display surface D2. In the present embodiment, as an example, in a matrix arrangement of a plurality of illumination arrangement control elements of a predetermined size on the second display surface D2, three illumination arrangement control elements are arranged vertically and three horizontally in the matrix.

When a plurality of illumination arrangement control elements are controlled at a high refresh rate (e.g., 180 Hz), the color patterns C1-1 to C1-9 displayed in each frame remain as afterimages on the retina of the eye E1 after being irradiated as illumination light to the eye E1. Afterimage images A1-1, A1-2, and A1-3 correspond to afterimages on the retina of the eye E1 when illumination light is irradiated to the eye E1 by the color pattern C1. In the color pattern C1, only colors corresponding to three viewpoints are arranged in each time-division frame, but a total of nine viewpoints are generated by the effect of afterimages as described above.

As described above, in color pattern C1, each pixel is assigned one of the three colors red, green, and blue once per time division period. White color is reproduced by the time average of the three colors of illumination light red, green, and blue projected onto the retina.

Fig. 3 is a diagram showing an example of the relationship between the color pattern C1 according to the present embodiment and the incident position of the illumination light on the pupil P1. Fig. 3 shows color patterns C1-1, C1-2, and C1-3 of the color pattern C1 shown in Fig. 2, and the corresponding positions on the pupil P1 where the illumination light is incident on the pupil P1. An xy orthogonal coordinate system is shown for the color pattern C1, and an XYZ orthogonal coordinate system is shown for the pupil P1.

When the color pattern C1-1 is displayed on the second display surface D2, illumination light of three colors, red, green, and blue, is incident on the pupil P1 at a position indicated by the light beam incidence position V1-1. In the color pattern C1-1, the colors are displayed on the upper side of a matrix of a predetermined size, that is, on the pixels in the first row. In this case, at the light beam incidence position V1-1, the illumination light of each color is incident on positions aligned in the left-right direction below the center of the pupil P1. Similarly, in the color pattern C1-2, the colors are displayed on the center of a matrix of a predetermined size, that is, on the pixels in the second row. In this case, at the light beam incidence position V1-2, the illumination light of each color is incident on positions aligned in the left-right direction at the center of the pupil P1. In the color pattern C1-3, the colors are displayed on the lower side of a matrix of a predetermined size, that is, on the pixels in the third row. In this case, at the light beam incidence position V1-3, the illumination light of each color is incident on positions aligned in the left-right direction above the center of the pupil P1.

Here, the illumination light emitted from the illumination unit 12 is focused as a real image on the pupil P1 by the lenses L1 and L2. Therefore, the color arrangement of the illumination light incident on the pupil P1 at the light ray incidence positions V1-1, V1-2, and V1-3 is inverted in the vertical and horizontal directions from the color arrangements shown by the color patterns C1-1, C1-2, and C1-3, respectively.

It is preferable that the arrangement of the plurality of illumination arrangement control elements arranged in a matrix of a predetermined size on the second display surface D2 is such that the illumination light of each color corresponding to each viewpoint that enters the pupil P1 covers the size of the pupil P1. The arrangement of the plurality of illumination arrangement control elements is specified by the size of the matrix of the predetermined size and the intervals between the plurality of illumination arrangement control elements.

The number of divisions in the time division is not limited to 9. The number of divisions may be determined according to the number of pixels (viewpoints) included in the viewpoint generation image IM2. For example, the number of divisions may be determined to match the number of viewpoints. In accordance with reproducing white by mixing three colors, red, green, and blue, by the afterimage effect, the number of divisions in the time division may be selected as a multiple of 3.

Referring to FIG. 4, another example of a color pattern when the number of divisions is 6 is shown. FIG. 4 is a diagram showing an example of a color pattern C2 according to the present embodiment. In the color pattern C2, the number of divisions in the time division is 6, and six viewpoints are included. The color pattern C2 changes in the time series order from the color pattern C2-1 to the color pattern C2-6. The residual image A2-1, the residual image A2-2, and the residual image A2-3 correspond to the residual image on the retina of the eye E1 when the illumination light is irradiated to the eye E1 by the color pattern C2. In the color pattern C2, each pixel is assigned one of the three colors, red, green, and blue, once for each time division period.

As described above, in the time division of this embodiment, the three colors of red, green, and blue are assigned to three of the lighting arrangement control elements included in a matrix of a predetermined size in each frame of the time division as the colors of the illumination light emitted by the lighting arrangement control elements arranged on the second display surface D2, and each of the three colors is assigned once to each of the lighting arrangement control elements included in the matrix of a predetermined size for each time division period.

Now, with reference to FIG. 5 to FIG. 7, a color pattern of a comparative example of this embodiment will be described.
5 is a diagram showing an example of a color pattern C10 according to a comparative example of this embodiment. In the color pattern C10, the number of time divisions is three, and nine viewpoints are included. The color pattern C10 changes in chronological order from color pattern C10-1 to color pattern C10-3. The afterimage image A10 corresponds to an afterimage on the retina of the eye E1 when illumination light is irradiated onto the eye E1 by the color pattern C10.
In color pattern C10, each pixel is assigned one of three colors, red, green, and blue, once per time division period.

6 is a diagram showing an example of a color pattern C20 according to a comparative example of this embodiment. In the color pattern C20, the number of time divisions is six, and nine viewpoints are included. The color pattern C20 changes in chronological order from color pattern C20-1 to color pattern C20-6. The afterimage image A20-1 and afterimage image A20-2 correspond to afterimages on the retina of the eye E1 when illumination light is irradiated onto the eye E1 by the color pattern C20.

In color pattern C20, two of the three colors red, green, and blue are assigned to each pixel once for each time division period. In other words, in color pattern C20, two of the three colors red, green, and blue are mixed in each pixel by time averaging, so that it appears as if either cyan, magenta, or yellow is displayed.

When only one or two of the three colors are assigned to each pixel for each time division period, as in color pattern C10 or color pattern C20, the focusing response is not sufficiently induced compared to color pattern C1 according to the present embodiment. Also, in color pattern C10 or color pattern C20, the illumination light projected onto the retina does not become white even when averaged over time, so that the image resolution when color pattern C10 or color pattern C20 is used is lower than that of color pattern C1 according to the present embodiment.

7 is a diagram showing an example of a color pattern C30 according to a comparative example of this embodiment. In the color pattern C30, the number of divisions in the time division is nine, and nine viewpoints are included. The color pattern C30 changes in the time series order of color pattern C30-1 to color pattern C30-9. In each of the color patterns C30-1 to C30-9, white is assigned to any one pixel of a matrix of a predetermined size. The residual image A30 corresponds to a residual image on the retina of the eye E1 when illumination light is irradiated onto the eye E1 by the color pattern C30.

In the color pattern C30, white is displayed at each pixel, so the illumination light projected onto the retina is white even when averaged over time. In the color pattern C30, flickering is more noticeable than in the color pattern C1 according to the present embodiment.

[Configuration of image display device 1]
Here, the detailed functional configuration of the image display device 1 will be described with reference to Fig. 8. Fig. 8 is a diagram showing an example of the functional configuration of the image display device 1 according to the present embodiment. The image display device 1 includes a first display unit 11, an irradiation unit 13, and a control device 14.
The irradiation unit 13 includes an illumination unit 12 and a light source unit 15. The functions of the first display unit 11, the illumination unit 12, and the light source unit 15 are as described in Fig. 1. The following description will focus on the functional configuration of the control device 14.

The control device 14 controls the entire image display device 1. The control device 14 displays a parallax image IM1 on the first display surface D1, and displays a viewpoint generation image IM2 on the second display surface D2. The control device 14 includes a control unit 140 and a storage unit 145.

The control unit 140 controls the entire control device 14. The control unit 140 includes a reading unit 141, an image generating unit 142, an illumination arrangement surface control unit 143, and an image display surface control unit 144. These functional units included in the control unit 140 are realized, for example, by a central processing unit (CPU) (not shown) executing various programs stored in a storage unit 145 (described later). In addition, some or all of the functional units may be hardware functional units such as an application specific standard product (ASSP) or an application specific integrated circuit (ASIC).

The reading unit 141 reads out various information pre-stored in the storage unit 145. The various information includes parallax image data and a parameter set for time division. The parallax image data is image data of a parallax image IM1. The parallax image data may be image data of a moving image or may be image data of a still image. The parameter set for time division includes information indicating the number of divisions in time division and a color pattern for generating a viewpoint generation image IM2.

The image generating unit 142 generates a parallax image IM1 based on the parallax image data read from the storage unit 145 by the reading unit 141 and information indicating the number of divisions and the color pattern indicated by the time-division parameter set.

The illumination arrangement surface control unit 143 controls the light source unit 223 and the second display unit 221 to display the viewpoint generated image IM2 on the second display surface D2. The illumination arrangement surface control unit 143 controls a plurality of LEDs provided in the light source unit 223 via a plurality of illumination arrangement control elements arranged on the second display surface D2 to display the viewpoint generated image IM2 on the second display surface D2. Here, the illumination arrangement surface control unit 143 uses the number of divisions indicated by the time-division parameter set read out from the storage unit 145 by the reading unit 141 and information indicating the color pattern in controlling the display of the viewpoint generated image IM2 on the second display surface D2.

Here, in time division, the lighting arrangement surface control unit 143 assigns the three colors of red, green, and blue to three of the lighting arrangement control elements included in a matrix of a predetermined size in each frame of the time division as the colors of the lighting light emitted by the lighting arrangement control elements, and assigns each of the three colors once to each of the lighting arrangement control elements included in the matrix of a predetermined size for each time division period.

The image display surface control unit 144 controls the first display unit 21 to display the parallax image IM1 on the first display surface D1. The image display surface control unit 144 controls a plurality of display elements arranged on the first display surface D1 to display the parallax image IM1 on the first display surface D1. Here, the image display surface control unit 144 uses information indicating a color pattern read out from the storage unit 145 by the reading unit 141 in controlling the display of the parallax image IM1 on the first display surface D1. The image display surface control unit 144 generates a plurality of viewpoint images IV1 corresponding to the colors respectively arranged at the plurality of viewpoints according to the positions of the plurality of viewpoints in the matrix indicated by the color pattern.

With the above-mentioned configuration, the control unit 140 controls the display elements arranged on the first display surface D1 based on the parallax image IM1, and also controls a plurality of lighting arrangement control elements arranged on the second display surface D2 to irradiate one of three colors of illumination light onto the first display surface D1 from any of the plurality of lighting arrangement control elements based on a predetermined time division, with a matrix of a predetermined size as a unit in the matrix arrangement.

As described above, the image display device 1 according to this embodiment is an image display device that displays a stereoscopic image, and includes a plurality of display elements, a plurality of illumination arrangement control elements, and the control unit 140.
The plurality of display elements are arranged in a matrix on a transmissive image display surface (first display surface D1 in this embodiment).
The multiple lighting arrangement control elements are arranged in a matrix on a lighting arrangement surface (in this embodiment, the second display surface D2) that variably controls the arrangement of illumination light relative to an image display surface (in this embodiment, the first display surface D1) that is irradiated from the back of the image display surface (in this embodiment, the first display surface D1).
The control unit 140 controls the display element based on parallax image data (in this embodiment, parallax image IM1), which is image data generated based on parallax, and causes a plurality of lighting arrangement control elements to irradiate one of three colors (in this embodiment, red, green, and blue) of illumination light onto an image display surface (in this embodiment, first display surface D1) from any of the plurality of lighting arrangement control elements based on a predetermined time division with a matrix of a predetermined size as a unit in a matrix arrangement.
In time division, three colors (red, green, and blue in this embodiment) are assigned to three of the lighting arrangement control elements included in a matrix of a predetermined size in each frame of the time division as the colors of the lighting light emitted by the lighting arrangement control elements, and each of the three colors (red, green, and blue in this embodiment) is assigned once to each of the lighting arrangement control elements included in a matrix of a predetermined size per time division period.
The parallax image data (in this embodiment, parallax image IM1) is composed of three types of images (in this embodiment, viewpoint image IV1-1, viewpoint image IV1-2, viewpoint image IV1-3) based on the positions in a matrix of a predetermined size of three lighting arrangement control elements that irradiate illumination light among the lighting arrangement control elements included in a matrix of a predetermined size in each frame of time division, and the colors assigned to the three lighting arrangement control elements.

With this configuration, the image display device 1 according to this embodiment can reproduce white color by mixing the three colors of illumination light, red, green, and blue, by time averaging for each frame of time division, and therefore can provide more natural stereoscopic vision with naked eyes. By more natural, we mean more natural than when white color is not reproduced as illumination light.
Natural stereoscopic vision includes alleviating convergence accommodation conflict and increasing the sense of depth due to focus blur. In the image display device 1 according to the present embodiment, it is possible to stimulate a sufficient focus accommodation response and alleviate the convergence accommodation conflict. In the image display device 1 according to the present embodiment, it is possible to improve the image resolution and increase the sense of depth because white light can be reproduced as illumination light by time averaging.

[First embodiment]
9 to 13, the measurement results of the focus adjustment response of an image display device according to a first example of the present embodiment will be described in comparison with a comparative example. In the results shown below, the results obtained using the image display device 1a shown in FIG. 9 are shown in FIG. 10 to 13.

9 is a diagram showing an example of the configuration of the image display device 1a according to the present embodiment 1. The image display device 1a includes an achromatic lens L3 instead of the lens L1 and the lens L2 included in the image display device 1 shown in FIG. 1. Since the illumination light irradiated from the second display surface D2 to the first display surface D1 includes three colors, red, green, and blue, when the illumination light irradiated from the first display surface D1 is refracted by the lens L1 and the lens L2, the emission direction may vary due to chromatic aberration, and the focusing response may become weak. In the image display device 1a, since the achromatic lens L3 is included, the chromatic aberration of the illumination light irradiated from the first display surface D1 is suppressed.

Figures 10, 11, 12, and 13 are diagrams showing examples of measurement results of the focusing response when the above-mentioned color patterns C1, C10, C20, and C30 are used, respectively. Figures 10, 11, 12, and 13 each show the focusing response versus time when four observers (observers H1 to H4) each observe the second display surface D2 provided in the image display device 1a. In Figures 10, 11, 12, and 13, the vertical axis shows the refractive index (diopter value) of the lens (crystalline lens) of the eye.
As described above, the highest focus accommodation response was observed when the color pattern C1 according to this embodiment was used.

[Second embodiment]
A second example of the present embodiment will be described below. In this example, an example in which the image display device is a stationary three-dimensional display device will be described.
Fig. 14 is a diagram showing an example of the configuration of an image display device 2 according to this embodiment. The image display device 2 is a stationary three-dimensional display device. The stationary three-dimensional display device is, for example, a head-up display or a large display device used for public viewing. In the example shown in Fig. 14, the image display device 2 is a large display device. The observer observes the image display device 2 from a position at a predetermined distance. In other words, the image display device 2 and the observer's eye E2 are separated by a predetermined distance or more.

The image display device 2 includes a first display unit 21, an irradiation unit 22, a lens unit 23, and a control device 24. The configuration of the first display unit 21 is similar to the configuration of the first display unit 11 shown in Fig. 1. The first display unit 21 includes a first display surface 211.
The irradiation unit 22 includes a second display unit 221 and a light source unit 223. The second display unit 221 has the same configuration as the illumination unit 12 shown in Fig. 1. The second display unit 221 includes a second display surface 222. The configuration of the light source unit 223 is the same as the configuration of the light source unit described in Fig. 1.
The lens unit 23 has the same configuration as the lenses L1 and L2 shown in Fig. 1. That is, the lens unit 23 includes two Fresnel lenses stacked in the depth direction.

According to the image display device 2, the illumination light projected onto the retina of the observer's eye E2 is a mixture of the three colors of red, green, and blue, which are averaged over time for each frame of time division, to reproduce white light. This improves the image resolution and increases the sense of perspective, allowing for a more natural stereoscopic view than when white light is not reproduced as illumination light.

[Third Example]
A second example of the present embodiment will be described below. In this example, an example in which the image display device is a head mounted display device will be described.
15 is a diagram showing an example of the configuration of an image display device 3 according to this embodiment. The image display device 3 is a display device (head mounted display) that is worn on the head of an observer of the image display device 3. The display device includes a head mounted display (HMD).

The observer wears the image display device 3 on his/her head and observes the image display device 3 from a close distance. In other words, the image display device 3 and the observer's eye E3 are separated by a predetermined distance or less. In addition, the distance between the image display device 3 and the observer's eye E3 and the direction of the image display device 3 with respect to the observer's eye E3 do not change because the image display device 3 is worn on the observer's head.

The image display device 3 includes a first display unit 31, an irradiation unit 32, a lens unit 33, a control device 34, and a lens unit 35. Comparing the image display device 3 according to this embodiment (FIG. 15) with the image display device 2 according to the second embodiment (FIG. 14), the lens unit 35 is different. Here, the functions of the first display unit 31, the irradiation unit 32, the lens unit 33, and the control device 34 are the same as the functions of the first display unit 21, the irradiation unit 22, the lens unit 23, and the control device 24 included in the image display device 2 (FIG. 14), respectively.

As described above, since the image display device 3 is worn on the observer's head, the distance between the second display unit 321 and the eye E3 is equal to or less than a predetermined distance. The lens unit 35 is provided in front of the second display unit 321. The lens unit 35 is used to generate a virtual image of the first display surface 311 arranged near the eye at a position far away from the eye.

According to the image display device 3, a focus accommodation response can be induced in the observer's eye E3, thereby alleviating the fatigue and discomfort of the eye E3 caused by convergence accommodation conflict, and enabling a more natural stereoscopic vision than when a focus accommodation response is not induced.

In the above embodiment, an example of the case where any of the three colors of illumination light, red, green, and blue, is irradiated onto the image display surface (in the above embodiment, the first display surface D1) has been described, but the present invention is not limited to this. Any of four or more colors of illumination light may be irradiated onto the image display surface. For example, any of four colors of illumination light may be irradiated onto the image display surface. The four colors of illumination light are, for example, illumination light of four colors, red, green, blue, and yellow (also called four primary colors). The four primary colors of illumination light, red, green, blue, and yellow, are suitable for reproducing the colors of the natural world in a stereoscopic image. The four colors of illumination light may be illumination light of four colors, red, green, blue, and cyan. In addition, any of five colors of illumination light, for example, red, green, blue, yellow, and cyan, may be irradiated onto the image display surface.

When any one of four or more colors of illumination light is irradiated onto the image display surface, the image display device is an image display device that displays a stereoscopic image, and is equipped with a plurality of display elements, a plurality of illumination arrangement control elements, a lens, and a control unit.
The plurality of display elements are arranged in a matrix on a transmissive image display surface. The plurality of illumination arrangement control elements are arranged in a matrix on an illumination arrangement surface that variably controls an arrangement of illumination light, which is irradiated onto the image display surface from the rear side of the image display surface, on the image display surface. The lens generates a real image on the illumination arrangement surface. The control unit controls the display elements based on parallax image data, which is image data generated based on parallax, and causes any of the plurality of illumination arrangement control elements to irradiate the image display surface with any of four colors of illumination light from any of the plurality of illumination arrangement control elements based on a predetermined time division with a matrix of a predetermined size as a unit in the matrix arrangement.
In this time division, the four or more colors are assigned to four or more of the lighting arrangement control elements included in the matrix in each frame of the time division as the colors of the illumination light emitted by the lighting arrangement control elements, and the four or more colors are assigned once to each of the lighting arrangement control elements included in the matrix for each period of the time division.
The parallax image data consists of four or more types of images based on the imaging positions of the lens of the matrix of four or more lighting arrangement control elements that irradiate the illumination light among the lighting arrangement control elements included in the matrix in each frame of the time division, and based on the colors assigned to the four or more lighting arrangement control elements.

In addition, a part of the image display device 1 in the above-mentioned embodiment, for example, the control unit 140, may be realized by a computer. In that case, a program for realizing this control function may be recorded in a computer-readable recording medium, and the program recorded in the recording medium may be read into a computer system and executed to realize the control function. In addition, the "computer system" here refers to a computer system built into the image display device 1, and includes hardware such as an OS and peripheral devices. In addition, the "computer-readable recording medium" refers to portable media such as a flexible disk, an optical magnetic disk, a ROM, a CD-ROM, and a storage device such as a hard disk built into a computer system. Furthermore, the "computer-readable recording medium" may include a medium that dynamically holds a program for a short time, such as a communication line when a program is transmitted via a network such as the Internet or a communication line such as a telephone line, and a medium that holds a program for a certain period of time, such as a volatile memory inside a computer system that is a server or client in that case. In addition, the above-mentioned program may be a program for realizing a part of the above-mentioned function, and may further be a program that can realize the above-mentioned function in combination with a program already recorded in the computer system.
In addition, a part or the whole of the image display device 1 in the above-mentioned embodiment may be realized as an integrated circuit such as an LSI (Large Scale Integration). Each functional block of the image display device 1 may be individually made into a processor, or a part or the whole may be integrated into a processor. The integrated circuit method is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. In addition, when an integrated circuit technology that replaces LSI appears due to the advancement of semiconductor technology, an integrated circuit based on that technology may be used.

Although one embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to the above, and various design changes, etc. are possible within the scope that does not deviate from the gist of the present invention.

Reference Signs List 1, 1a, 2, 3... image display device, 140... control unit, D1, 211, 311... first display surface, D2, 222, 322... second display surface, IM1... parallax image, IV1... viewpoint image

Claims (6)

An image display device for displaying a stereoscopic image,
A plurality of display elements arranged in a matrix on a transmission type image display surface;
a plurality of illumination arrangement control elements arranged in a matrix on an illumination arrangement surface for variably controlling an arrangement of illumination light, which is irradiated onto the image display surface from a rear surface of the image display surface, with respect to the image display surface;
a lens that generates a real image of the illumination arrangement plane so as to form an image on a pupil;
a control unit that controls the display elements based on parallax image data, which is image data generated based on parallax in the left-right and up-down directions, and controls the illumination arrangement control elements so that one of three colors of illumination light is irradiated onto the image display surface via the illumination arrangement control elements based on a predetermined time division, with a matrix of a predetermined size as a unit in a matrix arrangement;
Equipped with
the number of divisions of the time division is equal to the number of the plurality of the lighting arrangement control elements included in the matrix of the predetermined size;
In the time division, as colors of illumination light emitted by the illumination arrangement control elements, the three colors are respectively assigned to only three of the illumination arrangement control elements included in the matrix that are arranged side by side in the left-right direction or the up-down direction in each frame of the time division, and the three colors are respectively assigned once to each of the illumination arrangement control elements included in the matrix for each cycle of the time division,
the control unit causes the illumination light emitted from each of the plurality of illumination arrangement control elements in each frame of the time division to be incident on a plurality of positions on the pupil, and
the parallax image data is composed of three types of images based on a plurality of image-forming positions on the pupil by the lenses of the matrix of three illumination arrangement control elements that irradiate the illumination light among the illumination arrangement control elements included in the matrix in each frame of the time division, and based on the colors assigned to the three illumination arrangement control elements,
The parallax image data is displayed as a high-resolution, full-color, super multi-view stereoscopic image based on the parallax. The image display device according to claim 1 , wherein the lens includes an achromatic lens. The image display device according to claim 1 or 3, wherein the image display device is a display device that is worn on the head of an observer of the image display device. The image display device according to claim 1 , wherein the image display device is a stationary three-dimensional display device. An image display device for displaying a stereoscopic image,
A plurality of display elements arranged in a matrix on a transmission type image display surface;
a plurality of illumination arrangement control elements arranged in a matrix on an illumination arrangement surface for variably controlling an arrangement of illumination light, which is irradiated onto the image display surface from a rear surface of the image display surface, with respect to the image display surface;
a lens that generates a real image of the illumination arrangement plane so as to form an image on a pupil;
a control unit that controls the display elements based on parallax image data, which is image data generated based on parallax in the left-right and up-down directions, and controls the illumination arrangement control elements so that one of three colors of illumination light is irradiated onto the image display surface via the illumination arrangement control elements based on a predetermined time division, with a matrix of a predetermined size as a unit in a matrix arrangement;
An image display method of the image display device comprising:
the number of divisions of the time division is equal to the number of the plurality of the lighting arrangement control elements included in the matrix of the predetermined size;
an illumination light arrangement step in which, in the time division, the three colors are assigned as colors of illumination light irradiated by the illumination arrangement control elements to only three of the illumination arrangement control elements included in the matrix that are arranged side by side in the left-right direction or the up-down direction in each frame of the time division, and the three colors are assigned once to each of the illumination arrangement control elements included in the matrix for each cycle of the time division;
an illumination step of irradiating the image display surface with illumination light irradiated from each of the plurality of illumination arrangement control elements in each frame of the time division so as to be incident on a plurality of positions on the pupil;
a parallax image generating step of generating, as the parallax image data, three types of images based on a plurality of imaging positions on the pupil by the lenses of the matrix of three illumination arrangement control elements that irradiate the illumination light among the illumination arrangement control elements included in the matrix in each frame of the time division, and based on the colors assigned to the three illumination arrangement control elements;
a display step of displaying the parallax image data as a high-resolution, full-color, super multi-view stereoscopic image based on the parallax;
An image display method comprising: An image display device for displaying a stereoscopic image,
A plurality of display elements arranged in a matrix on a transmission type image display surface;
a plurality of illumination arrangement control elements arranged in a matrix on an illumination arrangement surface for variably controlling an arrangement of illumination light, which is irradiated onto the image display surface from a rear surface of the image display surface, with respect to the image display surface;
a lens that generates a real image of the illumination arrangement plane so as to form an image on a pupil;
a control unit that controls the display elements based on parallax image data, which is image data generated based on parallax in the left-right and up-down directions, and controls the plurality of illumination arrangement control elements to irradiate the image display surface with any of illumination lights of a number of colors, which is four or more, via the plurality of illumination arrangement control elements based on a predetermined time division, with a matrix of a predetermined size as a unit in a matrix arrangement;
Equipped with
the number of divisions of the time division is equal to the number of the plurality of the lighting arrangement control elements included in the matrix of the predetermined size;
In the time division, as colors of illumination light to be irradiated by the illumination arrangement control elements, the colors are respectively assigned to only the lighting arrangement control elements of the number of colors that are arranged side by side in the left-right direction or the up-down direction among the lighting arrangement control elements included in the matrix in each frame of the time division, and the colors are respectively assigned once to each of the lighting arrangement control elements included in the matrix for each cycle of the time division,
the control unit causes the illumination light emitted from each of the plurality of illumination arrangement control elements in each frame of the time division to be incident on a plurality of positions on the pupil, and
the parallax image data is composed of images of the number of colors based on each of a plurality of image formation positions on the pupil by the lenses of the matrix of lighting arrangement control elements of the number of colors that irradiate the illumination light among the lighting arrangement control elements included in the matrix in each frame of the time division, and each of colors assigned to the lighting arrangement control elements of the number of colors;
The parallax image data is displayed as a high-resolution, full-color, super multi-view stereoscopic image based on the parallax.

Images