JP7556582B2 - Systems and methods for improving binocular vision - Patents.com - Google Patents

Description

The present disclosure relates generally to methods and systems for improving binocular vision, and more particularly to methods and systems for improving vision in a viewer's weaker eye by displaying a virtual object moving from a first position to a second position based on information from the viewer's eyes.

Amblyopia and strabismus are two of the most common eye disorders. Amblyopia is associated with one eye having significantly weaker vision in relation to the other eye. Amblyopia often results from abnormal visual development when the patient is young. Amblyopia is the most common cause of reduced monocular vision among children and young adults. Strabismus is associated with a misalignment between the patient's two eyes. Because the patient has strabismus, one of the eyes may point in a different direction relative to the other eye. Most strabismus is caused by abnormal neuromuscular control of eye movements.
Numerous methods for diagnosis and treatment have been proposed for amblyopia and strabismus. For amblyopia, the weaker eye is usually corrected with glasses or contact lenses, or patching therapy, and in some other cases, stimulation using frequency- or color-contrast objects may improve the vision of the amblyopic eye. A combination of several types of stimulation has been proven to be more effective. For strabismus, eye movements are often used to improve eye muscle correlation to correct the misalignment between the two eyes.
In recent years, there have been some advances in the use of medical devices to treat amblyopia and strabismus. However, these conventional devices do not provide a means to train binocular gaze between different depths. Innovative systems and methods for improving binocular vision are needed.

International Publication No. PCT/US20/59317

The present disclosure relates to systems and methods for improving binocular vision in viewers with abnormal or poor vision in one eye compared to the other eye by generating visual stimuli, such as virtual images, that stimulate the optic nerve of the human eye to enhance eye movement, which may treat eye conditions such as strabismus, amblyopia, convergence disorders, and other eye movement disorders. The normal eye refers to the first eye, which may be the right or left eye of the viewer. The poorer or abnormal eye (collectively the "weaker eye") refers to the second eye (the remaining eye other than the normal eye), which may be the right or left eye of the viewer. The present disclosure describes systems and methods that generate virtual images moving between two different depths based on information of the viewer's eyes to stimulate and then strengthen/treat the viewer's poorer eye, ultimately improving or even restoring the viewer's binocular vision.

A system for improving binocular vision includes an eye tracking module and a virtual image module. The eye tracking module is configured to track both eyes of a viewer to provide eye-related information, such as pupil location, pupil size, gaze angle (field of view), and convergence angle of each eye of the viewer. Such eye information may be used to determine the location of the viewer's pupils, as well as the viewer's gaze location and gaze depth.

The virtual image module may be configured to display a first virtual object based on information of the viewer's eyes, such as the location of both of the viewer's pupils, by projecting a number of normal light signals to a first eye of the viewer to form a normal image and a number of corresponding adjustment light signals to a second eye of the viewer to form an adjustment image. The first virtual object is displayed at a first target location and a first target depth. The first eye of the viewer perceives the normal image of the first virtual object and the second eye of the viewer perceives the adjustment image of the first virtual object. The first target depth is related to a first angle between the first normal light signal projected into the viewer's eye and the corresponding first adjustment light signal.

Moreover, in order for the first eye of the viewer to perceive a normal image and the second eye of the viewer to perceive an accommodation image simultaneously, the virtual image module may need to adjust the directions and locations of the multiple normal light signals and the multiple corresponding accommodation light signals based on the locations of the pupils of the viewer's eyes provided by the eye tracking module to project the multiple normal light signals and the multiple corresponding accommodation light signals into the first eye and the second eye of the viewer, respectively.

The virtual image module displays a first virtual object moving from a first target location and a first target depth to a second target location and a second target depth. The first target depth is different from the second target depth. The second target depth is related to a second angle between the second normal light signal and the corresponding second modulation light signal. As the first virtual object moves, the viewer's eyes follow the movement. As a result, the movement of the first virtual object between the two visual depth planes enhances the movement of the weaker eye and provides more stimulation to the weaker eye. As a result, the vision of the weaker eye is improved and then binocular vision may eventually be re-established.

When the virtual image module displays a first virtual object moving from a first target location and a first target depth to a second target location and a second target depth, the adjustment light signal projected to the second eye of the viewer may change the light direction more than the normal light signal projected to the first eye of the viewer. As a result, the eye with poor vision must move (exercise) more than the normal eye to follow the movement of the first virtual object. Instead, when the virtual image module displays an object moving from a first target location and a first target depth to a second target location and a second target depth, the normal light signal projected to the first eye of the viewer does not change the light direction. In this situation, the first eye of the viewer does not need to move at all, while the second eye of the viewer must move more to follow the movement of the first virtual object.

To improve vision in the viewer's weaker eye, the virtual image module may distinguish between the contrast and spatial frequency of an accommodation image for the viewer's second eye and the contrast and spatial frequency of a normal image for the viewer's first eye. Specifically, the virtual image module is configured to generate an accommodation image having a higher contrast or lower spatial frequency than the corresponding normal image.

The virtual image module may display the first virtual object, including selecting an appropriate contrast and spatial frequency, and a direction and speed of motion of the first virtual object, based on the visual evoked potential (VEP) of either eye or both eyes of the viewer. The system for improving binocular vision may further include a VEP measurement module that measures the VEP of the viewer's eyes.

When the viewer is able to perform fixation, the virtual image module may display a first virtual object at the viewer's gaze location and gaze depth provided by the eye tracking module. When the viewer moves his/her gaze from the first gaze location and first gaze depth to the second gaze location and second gaze depth, the virtual image module moves the first virtual object according to the viewer's gaze. The virtual image module may display a second virtual object at a predetermined location and a predetermined depth. When the viewer moves his/her gaze to move the first virtual object within a predetermined spatial range of the second virtual object for a predetermined period of time, the second virtual object is modified to interact with or respond to the first virtual object.

Additional features and advantages of the present disclosure will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of the disclosure. The objectives and other advantageous advantages of the present disclosure will be realized and attained by the structures and methods particularly pointed out in the written description and claims thereof as well as the appended drawings. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are intended to provide further explanation of the invention as claimed.

FIG. 1 is a structural diagram illustrating an embodiment of a system with various modules according to the present invention. 1 is a schematic diagram illustrating an embodiment of a system for improving binocular vision as a head-wearable device according to the present invention; 2 is a schematic diagram illustrating an embodiment of a virtual image module in accordance with the present invention; 1A-1C are schematic diagrams illustrating an embodiment of moving a virtual object between two different depths in accordance with the present invention; 1 is a schematic diagram illustrating an embodiment of moving a virtual object between multiple different depths in accordance with the present invention; 1 is a schematic diagram illustrating an embodiment of moving a virtual object between multiple different depths in accordance with the present invention; 1 is a schematic diagram illustrating an embodiment of moving a virtual object between multiple different depths in accordance with the present invention; 1 is a schematic diagram illustrating an embodiment of low vision eye movement following the movement of a virtual object according to the present invention; 1 is a schematic diagram illustrating an embodiment of low vision eye movement following the movement of a virtual object according to the present invention; FIG. 1 is a schematic diagram showing an example of an abnormal eye, in accordance with the present invention. FIG. 1 is a schematic diagram showing an example of an abnormal eye, in accordance with the present invention. FIG. 1 is a schematic diagram showing an example of an abnormal eye, in accordance with the present invention. FIG. 1 is a schematic diagram showing an example of an abnormal eye, in accordance with the present invention. 1 is a schematic diagram illustrating an embodiment of adjusting light direction and angle based on viewer eye information in accordance with the present invention. 3 is a photograph showing an example of a contrast sensitivity function according to the present invention. 1 is a schematic diagram illustrating an embodiment of adjusting the spatial frequency of a virtual object due to different display depths, in accordance with the present invention; 1 is a schematic diagram illustrating an embodiment of moving a first virtual object to overlay a second virtual object based on a viewer's eye gaze in accordance with the present invention; 1 is a schematic diagram illustrating an embodiment of moving a first virtual object to overlay a second virtual object based on a viewer's eye gaze in accordance with the present invention; 2 is a schematic diagram illustrating the relationship between virtual binocular pixels and corresponding pairs of normal and accommodation pixels in accordance with the present invention; 2 is a schematic diagram illustrating the optical path from an optical signal generator to a combiner to a viewer's retina in accordance with the present invention. 2 is a schematic diagram illustrating a virtual binocular pixel formed by a normal light signal and an adjusted light signal in accordance with the present invention; 4 is a table illustrating an embodiment of a look-up table in accordance with the present invention.

The terminology used in the description presented below is intended to be interpreted in its broadest reasonable manner, even when used in connection with a detailed description of certain specific embodiments of the technology. Certain terms may even be highlighted below, however, any terminology intended to be interpreted in any restrictive manner is specifically defined as such in this "Description of the Preferred Embodiments" section.

The present disclosure relates to systems and methods for improving binocular vision in viewers with one eye that is abnormal or has poorer vision than the other. Some people are born with abnormal or poorer vision in one eye than the other, such as conditions including strabismus (including exotropia, hypotropia, hypertropia, and esotropia), amblyopia (lazy eye), convergence disorders, and other eye movement disorders, due to illness or accident.
The normal eye refers to the first eye 50, which may be the right or left eye of the viewer. The weaker or abnormal eye (collectively the "weak eye") refers to the second eye 60 (the remaining eye other than the normal eye), which may be the right or left eye of the viewer. This disclosure describes systems and methods that, based on the viewer's eye information, generate a virtual image moving between two different depths to stimulate and then strengthen/treat the viewer's weaker eye, ultimately improving or even restoring the viewer's binocular vision.

As shown in FIG. 1, the system for improving binocular vision includes an eye tracking module 110 and a virtual image module 120. The eye tracking module 110 is configured to track a viewer's eyes to provide eye-related information, such as eye movement, pupil location, pupil size, gaze angle (viewing angle), and convergence angle of each viewer's eye. The eye tracking module may include a first camera 112 for tracking a first eye and a first camera 114 for tracking a second eye 2. The virtual image module 120 is configured to display a first virtual object based on the viewer's eye information, such as the position of both viewer's pupils, by projecting a number of normal light signals to the viewer's first eye to form a normal image, and projecting a corresponding number of accommodation light signals to the viewer's second eye to form an accommodation image.
The virtual image module 120 includes a normal light signal generator 10, a normal combiner 20, an adjusted light signal generator 30, and an adjusted combiner 40. The normal light signal generator 10 generates a number of normal light signals that are redirected by the normal combiner 20 to project into a first eye of a viewer to form a normal image. The adjusted light signal generator 30 generates a number of adjusted light signals that are redirected by the adjusted combiner 40 to project into a second eye of a viewer to form an adjusted image. The virtual image module 120 may further include a control unit 125 for processing and storing data.

System 100 may further include a VEP measurement module 130, a user interface 140, a real object measurement module 150, and a feedback module 160. VEP measurement module 130 measures the VEP of a viewer's eye, which is an electrical signal generated in the visual cortex in response to a visual stimulus. Virtual image module 120 may display a virtual object in a manner based on the VEP of either or both eyes of the viewer.
The user interface 140 allows the viewer or trainee to control various functions of the system 100. The user interface 140 may be operated by voice, hand gestures, finger/foot movements in the form of pedals, keyboard, mouse, knobs, switches, stylus, buttons, sticks, touch screen, etc. The real object measurement module 150 measures the location and depth of the real object interacting with the first virtual object. The real object measurement module 150 may also capture images and videos of the environment. The feedback module 160 provides feedback such as sound and vibration to the viewer when certain conditions are met. The external server 170 is not part of the system 100 and can provide additional computing power for more complex calculations. Each of these modules and the external server may communicate with each other in a wired or wireless manner. Wireless approaches may include Wi-Fi, Bluetooth, near field communication (NFC), Internet, telecommunications, radio frequency (RF), and the like.

As shown in Figure 2, the system 100 further includes a support structure that is wearable on a viewer's head. The normal light signal generator 10, the adjusted light signal generator 30, the normal combiner 20, and the adjusted combiner 40 are supported by the support structure. In one embodiment, the system is a head-wearable device such as virtual reality (VR) goggles and a pair of augmented reality (AR)/mixed reality (MR) glasses.
In this situation, the support structure may be the frame of a pair of glasses, with or without lenses. The lenses may be prescription lenses used to correct myopia, hyperopia, etc. Additionally, an eye tracking module including a first camera 112 and a second camera 114 is carried by the support structure to track both eyes of the viewer. A real object measurement module 150 may also be carried by the support structure to measure the location and depth of a real object.

The eye tracking module 110 is configured to track the location of at least both of the viewer's pupils. In addition, the eye tracking module may be configured to provide more information about the viewer's eyes, including, but not limited to, eye movements, pupil size, gaze angle (field of view), and convergence angle of each eye of the viewer. Such eye information may be used to determine the viewer's gaze location and gaze depth as well as the direction and location to project light signals related to the virtual object. Again, the eye tracking module may include a first camera 112 for tracking the first eye 50 and a second camera 114 for tracking the second eye 60.

In addition to a conventional eye tracking module, the first camera 112 and the second camera 114 may be constructed with micro-electromechanical system (MEMS) technology. The first camera 112 and the second camera 114 may use infrared emitters and sensors to detect and derive various eye information. The eye tracking module 110 may further include an integrated inertial measurement unit (IMU), as well as electronics that measure and report the body's specific forces, angular velocity, and possibly body orientation using a combination of accelerometers, gyroscopes, and possibly magnetometers.

As shown in FIG. 3 , the virtual image module 120 is configured to display a first virtual object 70, such as a tennis ball, based on information of the viewer's eyes, such as the positions of both pupils 52, 62, by projecting a number of normal light signals to the viewer's first eye 50 to form a normal image 122, and a corresponding number of adjusted light signals to the viewer's second eye 60 to form an adjusted image 124.
A first virtual object 70 is displayed at a first target location and a first target depth (collectively "first target position" or "T1"). The virtual image module 120 includes a normal light signal generator 10 that generates a number of normal light signals, such as 12 for NLS_1, 14 for NLS_1, and 16 for NLS_3, a normal combiner 20 that redirects the number of normal light signals toward a normal retina 54 of the viewer, an adjusted light signal generator 30 that generates a number of adjusted light signals, such as 32 for ALS_1, 34 for ALS_2, and 36 for ALS_3, and an adjusted combiner 40 that redirects the number of adjusted light signals toward an adjusted retina 64 of the viewer. The viewer has a normal eye 50 that includes a normal pupil 52 and a normal retina 54, and a poorly visual eye 60 that includes an adjusted pupil 62 and an adjusted retina 64. The diameter of a human pupil may typically range from 2 mm to 8 mm, depending in part on the environmental light. Normal pupil size in adults ranges from 2 to 4 mm in diameter in bright light and 4 to 8 mm in darkness.
The multiple normal light signals are redirected by the normal coupler 20, pass through the normal pupil 52, and are finally received by the normal retina 54. The normal light signal NLS_1 is the light signal furthest to the right that the normal eye of the viewer can see on a particular horizontal plane. The normal light signal NLS_2 is the light signal furthest to the left that the normal eye of the viewer can see on the same horizontal plane. Upon receiving the redirected normal light signals, the viewer perceives multiple normal pixels (forming a normal image) for the first virtual object 70 at a first target position T1 within an area A bounded by the extent of the redirected normal light signals NLS_1 and NLS_2. The area A is called the field of view (FOV) for the normal eye 50. Similarly, the multiple accommodation light signals are redirected by the accommodation coupler 40, pass through the center of the accommodation pupil 62, and are finally received by the accommodation retina 64. The adjusted light signal ALS_1 is the light signal furthest to the right that the viewer's weaker eye can see on a particular horizontal plane. The adjusted light signal ALS_2 is the light signal furthest to the left that the viewer's weaker eye can see on the same horizontal plane. Upon receiving the redirected adjusted light signals, the viewer perceives a number of accommodation pixels (forming an accommodation image) for the virtual object 70 within an area B bounded by the extent of the redirected adjusted light signals ALS_1 and ALS_2.
Region B is referred to as the field of view (FOV) for the weak eye 60. When multiple normal and accommodation pixels are both displayed in region C overlapping with regions A and B, at least one normal light signal representing one normal pixel and a corresponding accommodation light signal representing one accommodation pixel are fused to display a virtual binocular pixel at a specific depth in region C. The first target depth D1 is related to the angle Θ1 of the redirected normal light signal 16' and the redirected accommodation light signal 36' projected into the viewer's retina. Such angle is also referred to as the convergence angle.

As described above, the viewer's first eye 50 perceives the normal image 122 of the first virtual object 70, and the viewer's second eye 60 perceives the accommodation image 124 of the first virtual object 70. For a viewer with appropriate image fusion capabilities, the viewer's brain fuses the normal image 122 and the accommodation image 124 into one binocular virtual image, so that the viewer perceives a single first virtual object at the first target location and the first target depth. However, a viewer may not have appropriate image fusion capabilities if the eye's vision is poor. In this situation, the viewer's first eye 50 and second eye 60 may perceive the normal image 122 at the location and depth of the first normal image and the accommodation image 124 at the location and depth of the first accommodation image, respectively (diplopia). The location and depth of the first normal image may be close to, but different from, the location and depth of the first accommodation image. Additionally, the location and depth of both the first normal image and the first adjustment image may be close to a first target location and a first target depth, where the first target depth D1 is again related to a first angle Θ1 between the first normal light signal 16' and the corresponding first adjustment light signal 36' projected into the viewer's eye.

Furthermore, for a first eye 50 of a viewer perceiving a normal image 122 and a second eye 60 of a viewer perceiving an accommodation image 124, the virtual image module 120 may need to adjust the direction and location of a number of normal light signals and a corresponding number of accommodation light signals projected into the first eye 50 and second eye 60 of the viewer, respectively, based on the location of the viewer's pupil provided by the eye tracking module 110.

The virtual image module 120 displays a first virtual object 70 moving from a first target position T1 to a second target location and a second target depth (collectively "second target location" or "T2"). The first target depth D1 is different from the second target depth D2. The second target depth D2 is related to a second angle Θ2 between the second normal light signal 16' and the corresponding second modulation light signal 38'. Information obtained from the eye tracking module, including the location and tracking ability of both pupils of the viewer, particularly the viewer's pupil of the weaker vision eye, are factors to be considered for selecting the second target location and the second depth. The tracking ability of the viewer's eyes can be evaluated by whether and how quickly the pupils follow the movement of the first virtual object. The better the tracking ability of the viewer's eyes, the farther the second target location may be from the first target location.
When the virtual image module 120 displays the first virtual object 70 at the first target position T1 and then at the second target position T2 within the duration of vision, for example, 1/8 seconds, the viewer's eyes perceive the first virtual object 70 moving from the first target position T1 to the second target position T2 and follow its movement. As a result, the movement of the first virtual object 70 between the two visual depth planes enhances the movements of the weaker eyes, including smooth pursuit eye movements and vergence eye movements, and more stimulation is provided to the weaker eyes. As a result, the vision of the weaker eyes is improved, and binocular vision, including image fusion function, may be eventually re-established.

In one embodiment shown in Fig. 4, when the virtual image module 120 displays the first virtual object 70 moving from a first target position T1 to a second target position T2, the adjustment light signal projected to the second eye 60 of the viewer changes the light direction more than the normal light signal projected to the first eye 50 of the viewer. In other words, Θ4 is greater than Θ3. As a result, the weaker eye must move (move) more than the normal eye to follow the movement of the first virtual object 70. In strabismus and amblyopia conditions, the faster the weaker eye moves than the normal eye, the smaller or even disappear the visual difference between the two eyes.
In another embodiment shown in Figures 5A-5C, the normal light signal projected to the viewer's first eye 50 does not change light direction when the virtual image module 120 displays a virtual object 70 moving from a first target position T1 to a second target position T2. In this situation, the viewer's first eye 50 does not need to move at all, while the viewer's second eye has to move more to follow the movement of the first virtual object. Based on the reasons described above, the strabismus condition may be treated more efficiently since the weaker eye is forced to make all the movements.

After the first virtual object 70 moves from the first target position T1 to the second target position T2, the virtual image module 120 may further display the first virtual object 70 moving from the second target position to a third target location and a third target depth (collectively "third target position" or "T3"). Again, for the movement from the second target position T2 to the third target position T3, according to two alternative embodiments, (1) the modulated light signal projected to the second eye 60 of the viewer may change the light direction more than the normal light signal projected to the first eye 50 of the viewer, and (2) the normal light signal projected to the first eye 50 of the viewer does not change the light direction as shown in Figures 5A-5C. Similarly, the virtual image module 120 may display the first virtual object 70 moving through several target positions successively, such as T1 → T2 → T3 → T4, as required by the training program.

To avoid complications, Figures 4 and 5A-5C do not illustrate the movement of the pupil following the movement of the first virtual object 70. Figures 6A and 6B show that when the first virtual object 70 moves from the first target position T1 to the second target position T2, the second eye 60 actually moves from the left hand side to the center, while the first eye 50 remains in approximately the same place.

Furthermore, in order for the viewer to perceive the virtual object at the target depth, the system 100, including the eye tracking module 110 and the virtual image module 120, may first need to be calibrated for the viewer. Because every viewer's eyes have different physical characteristics, including interpupillary distance (IPD), the system must be specifically calibrated for the viewer to ensure that the viewer perceives the virtual object displayed at the target depth with normal and accommodation light signals projected into the viewer's eyes.

The viewer's eye information, such as pupil location, pupil size, gaze angle (viewing angle), and convergence angle, obtained from the eye tracking module 110, may be used by the virtual image module 120 to determine the direction and speed of movement of the first virtual image 70. For example, when the weaker eye 60 is unable to follow the movement of the first virtual object 70, the virtual image module 120 may move the first virtual object 70 back to a previous position where the viewer's weaker eye 60 can still perceive the accommodation image and slow down the movement. The virtual image module 120 may also use the viewer's eye information to determine the direction and angle of the normal and accommodation light signals to ensure that both eyes, and in particular the weaker eye 60, can receive the light signals and perceive the first virtual object 70.
7A-7D show four conditions, respectively: hypotropia (eye points downward), hypertropia (eye points upward), exotropia (eye points outward), and esotropia (eye points inward), in which the pupil of the weaker eye 60 cannot randomly turn to coordinate with the pupil of the normal eye 50. As shown in FIG. 8, using exotropia as an example, the virtual image module 120 may adjust the direction and angle of the normal light signal and the accommodation light signal, respectively, so that the weaker eye 60 can perceive an accommodation image.

In some embodiments, the system 100 may be used to correct exotropia and other similar conditions. The virtual image module 120 may display a first virtual object 70 moving slowly towards and/or away from the viewer's eye 60. The eye tracking module 110 provides the viewer's eye information, such as pupil location, pupil size, and viewing angle, to the virtual image module 120, which accordingly projects a normal light signal through which the normal eye perceives a normal image and an accommodation light signal through which the weaker eye perceives an accommodation image, respectively. The virtual image module 120 then moves the accommodation image slightly away from the initial location of the weaker eye in the appropriate direction to correct. As a result, the accommodation image perceived by the weaker eye may become slightly blurred and/or both of the viewer's eyes perceive diplopia (when image fusion fails such that the normal image is separated from the accommodation image).
The human brain has the tendency and ability to automatically perform binocular vergence accommodation to obtain a clear image of an object. As a result, the viewer may slightly change the orientation of his or her weaker eye in an attempt to regain a clear image or to avoid double vision. When the viewer is successful, the feedback module 160 may provide the viewer with feedback, such as audio or vibration. The virtual image module 120 may repeatedly move the accommodation image back and forth to train the weaker eye to orient in the proper direction, which may reduce misalignment of the viewer's two eyes.

To improve the vision of the viewer's weaker eye 60, the virtual image module 120 may distinguish between the contrast and spatial frequency of the accommodation image 124 for the viewer's second eye 60 and the contrast and spatial frequency of the normal image 122 for the viewer's first eye 50. Specifically, the virtual image module 120 is configured to generate an accommodation image 122 that has a higher contrast or lower spatial frequency than the corresponding normal image 124. The virtual image module 120 may select the appropriate contrast and spatial frequency for the accommodation image 122 based at least in part on the contrast sensitivity function of the weaker eye such that the viewer's weaker eye 60 receives a stronger stimulus and perceives the accommodation image 124 clearly.
Each eye has its own contrast sensitivity function, an example of which is shown in FIG. 9. The virtual image module 120 may adjust the contrast and/or spatial frequency of the normal image 122 so that the normal eye 50 receives less stimulation and perceives the normal image 122 with lower contrast and higher spatial frequency, thereby exercising the weaker eye 60 and training it more to contribute to the viewer's vision. Without the system 100 for improving binocular vision, the viewer would avoid exercising their weaker eye, which cannot provide a clear image, and as a result, the weaker eye 60 would have even weaker vision or even become blind.

Contrast is the difference in brightness or color that makes an object (or a representation of an object in an image or display) distinguishable. In real-world visual perception, contrast is determined by the difference in color and brightness of an object and other objects within the same field of view. Through contrast sensitivity testing, a contrast sensitivity curve (also known as a contrast sensitivity function) of a viewer's eye can be plotted with angular frequency on the horizontal axis and contrast threshold on the vertical axis. Images shown to a viewer for such testing have varying contrast on the vertical coordinate and angular frequency on the horizontal coordinate.
A viewer sequentially views parallel gratings of varying width and contrast, known as sinusoidal gratings, to plot a curve. The width of the gratings and the distance between the gratings represent angular frequency measured in cycles per degree. Studies have demonstrated that a mid-level angular frequency of approximately 5-7 cycles per degree is optimally detected by most individuals compared to low and high levels of angular frequency. Contrast threshold can be defined as the minimum contrast that a patient can resolve.

As described above, the virtual image module 120 may project an accommodation image 124 with higher contrast and/or lower spatial frequencies to the viewer's weaker eye 60. Below are some examples:
For higher contrast, the adjustment image 124 has a higher luminance than the normal image 122, the adjustment image 124 is colorful while the normal image 122 is black and white (with gray scale), the adjustment image 124 is green while the normal image 122 is red. For lower spatial frequencies, the adjustment image 124 is a foreground view of the image while the normal image 122 is a background view of the same image. The virtual image module 120 first splits the image into lower spatial frequency and higher spatial image portions, and then projects the lower spatial frequency portion, such as the foreground view, to the weaker eye 60 and the higher spatial frequency portion, such as the background view, to the normal eye 50. In this situation, the adjustment image 124 and the normal image 122 look different due to different patterns and spatial frequencies, but are derived from the same image, so the adjustment image 124 (lower spatial frequency portion) is well related to the normal image 122 (higher spatial frequency portion) for binocular vision.

As shown in Fig. 10, while the virtual image module 120 moves the first virtual object 70 through different positions with different target depths, the spatial frequency of the first virtual object 70 may be adjusted according to the target depth. When moving the same virtual object to a position with a larger target depth, the spatial frequency of the exact same virtual object automatically increases as the object moves further away from the viewer. As a result, the weaker eye 60 may have difficulty in clearly perceiving the virtual object as it moves away from the viewer.
In order to maintain a clear perception of the accommodation image for the viewer's weaker eye, the virtual image module may adjust the spatial frequency of the first virtual object, including both the accommodation image and the normal image. As a result, the first virtual object has a lower spatial frequency when displayed at a greater depth. As shown in FIG. 10, when moving the first virtual object 70 to a position with a greater depth, for example, from a first target position T1 to a second target position T2 and then to a third target position T3, the spatial frequency of the first virtual object 70 is reduced to overcome the above problem.

The system 100 for improving binocular vision may further include a VEP measurement module 130 for measuring the visual evoked potential (VEP) of the viewer's eyes. VEP measures electrical signals generated in the visual cortex in response to visual stimuli. VEP refers to electrical potentials recorded from the scalp overlying the visual cortex, extracted from the electroencephalogram by signal averaging. Recording electrodes are typically placed on the midline of the occipital scalp at the back of the head.
The VEPs are used to quantify the functional integrity of the optic nerve, the visual pathway from the eye to the visual cortex of the brain, and the occipital cortex, and as a result, the VEPs provide important information for the virtual image module to adjust the manner in which the first virtual object 70 is displayed, including the speed and direction of movement of the first virtual object 70, as well as the contrast and spatial frequency of the adjustment image 124.

The VEP measurement module 130 may continually measure the VEP of either or both eyes of the viewer and provide such information to the virtual image module. Measurements may be taken frequently in a real-time manner or occasionally after the viewer's weaker eye shows some improvement. The VEP may reflect the extent to which the viewer's weaker eye can perceive an accommodative image.
The VEP may also reflect whether the viewer's eyes, particularly the weaker eye, fixate for a period of time. For example, when the viewer's weaker eye loses fixation and moves somewhat, the VEP of the weaker eye may fluctuate. Nevertheless, when the viewer's weaker eye maintains fixation, the VEP of the weaker eye may remain approximately the same. Thus, the virtual image module may display the first virtual object, including selecting an appropriate contrast and spatial frequency, as well as the direction and speed of the movement of the first virtual object, based on the VEP of either or both eyes of the viewer.

The virtual image module 120 may display the first virtual object 70 based on the VEP of either or both eyes of the viewer, even if the system does not include a VEP measurement module. A separate VEP measurement module may communicate the VEP of either or both eyes of the viewer to the system via wired or wireless techniques. Wireless techniques may include Wi-Fi, Bluetooth, Near Field Communication (NFC), Internet, telecommunications, radio frequency (RF), etc. The VEP of either or both eyes of the viewer may also be input to the system via a user interface of the system, e.g., a keyboard and mouse.

The system 100 for improving binocular vision starts by projecting an accommodation image with a suitable contrast and spatial frequency to exercise the viewer's weaker eye with a sufficiently strong stimulus. As the viewer's weaker eye becomes stronger, the virtual image module may gradually decrease the contrast of the accommodation image and/or gradually increase the spatial frequency of the accommodation image until the accommodation image is very close to the normal image while the viewer's weaker eye can still clearly perceive the accommodation image. At the same time, the viewer's weaker eye is trained to follow the movement and perceive the depth of the first virtual object by moving the first virtual object from a first position to a second position.
When the viewer is able to perform a gaze, the virtual image module may display a first virtual object at the viewer's gaze location and gaze depth (collectively, the "gaze position") provided by the eye tracking module. When the viewer moves his or her gaze from the first gaze location and first gaze depth (collectively, the "first gaze position" or "F1") to a second gaze location and second gaze depth (collectively, the "second gaze position" or "F2"), the virtual image module moves the first virtual image according to the viewer's gaze.

Employing the above mechanism of moving the first virtual object according to the movement of the viewer's gaze, many games may be designed to train the viewer's ability of gaze and image fusion. The virtual image module may display a second virtual object 75 at a predetermined location and a predetermined depth. When the viewer moves his/her gaze to move the first virtual object within a predetermined spatial range of the second virtual object or to superimpose it on the second virtual object during a predetermined period of time, the second virtual object is modified to interact with the first virtual object.
11A and 11B, a first virtual object 70 is an aiming cursor that is moved from F1 to F2 to F3 according to the viewer's gaze, and a second virtual object 75 is a target for shooting, such as a fighter jet. When the first virtual object 70, e.g., the aiming cursor, is moved to overlay a second virtual object 75, e.g., a fighter jet, the second virtual object 75 is altered to display an explosion to provide feedback to the viewer that the second virtual object 75 has been hit and destroyed.
In a second example, the first virtual object is a bowling ball and the second virtual object is a set of bowling pins, and the viewer then moves their gaze to superimpose the bowling ball onto (hit) the set of bowling pins, and the virtual image module responds to the hit by displaying a portion of the total number of bowling pins toppling, providing feedback to the viewer.
In a third example, the first virtual object is a "snake" with multiple blocks that are moved by the viewer's gaze to "eat" additional blocks that are second virtual objects. These blocks may be formed by combining various patterns with different contrasts and spatial frequencies. As a result, by moving the "snake", the stimulation of the viewer's visual cortex is enhanced, facilitating better training results.

The system 100 may interact with reality. Similar to the shooting game example above, the viewer may move his/her gaze to move a first virtual object, such as an aiming cursor, within a predetermined spatial range of a real object, such as a teapot on a table, for a predetermined period of time. The virtual image module then displays a virtual object, such as fireworks, as feedback to indicate to the viewer that the real object has been successfully hit. The feedback conditions, such as the predetermined spatial range, the predetermined period of time, the virtual object for feedback, and other parameters of the training game may be preset by the viewer or trainer. As a result, the training game may be set to require that the aiming cursor must overlap the real object for three seconds.

The system 100 may further include a real object measurement module 150 that measures the location and depth of real objects, such as a clock and a painting hanging on a wall. The real objects may be moving objects, such as a remote-controlled airplane and a dog. The real object measurement module 150, configured to be connected to other modules of the system, may continuously or periodically measure the location and depth of real objects relative to itself (or the viewer) and communicate related information to the virtual image module to determine whether a feedback condition has been met. For example, upon receiving such information, the control module 125 may calculate the spatial distance between a first virtual object, such as a aiming cursor, and the real object to determine whether the first virtual object overlaps the real object. The distance between the real object and the real object measurement module 150 (or the viewer's eye) may change throughout time. In one situation, the real object 105, such as a remote-controlled airplane, may move during the game. In another situation, the system 100 may be worn by a viewer, such as a patient, who may move his or her head during the game. As a result, the distance between the real object and the viewer's eyes needs to be measured and calculated to accurately determine whether the feedback conditions are met. The real object measurement module 150 may include a gyroscope, an indoor/outdoor global positioning system (GPS), and distance measurement components (e.g., emitters and sensors) that accurately track the location and depth variations of the real object.

In addition to displaying virtual objects, such as an exploded plane, as feedback, the system 100 may provide other types of feedback, such as sound and vibration. As a result, the system 100 may further include a feedback module 160 that generates feedback to the viewer when a feedback condition is met. The feedback module may be a speaker that provides a sound, such as an explosion, or a vibration generator that provides various types of vibrations. The type of feedback may be set by the viewer or trainer through the user interface 140.

The virtual image module 120 and methods for generating virtual images 70, 75 at predetermined locations and depths, as well as methods for moving the virtual images as desired, are discussed in detail below. International Publication No. PCT/US20/59317, filed November 6, 2020, entitled "SYSTEM AND METHOD FOR DISPLAYING AN OBJECT WITH DEPTHS," is hereby incorporated by reference in its entirety.

正常瞳孔５２と調節瞳孔６２の間の距離は、瞳孔間距離（ＩＰＤ）である。同様に、第２の向きを変えられた正常光信号（第２の正常光信号）１８’と、対応する第２の向きを変えられた調節光信号（第２の調節光信号）３８’の間の第２の角度はΘ２である。第２の深度Ｄ２は、第２の角度Θ２に関係がある。詳細には、Ｔ２での仮想対象物７０の第２の仮想両眼画素７４の第２の深度Ｄ２は、同じ式により、第２の向きを変えられた正常光信号の光路の広がりと、対応する第２の向きを変えられた調節光信号の間の第２の角度Θ２によりおよそ決定できる。第２の仮想両眼画素７４は、第１の仮想両眼画素７２よりも見る人から遠く離れている（すなわち、より大きな深度を伴う）ように、見る人により知覚されるので、第２の角度Θ２は第１の角度Θ１よりも小さい。 12, a viewer perceives a first virtual object, a tennis ball 70, in front of him or her in region C. The image of the tennis ball virtual object 70 displayed at a first target position T1 (with depth D1) is represented by a first virtual binocular pixel 72 (at its center point), and the first virtual object 70 is represented by a second virtual binocular pixel 74 as it moves to a second target position T2 (with depth D2).
The first angle between the first redirected normal light signal 16' (first normal light signal) and the corresponding first redirected adjusted light signal (first adjusted light signal) 36' is Θ1. The first depth D1 is related to the first angle Θ1. In particular, the first depth of the first virtual binocular pixel of the first virtual object 70 can be determined by the optical path extent of the first redirected normal light signal and the first angle Θ1 between the corresponding first redirected adjusted light signal. As a result, the first depth D1 of the first virtual binocular pixel 72 can be approximately calculated by the following formula:

The distance between the normal pupil 52 and the accommodation pupil 62 is the interpupillary distance (IPD). Similarly, the second angle between the second redirected normal light signal 18' and the corresponding second redirected accommodation light signal 38' is Θ2. The second depth D2 is related to the second angle Θ2. In particular, the second depth D2 of the second virtual binocular pixel 74 of the virtual object 70 at T2 can be approximately determined by the optical path extent of the second redirected normal light signal and the second angle Θ2 between the corresponding second redirected accommodation light signal by the same formula. The second angle Θ2 is smaller than the first angle Θ1 because the second virtual binocular pixel 74 is perceived by the viewer to be farther away from the viewer (i.e., with a greater depth) than the first virtual binocular pixel 72.

Moreover, however, the redirected normal light signal 16' for NLS_2 and the corresponding redirected adjusted light signal 36' for ALS_2 together display a first virtual binocular pixel 72 with a first depth D1. The redirected normal light signal 16' for NLS_2 may present an image at the same or different viewing angle as the corresponding redirected adjusted light signal 36' for ALS_2. In other words, the first angle Θ1 determines the depth of the first virtual binocular pixel 72, but the redirected normal light signal 16' for NLS_2 may or may not be the parallax of the corresponding redirected adjusted light signal 36' for ALS_2. As a result, the red, blue, and green (RBG) color intensity and/or brightness of the normal light signal and the adjusted light signal may be approximately the same or slightly different due to shadows, viewing angles, etc., to better present some 3D effects.

As described above, multiple normal light signals are generated by the normal light signal generator 10, redirected by the normal combiner 20, and then scanned directly onto the normal retina to form a normal image 122 on the right retina (normal retinal image 86 in FIG. 13). Similarly, multiple adjusted light signals are generated by the adjusted light signal generator 30, redirected by the adjusted combiner 40, and then scanned onto the adjusted retina to form an adjusted image 124 on the adjusted retina (adjusted retinal image 96 in FIG. 13).
In the embodiment shown in Fig. 12, the normal image 122 includes 36 normal pixels in a 6x6 array, and the adjustment image 124 includes 36 adjustment pixels, also in a 6x6 array. In another embodiment, the normal image 122 may include 921,600 normal pixels in a 1280x720 array, and the adjustment image 124 may include 921,600 adjustment pixels, also in a 1280x720 array. The virtual image module 120 is configured to form the normal image 122 on the normal retina and the adjustment image 124 on the adjustment retina, respectively, to generate a number of normal light signals and a corresponding number of adjustment light signals. As a result, the viewer perceives a virtual object with a specific depth in region C due to image fusion.

12, a first normal light signal 16 obtained from the normal light signal generator 10 is received and reflected by the normal coupler 20. A first redirected normal light signal 16' reaches the viewer's normal retina through the normal pupil 52 to display normal retinal pixel R43. A corresponding adjusted light signal 36 obtained from the adjusted light signal generator 30 is received and reflected by the adjusted coupler 40. A first redirected light signal 36' reaches the viewer's adjusted retina through the accommodation pupil 62 to display adjusted retinal pixel L33.
As a result of the image fusion, the viewer perceives the first virtual object 70 at a first depth D1 determined by a first angle between the first redirected normal light signal and the corresponding accommodation light signal. The angle between the redirected normal light signal and the corresponding accommodation light signal is determined by the relative horizontal distance between the normal pixel and the accommodation pixel. As a result, the depth of the virtual binocular pixel is inversely correlated with the relative horizontal distance between the normal pixel forming the virtual binocular pixel and the corresponding accommodation pixel. In other words, the deeper the viewer perceives a virtual binocular pixel, the smaller the relative horizontal distance in the X-axis between the normal pixel and the accommodation pixel forming such virtual binocular pixel.
For example, as shown in Fig. 12, the second virtual binocular pixel 74 is perceived by the viewer to have a greater depth (i.e., farther away from the viewer) than the first virtual binocular pixel 72. As a result, the horizontal distance between the second normal pixel and the second accommodation pixel is smaller than the horizontal distance between the first normal pixel and the first accommodation pixel on the retinal images 122, 124. Specifically, the horizontal distance between the second normal pixel R41 and the second accommodation pixel L51 forming the second virtual binocular pixel 74 is four pixels long. However, the distance between the first normal pixel R43 and the first accommodation pixel L33 forming the first virtual binocular pixel 72 is six pixels long.

In one embodiment shown in Figure 13, the optical paths of multiple normal optical signals and multiple modulated optical signals from the optical signal generator to the retina are illustrated. Multiple normal optical signals generated from the normal optical signal generator 10 are projected onto the normal combiner 20 to form a normal combiner image (RSI) 82. These multiple normal optical signals are redirected by the normal combiner 20 and focused into a small normal pupil image (RPI) 84, passing through the normal pupil 52, and then finally reaching the normal retina 54 to form a normal retina image (RRI) 86 (normal image 122). Each of the RSI, RPI, and RRI comprises i x j pixels.
Each normal light signal NLS(i,j) travels from RSI(i,j) to RPI(i,j) and then to RRI(x,y) through the same corresponding pixels. For example, NLS(5,3) travels from RSI(5,3) to RPI(5,3) and then to RRI(2,4). Similarly, multiple accommodation light signals generated from the accommodation light signal generator 30 are projected onto the accommodation combiner 40 to form an accommodation combiner image (LSI) 92. These multiple accommodation light signals are redirected by the accommodation combiner 40 and focused into a small accommodation pupil image (LPI) 94, passing through the accommodation pupil 62, and then finally reaching the accommodation retina 64 to form an accommodation retina image (LRI) 96 (accommodation image 124). Each of the LSI, LPI, and LRI comprises i×j pixels. Each adjusted light signal ALS(i,j) travels from LCI(i,j) to LPI(i,j) and then to LRI(x,y) through the same corresponding pixel. For example, ALS(3,1) travels from LCI(3,1) to LPI(3,1) and then to LRI(4,6). The (0,0) pixel is the pixel at the top left corner of each image.
Pixels in the retina image are left-right flipped and up-down flipped with respect to the corresponding pixels in the combiner image. Based on the proper arrangement of the relative positions and angles of the light signal generator and the combiner, each light signal has its own light path from the light signal generator to the retina. The combination of one normal light signal displaying one normal pixel on the normal retina and one corresponding accommodation light signal displaying one accommodation pixel on the accommodation retina forms a virtual binocular pixel with a unique depth as perceived by the viewer. As a result, a virtual binocular pixel in space can be represented by a pair of normal retinal pixels and accommodation retinal pixels, or a pair of normal combiner pixels and accommodation combiner pixels.

The virtual object perceived by the viewer in region C may include many virtual binocular pixels, but in this disclosure is represented by one virtual binocular pixel. To accurately describe the location of the virtual binocular pixel in space, each location in space is provided with three-dimensional (3D) coordinates, for example XYZ coordinates. In alternative embodiments, other 3D coordinate systems can be used. As a result, each virtual binocular pixel has 3D coordinates, namely horizontal, vertical, and depth directions. The horizontal direction (or X-axis direction) is along the direction of the interpupillary line. The vertical direction (or Y-axis direction) is along the midline of the face and perpendicular to the horizontal direction. The depth direction (or Z-axis direction) is perpendicular to the frontal plane and perpendicular to both the horizontal and vertical directions. The horizontal and vertical coordinates are collectively referred to as locations in this invention.

14 illustrates the relationship between pixels in the normal combiner image, pixels in the adjusted combiner image, and virtual binocular pixels. As described above, pixels in the normal combiner image have a one-to-one correspondence with pixels in the normal retinal image (normal pixels). Pixels in the adjusted combiner image have a one-to-one correspondence with pixels in the adjusted retinal image (adjusted pixels). However, pixels in the retinal image are left-right inverted and up-down flipped with respect to the corresponding pixels in the combiner image.
For a normal retinal image with 36 (6×6) normal pixels and an accommodation retinal image with 36 (6×6) accommodation pixels, assuming that all light signals are within the FOV of the viewer's two eyes, there are 216 (6×6×6) virtual binocular pixels (shown as dots) in region C. The extent of the light path of one redirected normal light signal intersects with the extent of the light path of each redirected accommodation light signal at the same row of the image. Similarly, the extent of the light path of one redirected accommodation light signal intersects with the extent of the light path of each redirected normal light signal at the same row of the image. As a result, there are 36 (6×6) virtual binocular pixels on one layer and six layers in space.
Typically, two adjacent lines representing the spread of the optical paths that intersect to form a virtual binocular pixel are shown as parallel lines in FIG. 14, but there is a small angle between them. Normal pixels and corresponding accommodation pixels that are at approximately the same height on each retina (i.e., in the same row of the normal and accommodation retinal images) tend to fuse faster. As a result, normal pixels are paired with accommodation pixels that are in the same row of the retinal images to form a virtual binocular pixel.

As shown in FIG. 15, a look-up table is created to facilitate identifying pairs of normal and accommodation pixels for each virtual binocular pixel. For example, 36 (6×6) normal pixels and 36 (6×6) accommodation pixels form 216 virtual binocular pixels numbered from 1 to 216. The first (first) virtual binocular pixel VBP(1) represents a pair of normal pixel RRI(1,1) and accommodation pixel LRI(1,1). The second (second) virtual binocular pixel VBP(2) represents a pair of normal pixel RRI(2,1) and accommodation pixel LRI(1,1). The seventh (seventh) virtual binocular pixel VBP(7) represents a pair of normal pixel RRI(1,1) and accommodation pixel LRI(2,1). The thirty-seventh (37th) virtual binocular pixel VBP(37) represents a pair of normal pixel RRI(1,2) and accommodation pixel LRI(1,2). The two hundred and sixteenth (216th) virtual binocular pixel VBP(216) represents a pair of normal pixel RRI(6,6) and accommodation pixel LRI(6,6).
As a result, it is determined which pairs of normal and accommodation pixels can be used to generate corresponding normal and accommodation light signals to display the unique virtual binocular pixel of the virtual object in space for the viewer. In addition, each row of the virtual binocular pixel on the look-up table includes a pointer to a memory address that stores the depth (z) of the perceived VBP and the position (x, y) of the perceived VBP. Additional information such as size scale, number of overlapping objects, and depth within the sequence depth can also be stored for the VBP. The size scale may be the relative size information of the unique VBP compared to the standard VBP. For example, the size scale may be set to 1 when the virtual object is displayed in the standard VBP that is 1 m in front of the viewer. As a result, the size scale may be set to 1.2 for the unique VBP that is 90 cm in front of the viewer. Similarly, the size scale may then be set to 0.8 for the unique VBP that is 1.5 m in front of the viewer.
The size scale can be used to determine the size of the virtual object for display when it moves from a first depth to a second depth. The size scale may be a magnification in the present invention. The number of overlapping objects is the number of objects that overlap each other such that one object is completely or partially hidden behind another object. The in-sequence depth provides information about the sequence of depths of the various overlapping images. For example, three images overlap each other. The in-sequence depth of a first image in the foreground may be set to 1, and the in-sequence depth of a second image hidden behind the first image may be set to 2. The number of overlapping images and the in-sequence depth may be used to determine which parts of which images need to be displayed when the various overlapping images are in motion.

The look-up table may be created by the following process. In the first step, an individual virtual map is obtained based on the IPD of the viewer, which is created by the virtual image module during start-up or calibration and specifies the boundaries of the area C in which the viewer can perceive a virtual image with depth due to the fusion of the normal retinal image and the accommodative retinal image. In the second step, for each Z-axis direction (each point in the Z coordinate), a convergence angle is calculated that identifies a pair of normal pixels and accommodative pixels on the normal retinal image and the accommodative retinal image, respectively, independent of the location of the X- and Y-coordinates. In the third step, the normal pixel and accommodative pixel pair is moved along the X-axis direction to identify the X- and Z-coordinates of each pair of normal pixels and accommodative pixels at a specific depth, independent of the location of the Y-coordinate. In the fourth step, the normal pixel and accommodative pixel pair is moved along the Y-axis direction to determine the Y-coordinate of each pair of normal pixels and accommodative pixels. As a result, a 3D coordinate system, such as XYZ, of each pair of normal pixels and accommodative pixels on the normal retinal image and the accommodative retinal image, respectively, can be determined to create the look-up table. Additionally, the third and fourth steps are interchangeable.

The optical signal generators 10 and 30 may use as their light source a laser, or a light emitting diode ("LED"), including mini and micro light emitting diodes ("LED"), organic light emitting diodes ("OLED"), or superluminescent diodes ("SLD"), or a liquid crystal on silicon (LCoS), or a liquid crystal display ("LCD"), or any combination thereof. In one embodiment, the optical signal generators 10 and 30 are laser beam scanning projectors (LBS projectors) that may include light sources including red, green, and blue lasers, light color modifiers such as dichroic and polarization combiners, and two-dimensional (2D) adjustable reflectors such as 2D electromechanical system ("MEMS") mirrors.
The 2D adjustable reflector can be replaced by two one-dimensional (1D) reflectors, such as two 1D MEMS mirrors. The LBS projector generates and scans the light signals one by one in sequence to form a 2D image at a given resolution, for example, 1280x720 pixels per frame. As a result, one light signal is generated for one pixel and projected at a time towards the combiner 20, 40. In order for a viewer to see such a 2D image from one eye, the LBS projector must generate light signals for each pixel in sequence, for example, 1280x720 light signals, within the duration of vision, for example, within 1/18 seconds. As a result, the duration of each light signal is about 60.28 nanoseconds.

In another embodiment, the light signal generators 10 and 30 may be digital light processing projectors ("DLP projectors") capable of generating 2D color images at once. Texas Instrument's DLP technology is one of several technologies that can be used to manufacture DLP projectors. 2D monochromatic image frames, which may comprise, for example, 1280x720 pixels, are projected simultaneously toward the combiners 20, 40.

The couplers 20, 40 receive and redirect the multiple optical signals generated by the optical signal generators 10, 30. In one embodiment, the couplers 20, 40 reflect the multiple optical signals so that the redirected optical signals are on the same side of the couplers 20, 40 as the incident optical signals. In another embodiment, the couplers 20, 40 refract the multiple optical signals so that the redirected optical signals are on a different side of the couplers 20, 40 than the incident optical signals. The couplers 20, 40 then function as refractors. The reflectance can vary over a wide range, such as 20% to 80%, depending in part on the power of the optical signal generator. Those skilled in the art know how to determine an appropriate reflectance based on the characteristics of the optical signal generator and coupler. Furthermore, in one embodiment, the couplers 20, 40 are optically transparent to ambient (environmental) light from the opposite side of the incident optical signals so that a viewer can simultaneously observe a real-time image. The transparency can vary over a wide range depending on the application. For AR/MR applications, transparency is preferably greater than 50%, such as about 75% in one embodiment.

The couplers 20, 40 may be made from glass or plastic material, like a lens, coated with some material, such as metal, that makes it partially transparent and partially reflective. One advantage of using reflective couplers instead of prior art waveguides to direct the optical signal to the viewer's eye is that it eliminates the problem of undesirable diffraction effects, such as multiple shadows, color shifts, etc.

The above description of the embodiments is provided to enable any person skilled in the art to make and use the subject matter. Various modifications of these embodiments will be readily apparent to those skilled in the art, and the novel principles and subject matter disclosed herein may be applied to other embodiments without the use of innovative faculty.
The claimed subject matter set forth in the following claims is not intended to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein. Additional embodiments are contemplated within the spirit and true scope of the disclosed subject matter. Thus, it is intended that the present invention cover modifications and variations that come within the scope of the appended claims and their equivalents.

Claims (24)

1. A system for improving binocular vision, comprising:
an eye tracking module configured to provide information about a viewer's eyes;
a virtual image module configured to display a first virtual object based on the viewer's eye information obtained from the eye tracking module by projecting a number of normal light signals to a first eye of the viewer to form a normal image and a number of corresponding adjustment light signals to a second eye of the viewer to form an adjustment image,
the virtual image module displays the first virtual object moving from a first target location and a first target depth to a second target location and a second target depth;
the first target depth is related to a first angle between a first of the normal optical signals and a corresponding first of the modulated optical signals, and the second target depth is related to a second angle between a second of the normal optical signals and a corresponding second of the modulated optical signals, the first target depth being different from the second target depth;
the eye tracking module provides the gaze location and the gaze depth of the viewer's eye to the virtual image module, the first target location and the first target depth being a first gaze location and a first gaze depth, respectively, and the second target location and the second target depth being a second gaze location and a second gaze depth, respectively;
the virtual image module displays a second virtual object at a predetermined location and a predetermined depth, the second virtual object being modified to interact with the first virtual object as the first virtual object moves within a predetermined spatial range of the second virtual object for a predetermined period of time .
system. The system of claim 1, wherein the modulated light signal projected to the second eye of the viewer changes direction of light more than the normal light signal projected to the first eye of the viewer when the virtual image module displays the first virtual object moving from the first target location and the first target depth to the second target location and the second target depth. 2. The system of claim 1, wherein the normal light signal projected to the first eye of the viewer does not change direction of light when the virtual image module displays the first virtual object moving from the first target location and the first target depth to the second target location and the second target depth. The system of claim 1, wherein the virtual image module displays the first virtual object moving from the second target location and the second target depth to a third target location and a third target depth, while the modulated light signal projected to the second eye of the viewer changes light direction more than the normal light signal projected to the first eye of the viewer, the third target depth is related to a third angle between the third normal light signal and a corresponding third modulated light signal, and the third target depth is different from the second target depth. The system of claim 1, wherein the virtual image module adjusts the spatial frequency of the first virtual object when it moves from the first target depth to the second target depth such that it has a lower spatial frequency when displayed at a greater depth. The system of claim 1, wherein the eye tracking module provides binocular eye information of the viewer, including at least one of pupil location, pupil size, gaze angle, convergence angle, gaze location, and gaze depth. The system of claim 1, wherein the eye tracking module comprises a first camera for the first eye and a second camera for the second eye. The system of claim 1, characterized in that the virtual image module displays the first virtual object based on the location of the viewer's pupil provided by the eye tracking module such that the first eye perceives the normal image and the second eye simultaneously perceives an accommodation image. The system of claim 1, further comprising a VEP measurement module for measuring a visual evoked potential (VEP) of the viewer's eye, wherein the virtual image module displays the first virtual object based on the VEP obtained from the VEP measurement module. The system of claim 9, wherein the virtual image module moves the first virtual object based on the VEP of the viewer or the location of the pupil of the viewer. The system of claim 1, wherein the adjustment image has a higher contrast or a lower spatial frequency than the corresponding normal image. The system of claim 1, wherein the virtual image module selects the contrast and the spatial frequency for the accommodation image based on the visual evoked potential (VEP) of the viewer's eye. The system of claim 1, wherein the accommodation image is different from the normal image for binocular fusion but is sufficiently related to the normal image for binocular fusion. 2. The system of claim 1, wherein the second virtual object is modified to interact with the first virtual object when the first virtual object is moved over the second virtual object. The system of claim 1 , further comprising a real object measurement module configured to measure a location and a depth of a real object. 16. The system of claim 15 , further comprising: providing feedback to the viewer when the first virtual object is moved within a predetermined spatial range of the real object. 2. The system of claim 1, wherein the first virtual object is a crosshair cursor and the second virtual object is modified to display an explosion when the first virtual object overlaps the second virtual object. 2. The system of claim 1, wherein the first virtual object is a bowling ball and the second virtual object is a set of bowling pins that interact with the bowling ball when the first virtual object overlays the second virtual object. The system of claim 1 , wherein the virtual image module is calibrated with respect to the viewer such that the viewer perceives the first virtual object displayed at the gaze location and gaze depth. The system of claim 1, wherein the virtual image module further comprises a control unit that processes the normal light signal and the corresponding adjusted light signal. The system of claim 1, wherein the virtual image module further comprises a normal light signal generator that generates a number of the normal light signals for the normal image of the first virtual object, a normal combiner that redirects the number of normal light signals toward the retina of the first eye of the viewer, an adjustment light signal generator that generates a number of the adjustment light signals for the adjustment image of the first virtual object, and an adjustment combiner that redirects the number of adjustment light signals toward the retina of the second eye of the viewer. 22. The system of claim 21, further comprising a support structure mountable on a head of the viewer, wherein the eye tracking module is supported by the support structure, the normal light signal generator and the adjustment light signal generator are supported by the support structure, and the normal combiner and the adjustment combiner are supported by the support structure. The system of claim 1, further comprising a user interface configured for a person to control the eye tracking module and the virtual image module. The system of claim 1 , further comprising a feedback module configured to provide feedback to the viewer when a feedback condition set by the viewer is met.

Images