JP7753082B2 - Ophthalmic imaging device, method for controlling ophthalmic imaging device, and program - Google Patents

Description

The present invention relates to an ophthalmic imaging device, a control method for an ophthalmic imaging device, and a program.

Ophthalmic devices in practical use include fundus cameras and scanning laser ophthalmoscopes (SLOs) used to obtain two-dimensional images of the fundus of the examinee's eye. Devices that use low-coherence light for optical coherence tomography (OCT) to obtain tomographic images of the examinee's eye (hereinafter referred to as OCT devices) are also in practical use. Furthermore, combined SLO and OCT devices are also useful.

Meanwhile, the vitreous, an intraocular tissue, is normally a colorless, transparent, jelly-like substance, but it is known that it can change and liquefy with age, for example, or develop slight opacities. Such opacified areas cause scattering and reflection of measurement light, so even conventional SLO and OCT devices can capture these images. However, such vitreous opacities and other intraocular floaters are generally considered to be obstacles to observing and photographing the retina. Known methods for observing retina by avoiding such opaque areas of the vitreous (hereinafter referred to as vitreous opacified areas) include the method disclosed in Patent Document 1, which avoids these areas and the method disclosed in Patent Document 2, which quantifies the impact of opacity.

Japanese Patent Application Publication No. 8-336501 JP 2015-085176 A

As mentioned above, fundus observations are performed to take into account the effects of vitreous opacity, but no devices or methods exist for photographing or analyzing these areas.

One embodiment of the present invention has been developed in light of the above-mentioned circumstances, and one of its objectives is to provide an apparatus and method capable of photographing and analyzing vitreous opacities.

An ophthalmic imaging apparatus according to an embodiment of the present invention comprises:
an imaging head including a measurement light source that emits measurement light, a scanning optical system that scans the interior of the subject's eye with the measurement light in accordance with predetermined scanning information, and a light receiving optical system that receives return light of the measurement light from the subject's eye and generates a light receiving signal;
an image generating means for generating an image using the scanning information and the light receiving signal;
a storage means for storing a plurality of images of the same type relating to the same subject's eye under a predetermined movement, the images being generated;
means for detecting the predetermined movement based on the stored images;
a calculation means for identifying, using at least two of the stored images, an area where an opacity site exists in the vitreous body of the subject's eye, and where the opacity site exhibits a movement induced by the predetermined movement and different from the detected predetermined movement; and for calculating information about the movement of the opacity site using at least two of the stored images;
Equipped with.

One embodiment of the present invention makes it possible to photograph and analyze vitreous opacities.

1 is a diagram illustrating an example of a schematic configuration of an ophthalmologic imaging apparatus according to a first embodiment. FIG. 2 is a schematic diagram illustrating a schematic optical configuration of an imaging head according to the first embodiment. 3 is a flowchart showing a series of processes of an analysis method according to the first embodiment. FIG. 10 is a schematic diagram showing an example of a patient/examination selection screen. FIG. 10 is a schematic diagram showing an example of an OCT examination screen. FIG. 10 is a schematic diagram showing an example of a VTM examination screen. FIG. 10 is a schematic diagram showing an example of a screen for setting shooting parameters. 10 is a flowchart showing a series of processes performed during SLO moving image playback. 10 is a timing chart showing the operation of each part and the state of a moving object when capturing an SLO moving image. FIG. 10 is a schematic diagram showing an example of an analysis screen. 10 is a flowchart showing a series of processes performed during analysis processing. 10 is a flowchart showing a part of the processing of an analysis method according to a second embodiment. FIG. 10 is a schematic diagram showing an example of a VTM inspection screen in the second embodiment. FIG. 10 is a schematic diagram showing an example of a VTM inspection screen in the second embodiment. 1A and 1B are schematic diagrams illustrating the relationship between tomographic images obtained when sequential scanning is performed. FIG. 10 is a schematic diagram showing a modified example of the analysis screen. FIG. 10 is a schematic diagram showing a modified example of the analysis screen.

Below, exemplary embodiments for carrying out the present invention will be described in detail with reference to the drawings. However, the dimensions, materials, shapes, and relative positions of components described in the following embodiments are arbitrary and may be changed depending on the configuration of the device to which the present invention is applied or various conditions. In addition, the same reference numerals are used between drawings to indicate identical or functionally similar elements.

(First Example)
An ophthalmic imaging apparatus and a control method thereof according to a first preferred embodiment of the present invention will be described below with reference to the accompanying drawings. The ophthalmic imaging apparatus according to this embodiment is shown as one aspect of an ophthalmic imaging apparatus that uses an acquired image to extract, from the image, a moving object that is induced by the movement of the subject's eye, such as a vitreous opacity site, and that moves differently from the subject's eye.

Figure 1 shows an example of the general configuration of an ophthalmic imaging device according to a first embodiment of the present invention. Figure 1(a) shows the configuration of the ophthalmic imaging device, and Figure 1(b) is a block diagram showing the functional configuration of the control device 120 shown in Figure 1(a).

The ophthalmic imaging device according to the first embodiment comprises an imaging head 110 (imaging device), a control device 120, a display unit 130, and an input unit 140. In this embodiment, the ophthalmic image processing device is included within the control device 120. The input unit 140 is composed of a keyboard, pointing device, etc. that function as a GUI in cooperation with the display unit 130. The imaging head 110 incorporates an optical system for imaging a specific part of the subject's eye, and is configured integrally with a stage provided with an alignment mechanism that can move in three axes directions (up and down, left and right, and front and back) to align it with the subject's eye.

The control device 120 is communicatively connected to the imaging head 110, display unit 130, and input unit 140, and can control these. The control device 120 also generates and saves images from the imaging signals of the subject's eye acquired by the imaging head 110, and displays the saved images and related information on the display unit 130. In this embodiment, the imaging head 110 also has a built-in speaker that serves as an instruction generation unit (described below) that generates instructions to induce eye movement, and also functions as an audio interface that generates necessary instructions to the subject in accordance with instructions from the control device 120.

The control device 120 can be configured using any general-purpose computer, but may also be configured using a dedicated computer provided in the ophthalmic imaging device. The display unit 130 can be configured using any display, but may also be configured as a touch panel with functions integrated with the input unit 140. In this embodiment, the imaging head 110, control device 120, and display unit 130 are provided separately, but they may also be provided integrally.

<Explanation of the Optical System of the Imaging Head 110>
First, a description will be given of the configuration of the imaging head 110. Fig. 2 is a schematic diagram showing an example of the optical configuration of the imaging head 110.

In the imaging head 110, an objective lens 211 shared by both the SLO optical system and the OCT optical system is positioned facing the subject's eye E. A first dichroic mirror 212, a second dichroic mirror 213, and a beam splitter 210 are arranged on the optical axis of the objective lens 211. These optical components cause the optical axis of the objective lens 211 to branch into an OCT optical path L1, an SLO optical path L2, an anterior segment observation optical path L3, and a fixation lamp optical path L4. The OCT optical path L1 is an optical path for capturing tomographic images of the fundus of the subject's eye and the adjacent vitreous body, and the SLO optical path L2 is an optical path for capturing a frontal image of the fundus. The anterior segment observation optical path L3 is an optical path for observing the anterior segment, and the fixation lamp optical path L4 is an optical path for presenting a fixation lamp to encourage the subject's eye E to fixate.

The first dichroic mirror 212 branches the anterior eye observation optical path L3 from the optical axis of the objective lens 211 in the reflection direction according to the wavelength band of each light ray. The second dichroic mirror 213 branches the OCT optical path L1 from the optical axis of the objective lens 211 in the reflection direction according to the wavelength band of each light ray. The beam splitter 210 branches the fixation lamp optical path L4 from the optical axis of the objective lens 211 in the reflection direction according to the wavelength band, and branches the SLO optical path L2 in the transmission direction. Note that the optical paths provided in the transmission direction and reflection direction of each dichroic mirror, etc., may be reversed.

The SLO optical path L2 is provided with an SLO scanning means 214, a focus lens 215, and a lens 216, which are shared by an SLO scanning optical system for scanning the interior of the test eye while irradiating it with SLO measurement light, and a light receiving optical system for receiving return light from the test eye E. A partial reflection mirror 217 is provided behind the lens 216 (in the opposite direction on the optical path to the objective lens 211 with respect to the lens 216) at a position conjugate with the pupil of the test eye. The partial reflection mirror 217 reflects the SLO measurement light emitted from the SLO measurement light source 221 at the center of the optical axis, and transmits return light from the test eye E in other areas. The partial reflection mirror 217 separates the SLO measurement light and return light in the form of a beam splitter.

The system extending from the SLO measurement light source 221 provided in the reflected light path of the partial reflection mirror 217 to the objective lens 211 as described above is the SLO scanning optical system. The SLO scanning optical system scans the fundus of the subject's eye with the SLO measurement light. Meanwhile, the system extending from the objective lens 211 to the SLO photodiode 220 provided in the transmitted light path of the partial reflection mirror 217 is the SLO light receiving optical system. The SLO light receiving optical system guides the return light from the subject's eye E to the SLO photodiode 220, which is then transmitted to the control device 120 as an SLO light receiving signal.

The SLO scanning means 214, which is shared by both the SLO scanning optical system and the light receiving optical system, performs two-dimensional scanning on the fundus of the test eye with SLO measurement light emitted from the SLO measurement light source 221. The SLO scanning means 214 is also used to guide the returning light from each scanning position to the SLO photodiode 220. The SLO scanning means 214 consists of an X scanner that scans the SLO measurement light in the X direction, and a Y scanner that scans in the Y direction. In this embodiment, a polygon mirror is used for the X scanner because high-speed scanning is required, and a galvanometer mirror is used for the Y scanner because low-speed scanning is sufficient.

The focus lens 215 is driven by a motor (not shown) in the direction indicated by the arrow in the figure to adjust the focus. The SLO measurement light source 221 is a light source that combines and emits lasers of four different wavelengths, for example, blue, green, red, and infrared. The output laser wavelength can be selected from any of the four types and is controlled by the control device 120. The central wavelength of the laser light emitted from the infrared light source is set to, for example, a wavelength around 780 nm to enable wavelength separation from the OCT measurement light.

The first dichroic mirror 212 has the property of reflecting light in the wavelength band around 970 nm and transmitting light with wavelengths shorter than that. A lens 222, a split prism 223, a lens 224, and an anterior segment observation CCD 225 are arranged on the optical path for anterior segment observation (anterior segment observation optical path L3) formed by the first dichroic mirror 212. The CCD 225 captures an image of the anterior segment of the subject's eye illuminated by an anterior segment observation light source (wavelength 970 nm), not shown. The split prism 223 is arranged on the anterior segment observation optical path L3 so as to be conjugate with the pupil of the subject's eye E when the distance in the Z direction (front-to-back direction) between the subject's eye E and the imaging head 110 is appropriate. As a result, when the distance in the Z direction (front-to-back direction) of the imaging head 110 is appropriate, the CCD 225 can capture an unsplit image of the subject's eye's pupil. Furthermore, if the distance in the Z direction is inappropriate, the CCD 225 can capture an image of the subject's eye pupil separated in the deflection direction of the split prism. The output signal from the CCD 225 is sent to the control device 120, which detects the relative position of the subject's eye E and the imaging head 110 based on the signal received from the CCD 225.

The imaging head 110 is equipped with an alignment mechanism that moves the entire imaging head 110 in three dimensions (X, Y, Z) to align it with the subject's eye E. The control device 120 drives this alignment mechanism using three motors (not shown) to perform the alignment operation based on relative position information between the subject's eye E and the imaging head 110 obtained from the video signal from the CCD 225.

The OCT optical path L1 is used to capture fundus tomographic images of the subject's eye E. Arranged on the OCT optical path L1 are an OCT scanning means 226, a focus lens 227, and a lens 228, which are shared by the OCT scanning optical system that scans the fundus with measurement light and the OCT light receiving optical system that receives return light from the subject's eye E. The OCT scanning means 226 is used to scan the fundus with measurement light obtained from an OCT measurement light source 229. While the OCT scanning means 226 is illustrated as a single mirror in this embodiment, it is actually configured as an XY scanner consisting of two galvanometer mirrors that scan the measurement light in two directions, X and Y. The XY scanner may be configured using any deflection means depending on the desired configuration. The measurement light scanning means may also be configured as a deflection means that can deflect light in two dimensions using a single mirror, such as a MEMS mirror.

The focus lens 227 focuses the measurement light obtained from the OCT measurement light source 229 and emitted from the optical fiber 230 onto the fundus of the test eye, and is driven in the direction of the arrow in the figure by a motor (not shown). This focusing causes the measurement light emitted from the end face of the optical fiber 230 to form a point image on the fundus of the test eye, while the returning light is imaged on the end face of the optical fiber 230 and then re-enters the optical fiber 230.

The OCT measurement light source 229 may be, for example, an SLD (Super Luminescent Diode), which emits low-coherent light with a center wavelength of 850 nm and a wavelength band of 50 nm. Note that light sources other than SLDs, such as ASE (Amplified Spontaneous Emission), may also be used as the OCT measurement light source 229, as long as they are capable of emitting low-coherent light with a predetermined center wavelength and wavelength band.

Optical fiber 230 is connected to OCT measurement light source 229 via optical coupler 234 and optical fiber 231. Optical fiber 233, connected to optical coupler 234, is further connected to spectrometer 238. Light emitted from OCT measurement light source 229 and incident on optical fiber 231 is split into OCT measurement light and reference light by optical coupler 234, and the OCT measurement light is guided to optical fiber 230. After emerging from optical fiber 230, the OCT measurement light is guided toward test eye E via optical elements up to objective lens 211, which are provided on the optical path of the OCT optical system (OCT optical path L1). The above optical elements constitute the OCT scanning optical system. The measurement light irradiated onto test eye E is reflected and scattered by the fundus of test eye E, and returns as light through the OCT scanning optical system to reach optical coupler 234 again.

Meanwhile, a reference optical path is formed by the optical fiber 232, lens 235, dispersion compensation glass 236, and reference mirror 237 connected to the optical coupler 234. The reference light obtained by splitting the light emitted from the OCT measurement light source by the optical coupler 234 is emitted toward the reference mirror 237 via the optical fiber 232, lens 235, and dispersion compensation glass 236. The reference light reflected from the reference mirror 237 reaches the optical coupler 234 again via the same optical path. The reference mirror 237 is held in an adjustable position along the optical axis as indicated by the arrow in the figure by a motor and drive mechanism (not shown). This makes it possible to adjust the optical path lengths of the OCT measurement light and the reference light, which change depending on the subject's eye E, within the coherence length. The adjusted reference light and the returned light of the OCT measurement light are combined by the optical coupler 234 and guided as interference light to the spectrometer 238 via the optical fiber 233.

The spectrometer 238 is composed of a lens 239, a lens 241, a diffraction grating 240, and a line sensor 242. The interference light emitted from the optical fiber 233 is converted into parallel light via the lens 239, then dispersed by the diffraction grating 240, and imaged by the lens 241 on the line sensor 242. The above optical components constitute the OCT light receiving optical system. The line sensor 242 reads the dispersed interference light, generates a light receiving signal for forming an OCT tomographic image, and transmits this to the control device 120 every 20 microseconds.

A lens 218 and a fixation lamp panel 219 are provided on the fixation lamp optical path L4, which is formed in the reflection direction using a beam splitter 210 made of, for example, plain glass. The fixation lamp panel 219 is, for example, a high-brightness organic EL panel, and is controlled by the control device 120 to display various patterns selected by the examiner at the desired position. A visible image of this pattern is presented to the subject's eye E, acting as a fixation lamp to encourage the subject to fixate.

In the above-described embodiment, a Michelson interferometer was used as the interferometer, but a Mach-Zehnder interferometer may also be used. Depending on the difference in light intensity between the OCT measurement light and the reference light, it is desirable to use a Mach-Zehnder interferometer when the difference in light intensity is large, and a Michelson interferometer when the difference in light intensity is relatively small. In addition, while a spectral domain (SD) OCT device was used in this embodiment, a swept source (SS) OCT device using a wavelength-swept light source is also applicable. Furthermore, while a composite system of an SLO optical system and an OCT optical system is used as the imaging system in this embodiment, a composite system is not necessarily required, and the present invention can also be applied to an imaging head composed of only an SLO optical system or only an OCT optical system.

<Explanation of the control device>
Next, the control device 120 will be described. Fig. 1B is a block diagram showing the control configuration of the control device 120. The control device 120 according to this embodiment includes an image generation unit 121, a storage unit 122, a control unit 123, an analysis unit 124, an acquisition unit 125, and an instruction generation unit 128.

In this embodiment, for example, the image generation unit 121 functions as an image generation means that generates an image using a light receiving signal, etc., which will be described later. In this embodiment, for example, the memory unit 122 functions as a memory means that stores scanning information, light receiving signals, and images generated by the image generation unit 121. In this embodiment, for example, the control unit 123 functions as a control means that controls the imaging head 110, the image generation unit 121, and the memory unit 122. In this embodiment, for example, the analysis unit 124 functions as an analysis unit that analyzes the control information of the control unit 123 and the images (data) generated by the image generation unit 121 to generate necessary information. In this embodiment, for example, the acquisition unit 125 functions as an acquisition means that acquires necessary information from the analysis unit 124, the imaging head 110, the display unit 130, and the input unit 140. The analysis unit 124 also incorporates an information extraction unit 126 that extracts information regarding the movement of the eyeball of the subject's eye E, which will be described later, and the movement of areas within the eyeball, such as opacified areas of the vitreous body. The instruction generating unit 128 generates instructions (described below) to the subject, such as prompting the subject to blink, and the instruction generating unit may be included in the control device 120 or may be separately located, for example, in the imaging head 110.

Each component of the control device 120, other than the memory unit 122, can be implemented as a module executed by a processor such as a CPU (Central Processing Unit) or MPU (Micro Processing Unit). The processor may be, for example, a GPU (Graphical Processing Unit) or FPGA (Field-Programmable Gate Array). Each component of the control device 120 may also be implemented as a circuit that performs a specific function, such as an ASIC. The memory unit 122 may be implemented as any storage medium, such as an optical disk (e.g., a hard disk) or memory.

Before describing the actual measurement and analysis processes of the device as an embodiment of the present invention, we will first explain the major roles of each unit. The control unit 123 controls the device by outputting control signals to each unit based on the programs constituting each component of the control device 120 stored in the memory unit 122 and control information generated by the analysis unit 124. Controlled objects include, for example, the imaging head 110, the display unit 130, and each unit within the control device 120. The control device 120 or the control unit 123 functions as display control means for displaying application windows, etc., described below, on the display unit 130. Note that, hereinafter, explanations of the control of the acquisition unit 125, image generation unit 121, analysis unit 124, etc. within the control device 120 will be omitted, as they are controlled by the control unit 123.

The acquisition unit 125 acquires information necessary for device operation, image generation, and image analysis. Information necessary for image generation and analysis includes, for example, light reception signals and video signals for generating images, and information indicating the state of the optical system for converting light reception signals into image data. Light reception signals and video signals include, for example, the output of the SLO photodiode 220, CCD 225, or line sensor 242. Information indicating the state of the optical system for converting light reception signals into image data includes, for example, scanning information from the SLO scanning unit 214 and OCT scanning unit 226, the two scanning units of the imaging head 110. The acquisition unit 125 also functions as a GUI by cooperating with the display unit 130 to acquire information such as operations and instructions from the examiner, or acquires information such as operations and instructions from the examiner input via the input unit 140.

The image generation unit 121 generates images (data) using signals acquired by the acquisition unit 125, but can also generate video images (files) from multiple images generated from continuously acquired signals. Examples of images that can be generated include SLO images, tomographic images, and anterior segment images.

The memory unit 122 stores necessary information. This information includes, for example, SLO images, tomographic images, anterior segment images, SLO video images, tomographic video images generated by the image generation unit 121, and imaging parameters used to capture images of the subject's eye. The memory unit 122 also stores necessary information such as computer programs prepared to configure each component of the control device 120.

The analysis unit 124 includes an information extraction unit 126. The information extraction unit 126 analyzes the images (data) generated by the image generation unit 121 and generates necessary information, such as information regarding eye movement and information regarding the movement of areas with vitreous opacity. Furthermore, the necessary information also includes, for example, control information used when controlling the optical system and additional information to be stored together with the images.

Next, we will explain the actual basic operation of each component using the generation of SLO images and OCT images as examples. In the method for generating a frontal fundus (SLO) moving image, the control unit 123 turns on the SLO measurement light source 221 and then sends a scanning control signal, which is scanning information, to the SLO scanning means 214 to initiate scanning with the SLO measurement light. The returned light from the fundus of the subject's eye is converted into a light-receiving signal by the SLO photodiode 220. The image generation unit 121 samples this light-receiving signal and sequentially stores it in the memory unit 122 as pixel values of pixels corresponding to each scanning position, which is scanning information, to generate a single piece of frontal fundus image data. A still image can be displayed by reading this image data and displaying it on the display unit 130. Furthermore, by repeating this series of controls and continuously displaying the sequentially obtained still images in chronological order, a frontal fundus moving image can be displayed. Furthermore, while this frontal fundus moving image is being generated, the analysis unit 124 can detect the focus state by analyzing image data at different positions of the focus lens 215. The control unit 123 controls the position of the focus lens 215 based on the results of the focus state detection, thereby enabling adjustment control of the device, such as SLO autofocus adjustment.

In addition, by switching the wavelength range of the SLO measurement light emitted from the SLO measurement light source 221, it is possible to generate and save monochrome moving images, color still images, and color moving images, and these images can be selected as needed depending on the intended use of the image. For example, in the preview state before actual imaging, it is desirable to select monochrome moving images using only infrared wavelengths. For fundus diagnosis, it is desirable to output the blue, green, and red wavelength ranges exclusively and sequentially to obtain image data for each color, and then combine these to generate a color still image and display it.

Next, a method for generating an OCT tomographic image will be described. The control unit 123 turns on the OCT measurement light source 229 and further sends a scanning control signal to the OCT scanning means 226 to start two-dimensional scanning of the fundus of the test eye with the OCT measurement light. The interference light obtained from the return light of the OCT measurement light from the fundus is dispersed by the spectroscope 238 and converted into a received light signal by the line sensor 242, which is acquired by the acquisition unit 125. The image generation unit 121 generates image data in the depth direction (Z direction) of the fundus of the test eye by, for example, performing a Fourier transform on the received light signal acquired by the acquisition unit 125.

By correlating this image data with the scanning position, which is the scanning information, image data in the depth direction at a specified position is obtained. Acquiring image data consisting of a one-dimensional brightness or density distribution in the depth direction in this manner is called an A-scan. The OCT scanning means 226 scans the fundus of the subject's eye E with the OCT measuring light in a specified main scanning direction while repeatedly performing A-scans to acquire multiple consecutive A-scan images, thereby obtaining a single tomographic image. For example, if the control device 120 scans the OCT measuring light in the X direction, a tomographic image in the XZ plane is obtained, and if it scans in the Y direction, a tomographic image in the YZ plane is obtained. Scanning the fundus of the subject's eye E in this manner in a specified main scanning direction to acquire tomographic data in the scanning direction is called a B-scan, and the resulting tomographic image is called a B-scan image.

Furthermore, a 3D OCT image can be obtained by scanning in a direction perpendicular to the XZ or YZ plane of the B scan. It is also possible to record tomographic images as moving tomographic images by repeatedly performing B scans. In either case, a moving image includes a series of reproducible images taken at close times, different from a moving image, and the continuous capturing and storing of these images can be considered recording, while the continuous display of these images can be considered playback.

The flowcharts in Figures 3(a) to 3(c) illustrate a series of processes, from observing the subject's eye E to capturing images and analyzing them, in an ophthalmic imaging device comprising the imaging head 110, control device 120, display unit 130, and input unit 140 described above. Figure 3(a) shows the main processes performed during an examination. Figure 3(b) shows detailed processing for previewing and adjusting the device performed in step S303 in Figure 3(a) during normal OCT imaging. Figure 3(c) shows detailed processing for adjusting the device in the vitreous opacity region (vitreous turbidity motion) imaging mode (hereinafter referred to as VTM imaging mode), which is characteristic of this embodiment. Note that in this embodiment, the vitreous opacity region corresponds to a region in the vitreous of the subject's eye E, consisting of opacity in the vitreous itself, floaters in the vitreous caused by retinal detachment, etc., and blood seeping into the eye. It is also used as a general term for areas in the vitreous that can be identified as opacified when observing fundus images or tomographic images of the fundus.

<S301 Patient Selection>
Prior to the examination, after the imaging head 110 has completed startup preparation, the examiner displays the application window 400 shown in FIG. 4 on the display unit 130. Then, using the patient/examination selection screen tab 450, the examiner selects the patient/examination selection screen and inputs or selects the patient to be examined from that screen. For example, if it is an initial examination, the examiner inputs all necessary information such as the patient name in the patient input box 451. If it is a follow-up examination, the examiner selects the patient from the patient candidates searched and displayed in the patient list 452 in response to the partial input. When the input or selection is complete, the examiner presses the OK button 455. In response to this press, the flow proceeds to step S302.

<S302 Mode selection>
The ophthalmic imaging apparatus used in this embodiment is provided with an OCT imaging mode for capturing normal OCT tomographic images and a VTM imaging mode. In the VTM imaging mode, images of moving objects, such as vitreous opacities, floating within the vitreous body and moving relative to the retina of the subject's eye are captured. In step S302, the examiner selects the examination mode to be performed by selecting either the OCT examination screen tab 410 or the VTM examination screen tab 430 in the application window 400 shown in FIG. 4. When the examiner selects the OCT examination screen tab 410, the application window 400 displays an OCT examination screen capable of displaying an SLO moving image 412 and an anterior segment moving image 413 together with an acquired OCT tomographic image 411, as illustrated in FIG. 5.

The control device 120 displays the selected OCT examination screen on the display unit 130 and initializes the imaging head 110. That is, the control device 120 sets the fixation light panel 219 to illuminate the center of the fixation light panel 219 so that the line of sight of the subject's eye E is parallel to the optical axis of the objective lens 211 of the imaging head 110. The control device 120 also sets the focus lens 215 and focus lens 227 to the focus lens position (OD position) corresponding to a subject's emmetropic eye, i.e., 0 diopter.

<S303 Preview and Device Adjustment>
Next, the examiner makes various adjustments to the device while achieving the preview state. Here, the process executed when the OCT imaging mode is selected will be described. Details of the process executed when capturing a tomographic image in the normal OCT imaging mode will be described below with reference to the flowchart shown in FIG. 3B using the application window 400 shown in FIG. 5 .

<S311 Manual Alignment>
Using a GUI such as a slide bar provided in the window of the anterior eye segment moving image 413, the examiner manually adjusts the XY position of the imaging head 110 so that part of the pupil is imaged by the anterior eye segment observation system. This adjustment may be performed by providing a separate operation button on the screen, or by using a mouse or the like to specify a point on the screen where the center of the window should be located. Furthermore, the imaging head 110 and the subject's eye E can be adjusted in the optical axis (Z) direction by, for example, operating the wheel of the mouse. When the position is reached where part of the pupil is imaged, the examiner presses the start button 414 to begin auto-alignment.

<S312 Rough Auto Alignment>
When the examiner presses the start button 414, the image generation unit 121, control unit 123, analysis unit 124, and acquisition unit 125 in the control device 120 cooperate to start automatic alignment. First, the analysis unit 124 analyzes the anterior eye moving image 413 acquired by the acquisition unit 125 to determine the pupil center of the subject's eye E. The stage is driven to move the pupil center closer to the center of the anterior eye moving image, which is aligned with the optical axis of the imaging head 110, thereby adjusting the position of the imaging head 110 in the X and Y directions. Then, the positions of the imaging head 110 and the subject's eye E in the optical axis (Z) direction are adjusted based on, for example, the spacing between corneal reflection images of multiple anterior eye illumination light sources (not shown). The adjustments may be performed alternately and continuously or in parallel. When the position of the imaging head 110 falls within a predetermined tolerance range, the control unit 123 transitions the flow to step S313.

<S313 Fine Auto Alignment>
In step S313, the control unit 123 instructs the SLO measurement light source 221 to emit only infrared light and drives the SLO scanning unit 214 to start capturing (previewing) an SLO moving image. When the rough auto-alignment adjustment is performed, the edge of the pupil of the subject's eye in the anterior segment image becomes sharp. Furthermore, as described above, the pupil split by the split prism 223 provided on the anterior segment observation light path L3 becomes clearly observable. The control unit 123 drives the stage to adjust the XY direction position of the imaging head 110 so that it approaches the center of the anterior segment moving image, and also adjusts the Z direction position of the imaging head 110 so that the split amount of the split pupil becomes zero. When the fine auto-alignment confirms that the positional deviation falls within a predetermined range, the control unit 123 proceeds to step S314.

<S314 Autofocus>
In step S314, the control unit 123 starts acquiring an SLO moving image and simultaneously starts scanning the fundus with the OCT measurement light. After that, once acquisition of an appropriate SLO moving image is confirmed, autofocus adjustment is started. In this embodiment, the characteristics of confocal SLO are utilized, in which the output of the SLO photodiode 220, i.e., the light-receiving signal, is maximized when the fundus is properly focused. Then, by driving the focus lens 215 and monitoring the light-receiving signal, so-called hill-climbing AF autofocus is performed, in which the focus position is determined. Of course, this adjustment may be performed based on the sharpness of the image rather than the magnitude of the light-receiving signal.

Focus adjustment of the OCT optical system may also be performed in parallel by linking the focus lens 227 of the OCT optical system. Focus adjustment may also be performed manually, in which case it may be possible to operate it using a GUI such as the focus switch 415 provided on the OCT examination screen.

After the focus adjustment of the SLO optical system and the OCT optical system is completed, the information extraction unit 126 calculates the movement of the fundus of the test eye from the generated SLO moving image. The control unit 123 issues a drive instruction (for correcting the scanning position) to the OCT scanning means 226 to correct this movement, causing the scanning position of the OCT measurement light to track. This is known as fundus tracking. As a result, the movement of the test eye E is offset in the OCT moving image, making it possible to obtain an OCT tomographic moving image in which the influence of the movement of the test eye E is eliminated. When the illustrated hill-climbing AF ends and fundus tracking begins, the flow proceeds to step S315.

<S315 (One-time) Reference Optical Path Length Adjustment>
In this state, the control device 120 accepts an operation by the examiner on the reference optical path length adjustment unit 416 on the selected OCT examination screen. Then, in accordance with the received operation, the reference mirror 237 is driven to perform the OCT reference optical path length adjustment. This completes the device adjustment in the preview state in S303, and the control unit 123 proceeds to step S304 in FIG. 3A.

<S304 Capture image &check>
When the apparatus adjustment in the preview state is completed and the examiner presses the capture button 417, the above-mentioned B scan, for example, is executed and an OCT tomographic image is captured. After the imaging is completed, a confirmation screen (not shown) is displayed to display the captured tomographic image and confirm the tomographic image, and an OK button and a re-capture button, for example, may be provided on the confirmation screen. On such a confirmation screen, if the imaging result is OK, the OK button is pressed to save the tomographic image, and if it is NG, the re-capture button is pressed to discard the captured tomographic image, and the processing from step S303 is repeated.

<S305 Analysis>
After capturing an OCT tomographic image, selecting the analysis screen using tab 490 displays a screen for displaying the analysis results of the captured OCT tomographic image, such as the thickness of retinal layers, etc. Note that the analysis of the OCT tomographic image and the displayed content are the same as those of a general OCT device, and therefore will not be described here.

<VTM imaging mode>
Next, details of the processing performed in the VTM imaging mode, which is a feature of this embodiment, will be described. Most of the processing performed in the VTM imaging mode is similar to that in the OCT imaging mode described above and follows the flow described in FIG. 3A. In the VTM imaging mode, in the mode selection step S302, the examiner selects the display screen using the VTM examination screen tab 430 to select the VTM imaging mode. This opens the VTM examination screen in the application window 400 illustrated in FIG. 6. The VTM examination screen, compared to the OCT examination screen illustrated in FIG. 5, adds, for example, a video display switch for searching for moving objects in the vitreous body, such as vitreous opacity sites, an imaging switch corresponding to eye movement, and a laser light selection switch. The flow then proceeds to the preview and device adjustment processing in step S303. Details of the processing performed in step S303 in the VTM imaging mode are shown, for example, in FIG. 3C. In FIG. 3C, the processes from manual alignment in S311 to autofocus in S314 are the same as those in the OCT imaging mode, and therefore a description thereof will be omitted here.

<S316 Opaque site search>
After autofocusing on the fundus region is completed in step S314, the flow proceeds to step S316. In step S316, the examiner searches for an opacified region 432 in the vitreous body while dragging a fixation light mark 431 displayed on the SLO moving image 412 with a pointing device such as a mouse.
<S317 Manual Focus>
If an opacified area 432 is found, the examiner operates the focus adjustment switch 433 as necessary to adjust the position of the focus lens, etc., so that the opacified area 432 is in focus. The laser adjustment switch 434 located below the focus adjustment switch 433 adjusts the laser power of the SLO measurement light source 221. The examiner adjusts this switch to set the light intensity of the SLO measurement light so that the opacified area 432 is easily visible. Since observation without mydriasis is usually desirable, the IR radio button can be selected to select an IR laser light source. Of course, it is also possible to dilate the pupil of the examined eye with a mydriatic agent and record in color using a visible light laser. In this case, simply select the Color radio button. This outputs light in the blue, green, and red wavelength regions exclusively and sequentially as measurement light, thereby acquiring image data for each color. A color still image can be generated by combining the pixel values of the frames of each color obtained in this manner. By performing the above operations, the preview and device adjustment process in step S303 is completed, and the flow proceeds to step S304 after a predetermined time has elapsed or when an adjustment end button (not shown) is operated.

<S304 Capture image &check>
When the examiner presses the recording start (REC) button 435, the control unit 123 issues a predetermined instruction to the subject via the instruction generating unit 128. The instruction generating unit 128 instructs the subject to make a warning announcement, such as "Please blink when the chime sounds," via a speaker built into the imaging head 110. Then, for example, two seconds after the announcement ends, the instruction generating unit 128 instructs the speaker to continue sounding the chime as a predetermined instruction to encourage blinking, and the control unit 123 instructs the acquisition unit 125 to start recording a video for, for example, eight seconds. Thereafter, the auto-alignment is temporarily suspended, and when the analysis unit 124 detects from the video signal of the image for anterior eye observation that the blinking has ended, i.e., that the eyelids have opened (the brightness of the anterior eye image has decreased and the pupil has been observed), the control unit 123 instructs the auto-alignment to resume. The image frame at this time is marked or the like, and the timing is associated with the image and stored in the storage unit 122. Recording may start from this point. The blink detection described above may of course be detection of the start of a blink, or may be performed by monitoring the SLO light reception signal.

The examiner can play back the recorded SLO video to check it. That is, when recording is complete, GUI 436, which includes the play button, playback position selection bar, and other controls for performing post-capture operations, becomes active. Pressing the play button 436a here starts playback of the SLO video. Repeated playback is also possible by entering the number of repetitions in the Repeat text box. Pressing the rewind button also returns to the start of playback and allows for playback or pause. Pressing the stop button 436b returns the display screen to the fundus preview. The playback position selection bar 437 indicates the playback timing during the recording of the image being played back, and can also specify the recording timing for playback. The chime mark 437a and timing mark 437b above it indicate the time when the chime sounds and the eyelid opens (the brightness of the anterior eye image decreases and the pupil is observed), respectively. The examiner can use these marks to determine the start point of playback.

<Processing during recording and playback>
When playing back moving images, in addition to playing back the original images as they are, it is also possible to perform image processing on the original images to make it easier for the examiner to observe the opacified areas. In this embodiment, for example, retinal movement cancellation processing, which cancels the movement of the retinal pattern that forms the background when displaying the opacified area, opacity enhancement processing, which emphasizes the vitreous opacity in the original image, and processing that reduces the contrast of the retinal pattern that forms the background, are possible. The execution or selection of these processes can be performed via the display screen, for example, by selecting the display screen in tab 470 of the VTM settings screen in the application window 400 shown in Figure 7. More specifically, the selection is performed using three checkboxes in area 471 shown on the VTM settings screen.

Next, the processing performed to display opacified areas will be explained with reference to the flowchart in Figure 8. Here, we will explain an example of video playback in which IR laser is selected as the SLO measurement light, a monochrome video captured using near-infrared light is subjected to all image processing, and the video is repeatedly played back once.

<S801 Reading of moving images>
When the recording of the SLO moving image is completed, in step S801, the control device 120 reads out the moving image to be analyzed from the storage unit 122. Taking the case of vitreous opacity as an example, the moving image to be analyzed shows the opacified area 432 as a slightly dark shadow that moves within the subject's eye relative to the retinal pattern, as shown in Figure 6. Once the moving image is read out, the flow proceeds to step S802. Note that in the following description, the vitreous opacified area will be considered a moving object corresponding to the retinal pattern.

<S802 Cancellation of retinal pattern movement>
In the next step S802, a process for canceling the movement of the background retinal pattern is performed. Specifically, the analysis unit 124 first calculates the amount of positional shift of the retinal pattern between a series of frames caused by eye movement, using one image from a temporally adjacent frame as a reference frame. The amount of positional shift can be determined, for example, by comparing the central 50% portion (approximately 70% x 70%) of the target frame with the reference frame while shifting their positions, and determining the positional shift that maximizes image correlation. The calculated shift amount is then used to adjust the position of each frame to generate a moving image in which the movement of the retinal pattern has been canceled. By performing this process, a moving image can be generated in which the relative movement of the moving object with respect to the movement of the retinal pattern is extracted while reducing the influence of the movement of the retinal pattern.

In this embodiment, as described above, eye movement that results in retinal pattern movement occurs in response to the announcement to the subject, and moving images of the fundus of the subject's eye during this movement are acquired to detect moving objects that exhibit movement different from the retinal pattern movement. However, since the retinal pattern movement may be too large at the start of eye movement, it is advantageous to calculate the positional deviation between frames by going back in time from the end of recording when eye movement has settled down. Once the amount of positional deviation between each frame has been determined, the flow proceeds to step S803.

<S803 Generation of retinal pattern image>
In the next step S803, an arithmetic average is taken of each frame image aligned in step S802, reducing the influence of the moving object and generating an image of a retinal pattern that can serve as a background. For example, in this embodiment, assuming a case where the relative movement of the moving object is relatively large, an arithmetic average is taken of each frame to generate a retinal pattern that can serve as a background with the contrast of the moving object portion sufficiently reduced. Once the retinal pattern image is generated, the flow proceeds to step S804.

<S804: Generation of label image>
In the next step S804, the analysis unit 124 generates a difference image between each frame of the moving image generated in step S802 and the retinal pattern image generated in step S803. The difference image is then subjected to binarization processing to generate a labeled image that distinguishes between the area where the moving object exists and the retinal pattern background. Of course, the binarization processing may involve determining an appropriate threshold value and performing noise processing such as removing minute areas, removing missing areas, and filling in holes. After generating the labeled image, the flow proceeds to step S805.

<S805 Emphasis Processing, S806 Suppression Processing>
In the next step S805, a moving object is emphasized by, for example, detecting the edges of the labeled image. Then, the brightness values of pixels corresponding to the edges of the moving image in which the retinal pattern shift amount generated in step S802 has been canceled are changed, and the edges of the moving object are emphasized by contouring them in a color that is easily distinguishable from the red color often found in the retinal pattern, such as blue or green. Note that the method for emphasizing the moving object is not limited to this; for example, a hue may be added to the area labeled as a moving object. Furthermore, in step S806, a process for suppressing the display state of the background area is performed, such as by reducing the contrast or brightness of the background area. By performing such suppression process, the examiner can more clearly recognize the moving object in the image in which the moving object is superimposed on the retinal pattern. While it is preferable to perform both steps S805 and S806, it is also possible to perform only one of them if it is easy to identify the moving object.

Here, when the examiner presses the play button 436a, playback of the video begins. However, if processing has been selected using the three checkboxes in area 471 shown on the VTM settings screen, the video will be displayed after the above-mentioned series of processes have been performed before playback. These processes may be performed each time the video is played, or the results of the processes may be stored in the storage unit 122 for a predetermined number of days so that they can be reused. When the moving object enhancement process in step S805 and the retinal pattern image suppression process in step S806 are completed, the flow proceeds to step S807.

<S807 Saving video images>
In step S807, the recorded video is checked, and if the result is OK, the OK button 438 is pressed to save the video as the examination result. If the expected image was not recorded, the start recording (REC) button is pressed again. In this case, the current recorded data and its image processing results are discarded and re-imaging is performed. If, during this check, it is necessary to readjust the alignment or focus or re-search for opacities, the Start button is pressed again, and the examination procedure returns to step S312, where readjustment and re-imaging are performed. If OK is selected, the original image is saved, and the next examination can be performed. At this time, not only the original image but also the video after the image processing described above, i.e., the video with the fundus movement canceled and the labeled image, may be saved. Through the above processing, the imaging and confirmation process of step S304 in Figure 3(a) in VTM imaging mode is completed.

(Variation 1)
In the first embodiment described above, the process executed in step S304 when the recording start (REC) button 435 is pressed involves instructing the subject to blink by voice output from the instruction generating unit 128. However, the manner of instruction to the subject is not limited to this, and may be, for example, by changing the flashing or lighting pattern of the fixation light panel 219, or a combination of both. Furthermore, in such cases, the subject may also be instructed by voice, such as, "When the fixation light flashes (or changes its shape), please blink once."

(Variation 2)
In the first embodiment and the first modification described above, the operational instruction issued when the recording start (REC) button 435 is pressed in step S304 is an instruction to the subject to blink by audio output or the like, which induces eye movement. However, the instruction to induce eye movement is not limited to blinking, and may be an instruction to guide the line of sight by moving the position of a fixation light. The ophthalmologic imaging device according to the second modification illustrated here also provides a mode that uses this movement of the fixation light. In this case, the audio instruction mode in the second modification can be specified by, for example, switching between Blink and Fixation radio buttons provided above the recording start (REC) button.

By selecting this mode, eye movement can be induced by using the movement of the fixation light to guide the subject's gaze. In this case, it is recommended to provide a warning announcement to the subject in advance, such as, "When the fixation light moves, please immediately follow its movement." Then, when the examiner presses the recording start (REC) button 435, the control device 120 moves the fixation light displayed on the fixation light panel 219 to induce eye movement. The imaging head 110 begins capturing images of the subject's eye E undergoing eye movement and capturing and storing the SLO video image displayed on the display unit 130, i.e., recording. After recording for a fixed period of time, such as five seconds, recording is stopped in step S304 of the flowchart shown in Figure 3(a). During this time, anterior segment observation continues, so the start and end of eye movement can be detected from the video signal, and the detection results can be used to control the start and end of recording.

The operation of each part and the state of the moving object at this time are shown in Figure 9(c) as a timing chart. The timing chart shows, from top to bottom, the audio output channel, the horizontal X coordinate value of the fixation light, the relative velocity V of the moving center of gravity position with respect to the retinal pattern, and the recording status, with the horizontal axis indicating the passage of time from the moment the recording start button is turned ON. Also, as an example of how to move the fixation light presentation position, the presentation position is moved 3 mm horizontally (in the X direction) as shown in Figure 9(b) from the origin position on the optical axis, which is the initial position shown in Figure 9(a). After recording is completed, the fixation light is moved so as to slowly return from the moved fixation light presentation position to the initial presentation position.

When the recording start (RCE) button 435 is pressed, as described above, a warning announcement such as "The fixation light will move when the chime sounds. Please follow the movement of the fixation light immediately" is output from the speaker, and the chime is output one second later. As the chime ends, the fixation light moves from the presentation position shown in Figure 9(a) to the presentation position shown in Figure 9(b) in approximately one second. As the fixation light presentation position moves, eye movement occurs in the subject's eye following the fixation light, and one of the moving objects, for example, the center of gravity of a vitreous opacity site, begins to move. If vitreous liquefaction is progressing, this movement is accompanied by a predetermined delay. Therefore, the relative position of the opacity site with respect to the retinal pattern moves in the opposite direction to the movement of the retinal pattern (i.e., the direction of eye movement), resulting in a temporary negative relative velocity.

The opacified area then begins to move in the direction of eye movement, following the movement of the vitreous body. However, even after the fixation light stops at the presentation position shown in Figure 9(b), the opacified area continues to move at a substantially constant speed for, for example, several seconds. Then, after 3 to 6 seconds, it slows down and comes to a near standstill. In other words, the vitreous area moves in response to the movement of the background retinal pattern, but this movement is different from the movement of the retinal pattern. In this example, recording continues for 7 seconds from the start of the movement of the opacified area, and then stops.

Of course, this type of eye movement induction is not limited to one time, and can be repeated a predetermined number of times. In that case, for example, a return announcement such as "The fixation light will return slowly" is made, and after recording stops, the fixation light's presentation position is slowly returned to its original position over a period of about four seconds. Then, for example, a second chime is output just before the end of a further three-second waiting period, and recording is started again, and the measurement cycle is repeated.

Note that the fixation light movement parameters exemplified here can be set and changed, for example, via the VTM settings screen shown in Figure 7, in the same way as selecting video processing. Fixation movement area 472 displays changeable parameters for fixation light movement. Fixation light movement parameters include the movement direction (four directions - up, down, left, and right - can be selected using the direction of the arrow), the movement amount (specified as a converted distance on the retina), the number of repetitions, and the duration of the repetitions. Area 472 is provided with a GUI for setting these parameters. Additionally, Announcement area 473 is provided with a GUI for specifying whether or not to issue a signal or audio announcement when a blink instruction or a fixation light movement instruction is issued.

Of course, in addition to these simple individual parameter settings for the direction of fixation light movement, number of repetitions, and recording duration, the examiner may also be allowed to freely change various parameters. Furthermore, a selectable sequence for performing an examination using multiple appropriate parameters may be provided, allowing the examiner to select from among these options.

Furthermore, while the recording time for the SLO video image was set to a predetermined fixed time in this embodiment, it is also effective to set it to wait for the movement of the opacified area associated with the induced eye movement to subside. In this case, the analysis unit 124 simply confirms that the movement of the opacified area has subsided. This can be achieved, for example, by calculating a fluctuation parameter related to the change in the SLO video image, such as the sum of pixel value fluctuations between consecutive frames of the SLO video image, and the control unit 123 monitors this fluctuation parameter. Of course, an upper limit on the recording time, for example, 10 seconds, may be set, and if the movement of the opacified area does not subside, the system may transition to the next step after 10 seconds. On the other hand, it is also possible to monitor whether the subject is correctly tracking the movement of the fixation light and moving their gaze. In this case, the positional deviation amount calculated in step S802 of the flowchart in Figure 8 described above, i.e., the amount of retinal pattern movement, and the movement of the fixation light presentation position may be compared. Furthermore, if a large discrepancy is observed between these two, it may be useful to temporarily suspend the measurement and display a warning.

<S305 Analysis>
When the recording of the moving image is completed in step S807, the flow proceeds to step S305, where an analysis process for the moving object (opacified area) is executed. At this time, as shown in Fig. 10, after the stored moving image is analyzed, an analysis screen is activated on the display unit 130, which displays the state of the moving object highlighted for easy confirmation along with the analysis results. The analysis process executed here will be described below with reference to the flowchart in Fig. 11.

If a moving image in which the movement of the fundus of the subject's eye has been cancelled and a labeled image have not been saved during the previous recording, the analysis unit 124 first performs the series of processes shown in Figure 8. Specifically, the original image is read in step S801, and the image processing described above is performed in steps S802 to S804 to generate a labeled image. Of course, if the original image has been saved, it goes without saying that the analysis can begin with analyzing the moving object labeled with the labeled image. The analysis unit 124 performs the following processes in the subsequent steps S811 to S813.

<S811 Identification of moving object>
If there are multiple moving objects moving within the subject's eye in the final frame, they are identified by assigning an ID number to each. Then, the IDs assigned to each moving object in adjacent frames are inherited based on each moving object's parameters (area, shape, color, etc.). When the video is played back, this ID may be displayed in the form of a number associated with the moving object, as exemplified by the moving object ID number 491 illustrated in FIG. 10 . After the ID is assigned, the flow proceeds to step S812. In the embodiment shown here, edge enhancement processing is performed after ID assignment, but it may also be performed before ID assignment, or it may be performed only on moving objects to which IDs have been assigned. This processing may also not be performed.

<S812 Calculation of relative velocity of moving object>
First, to calculate the relative velocity, which is one of the feature quantities of each moving object, the area center of gravity coordinates of each moving object in each frame are calculated and a trajectory is determined. Next, the velocity of each moving object is calculated from the movement distance, which is the relative movement amount of the moving object between frames, and the frame rate. When there are multiple moving objects, it is also useful to determine the center of gravity or central position as a representative point and calculate and use the group trajectory and group velocity.

In this embodiment, the SLO moving image to be analyzed is a moving image in which the movement of the background retinal pattern has been canceled out. Therefore, the trajectory determined here is a two-dimensional relative position with respect to the fundus of the test eye, and the velocity calculated based on this is also a two-dimensional relative velocity. From here on, the relative velocity calculated in this embodiment refers to the projected component in the direction of the induced eye movement. Note that this may simply be a projection in the indicated direction, or the direction of the sum vector or average vector of the actual eye movement vectors may be used. However, the object of analysis is not limited to this relative velocity; it is also possible to use the change in absolute value of the velocity as the object of analysis, or the apparent velocity of the moving object rather than the velocity relative to the retinal pattern. Furthermore, these analysis objects may be provided switchably and switched as appropriate.

In any case, these indices relating to the speed of moving objects can be used as indices representing the fluidity of the vitreous. In this embodiment, the speed calculated for each frame, i.e., as a function of time, was used as an index representing this fluidity, but the average speed of a moving object between two frames obtained at specific different times may also be calculated. It is also possible to calculate statistical quantities such as the average speed over the recording time or variance, or the speed waveform at each time. In other words, at least two frames obtained at different times are required to obtain the dynamic characteristics of a moving object. This type of statistical analysis processing will be explained in detail later. Furthermore, the indices representing fluidity and the calculation methods are not limited to these. Acceleration may also be used, or changes in movement direction may be quantified. Furthermore, for the movement of a moving object caused by eye movement due to movement of the fixation light, the ratio of this relative speed to the movement speed of the retinal pattern may be calculated.

<S813 Calculation of Other Feature Amounts of Moving Object>
In this embodiment, after calculating the speed and other parameters of the moving object, other characteristics of the moving object are also obtained. These characteristics may include, for example, the average observed area and average observed density of each moving object over the recording time, as well as hue and various fluctuations. These parameters can be used to understand the state of the moving object, and if the moving object is a vitreous opacity, they can be used to appropriately determine the impact of the opacity on vision.

Using Figure 10, we will further explain an example of an analysis screen selected by tab 490 to play back and check the above-mentioned analysis results and stored video. Here, we will explain the case where a video image is analyzed that records the movement of a moving object that occurs when the line of sight is guided by a fixation light, as shown in Variation 2.

The video being analyzed is displayed as SLO video 412, and below this display area is a GUI display area 492 for controlling video playback. This display screen also provides a group of playback control buttons 493, a group of image processing control checkboxes 494, and a group of seek bars 495 for setting playback conditions.

The playback control button group 493 is a group of buttons that control playback operations such as play, stop, pause, and play from the beginning. The image processing control checkbox group 494 functions in the same way as the three checkboxes shown on the VTM settings screen shown in Figure 7.

Next, the seek bar display group 495 will be described in detail. The seek bar in the seek bar display group 495 indicates the current playback position in the series of captured video images. It moves to the right on the page as the video plays, and the playback position can be changed by operating the slider. A speed graph of the moving object being analyzed is displayed above the seek bar, with the time axis aligned with the corresponding video playback position. The dashed-dotted line on the graph is a marker indicating the current playback timing, and in conjunction with the graph display, it is possible to understand the timing of the current playback. In addition, the chime mark 437a drawn on the horizontal axis of the graph indicates the time when the chime sounds, similar to the mark on the display screen displayed on the VTM test screen in Figure 6, but the time when the chime sounds coincides with the start timing of the fixation light movement. The check boxes in the upper right corner of each graph will be described later.

The moving object to be analyzed can be selected by clicking on the moving object ID number 491 displayed on or near the moving object depicted in the SLO video image 412, or from the analysis object switching pull-down menu 496. The results of this selection are displayed, for example, as shown in Figure 10, in a form in which the outer periphery of the moving object is emphasized (see moving object with moving object ID number 1), or by displaying the moving object in color. In addition, the analysis results of the movement of the selected moving object are displayed near the analysis object switching pull-down menu 496. These analysis results will be explained in order below.

The average speed graph 497 shows the average speed change of the speed graphs for which the checkboxes in the upper right corner of the graph on the seek bar are checked. In other words, the average speed graph 497 represents the typical movement of the moving object being analyzed. The horizontal axis of the average speed graph 497, which is the time axis, can be overlaid with multiple speed graphs so that the timing of the chime output, i.e., the start of the fixation light movement, matches. Furthermore, the multiple speed graphs may be aligned so that the rise of the observed speeds matches, and may be switchable with the previously overlapped graphs.

The following two periods, a floating period and a tail period, are defined during the speed observation period, and the analysis result display 498 extracts and displays, for example, the following as other information related to the moving object:
(1) Floating period: A period during which the moving object continues to move at a substantially constant speed after the eye movement starts and the moving object starts to move. In this embodiment, this is defined as the period from when the observed speed increases to 90% of the maximum speed to when the observed speed decreases to 80%.
(2) Tail period: A period after the floating period in which the speed of the moving object gradually decreases. In this embodiment, this is defined as the period until the speed becomes 10% or less of the maximum speed, for example.
(3) Movement delay time (Dlay): Defined as the delay time from the bell mark (the start time of the fixation light movement) to the start time of the floating period.
(4) Moving object speed parameters: These are defined as the average speed VAve, the maximum speed Vmax, the reduction ratio RDecel in the tail period, etc. of the moving object selected for analysis.
(5) Moving object characteristic parameters: These are defined as the average observed area SAve, the average observed concentration DAve, etc.
Furthermore, as the measurement conditions, for example, a fixation light movement parameter 480 may be displayed. As the movement parameters, for example, the same items as those in the Fixation movement area 471 in the VTM settings screen can be displayed.

<S814 Video playback, etc.>
Using the GUI provided as described above, the examiner can switch the analysis target and check the analysis results. In addition, by freely playing back recorded videos, the movement of moving objects can be observed in detail. As described above, an example of a process for saving a video captured in VTM imaging mode and then immediately analyzing a moving object in the saved image and displaying the analysis results has been described. However, in this embodiment, this analysis and display of the analysis results can also be performed on videos recorded and stored in previous examinations. In such a case, the examiner can select the examination and video 454 to be analyzed using, for example, the tab 450 shown in FIG. 4 . After the patient is specified on the patient/examination selection screen, the screen appears. Since this process is commonly performed in general ophthalmic examination devices, a detailed description of it will be omitted here.

In this embodiment, moving objects are extracted by correcting for the movement of the retinal pattern in the background and then creating a difference image between each of a series of frames and the background image (background difference method). However, the extraction method is not limited to background difference method, and may be based on, for example, the results of optical flow calculations using block matching, or the results of an inter-frame difference method that detects moving objects from the logical product of the differences between three or more frame images.

Below, an example of extracting a moving object using optical flow results obtained by the block matching method will be described as Variation 3. Also, an example of extracting a moving object using the inter-frame difference method will be described as Variation 4.

(Variation 3)
In Variation 3, a block matching method (also known as a region-based method), which is one method for calculating optical flow, is adopted to detect moving objects using template matching. Specifically, one of two temporally adjacent images is used as the target image and the other as the reference image, and the position to which a pixel at a predetermined position in the reference image has moved in the target image is determined. More specifically, for example, a small square region centered on a predetermined pixel in the reference image, i.e., a partial image, is used as a template block, and a search is made for the position at which the highest degree of match between the images is achieved within an area twice the length and width of the partial image, i.e., four times the area, centered on the predetermined pixel position in the reference image. Then, a vector quantity with the predetermined pixel position in the reference image as the starting point and the position at which the highest degree of match is achieved in the target image as the optical flow for each pixel is used.

To check the degree of match between the two images, you can look at the sum of absolute differences or the correlation between the images. To shorten the calculation time, the calculation can be terminated when the value obtained by adding the residuals exceeds a certain threshold. Of course, the method for calculating optical flow is not limited to block matching; other methods, such as sequential residual matching, which have lower calculation costs, can also be used.

The target video for which optical flow is to be calculated can be, for example, the video read out in step 801 in the flow of Figure 8 or Figure 11. In other words, according to this modified example, it is possible to obtain a labeled image in step S804 by using a video in which the movement of the retinal pattern has not been canceled as the target image, without performing the processing in the following steps S802 and S803.

Moving objects, such as vitreous opacities, that are the subject of analysis in this modified example move differently from the background retinal pattern. When a moving object does not occupy a significant proportion of the entire screen, the majority of the optical flow of each pixel calculated for adjacent image pairs has a nearly uniform direction and magnitude, corresponding to the movement of the retinal pattern caused by eye movement. Therefore, by recognizing pixels in which a flow with a direction or magnitude separable from this is observed as a pixel that constitutes a moving object, it is possible to identify the moving object. Of course, this separation is not observed during times when the moving object in question moves in a manner similar to the movement of the retinal pattern, so it is not always possible to separate the moving object. However, by identifying pixels in which this separation can be confirmed as moving objects for a predetermined period of time during the observation period, it is possible to generate a labeled image that distinguishes the area where the moving object exists from the background retinal pattern. In this modified example, the subsequent processing is the same as that described in Example 1, so further explanation will be omitted.

Alternatively, the target image may be a moving image in which the movement of the retinal pattern obtained in step S802 has been cancelled, and a labeled image may be generated from this. In this case, the optical flow of the pixels that make up the retinal pattern will be at approximately 0 level, so the search for the position where the image match is highest will converge within a narrower range. This not only shortens the search time, but also makes it possible to easily separate and identify moving objects using a simpler binarization process based on the magnitude of the flow, instead of the separation process performed by analyzing the direction and magnitude of the optical flow of each pixel as described above. In this case, the amount of calculation can also be significantly reduced by manually or otherwise specifying the range in which the target moving object exists in advance.

(Variation 4)
In Modification 3, an example of detecting a moving object by calculating optical flow was shown, but as described below in Modification 4, it is also useful to use an inter-frame difference method using three adjacent images. The processing performed in this modification is basically the same as the processing shown in Figure 8 or Figure 11. That is, after reading out a moving image in step S801 and generating a moving image in which the movement of the retinal pattern has been canceled in step S802, the following processing is executed in the next step 804 of generating a labeled image.

Here, let N be the frame image from which a moving object is to be extracted and identified, and let N-1 and N+1 be the images before and after it. In this case, ND1 and ND2, which are the difference images between image N-1 and image N, and image N and image N+1, respectively, are created, and threshold processing is performed on these to obtain binary images. Here, by performing a logical AND operation on the two binary images and extracting the common area between them, it is possible to extract or identify the moving object in image N. Then, based on the results, a label image is generated that identifies the area where the moving object exists and the retinal pattern that is the background. In this modified example, the subsequent processing is the same as that described in the first embodiment, so further explanation will be omitted.

In this modified example, the selection of the previous and next images does not only target images from adjacent frames, but may also use images from frames separated by a predetermined time depending on the moving speed of the moving object. Furthermore, improvements can be made, such as combining these images in a timely manner to further ensure the extraction of the moving object. Furthermore, the threshold used for binarization may also reflect the variance in brightness values in images of multiple frames from a predetermined period in the past. By adopting such a method, the image will be less susceptible to changes in the background due to lighting conditions caused by the alignment status of the optical head, and improved detection performance can be expected.

As described above, the ophthalmic imaging device according to this embodiment includes an imaging head 110, an image generating means (image generating unit 121), a memory means (memory unit 122), a means for detecting the movement of the subject's eye (control unit 123), and an extraction means (information extracting unit 126). The imaging head includes a measurement light source (221, 229), a scanning optical system (214, 226), and a light receiving optical system (220, 238). The measurement light source emits SLO measurement light (or light for obtaining OCT measurement light). The scanning optical system scans the interior of the subject's eye with the measurement light in accordance with predetermined scanning information. The light receiving optical system receives the measurement light returning from the subject's eye and generates a light receiving signal. The image generating means can generate an image by, for example, using the scanning information to identify the position in the subject's eye from which the light receiving signal was acquired and converting the light receiving signal corresponding to the identified position into luminance information, etc. The storage means stores multiple images of the same type related to the same subject's eye, which are generated by the image generation means. The stored images are images obtained under predetermined movements, such as movements of the subject's eye in response to audio instructions or movements exceeding a predetermined threshold when the subject's eye is detected (movements that allow the detection of unique movements of vitreous opacities, etc.). Examples of images of the same type include frontal fundus images and fundus tomographic images. The means for detecting the subject's eye movement detects the movement of the subject's eye by comparing multiple stored images obtained from the same subject's eye. Moving objects, such as vitreous opacities, exist within the subject's eye, particularly in the vitreous. These moving objects move in response to the movement of the subject's eye, but because they are suspended in the vitreous, they move differently from the detected movement of the subject's eye. The extraction means extracts information about these moving objects using at least two of the images stored in the storage means.

The above-described ophthalmic imaging device may further include an instruction generating means (instruction generating unit 128) that issues instructions to the subject to induce eye movement. In this case, the instruction generating means may include audio output means, such as a speaker or chime, that generates audio instructions to the subject to induce blinking of the subject's eye. While this audio output means is provided in the imaging head 110 in this embodiment, it may also be located elsewhere than the imaging head; for example, it may be provided in an examination room, with the control unit 123 issuing instructions via communication or other means. As described above, the specified movement of the subject's eye is detected using images. However, if the specified movement is initiated by, for example, an audio instruction from the instruction generating means, the images following the instruction may be used to detect a moving object. Furthermore, the movement of the subject's eye may be detected not only based on images but also on received light signals. For example, the movement of the subject's eye may be detected based on the peak position of the received light signal or the movement of a boundary position that is thought to form an edge when the image is generated.

The imaging head may further include a fixation lamp display means (fixation lamp panel 219) that displays a fixation lamp at a predetermined position to guide the fixation of the subject's eye. In this case, the instruction generating means may issue an instruction to the fixation lamp display means to move the display position of the fixation lamp in order to induce movement of the subject's eye. Furthermore, the instruction generating means may repeat the various instructions described above a predetermined number of times. The means for detecting the movement of the subject's eye may also detect the movement of the subject's eye based on a received light signal.

The above-described ophthalmologic imaging device can further include a blink detection means for detecting blinking of the subject's eye as one of the movements of the subject's eye. This blink detection means can be, for example, a means for imaging the anterior segment of the subject's eye (anterior segment observation optical system) provided in the imaging head 110. In this case, the start of movement of the subject's eye can be detected based on the image acquired by this imaging means. The storage means can store the image generated by the image generation means in association with the detected movement of the subject's eye.

The above-mentioned ophthalmologic imaging device may further include a display control means (control unit 123) that controls a display means (display unit 130) that is connected to or integrated with the device. The display control means may further cause the display means to repeatedly display a moving image that is generated using multiple images based on light reception signals acquired successively over time.

The above-described embodiment can also be understood as a control method for controlling an ophthalmic imaging device. This control method includes a step of generating an image (image generation processing in step S304) using predetermined scanning information for scanning the interior of the subject's eye with measurement light and a light reception signal obtained by receiving the measurement light returned from the subject's eye. Then, multiple images generated by continuous acquisition are stored as a moving image in the memory unit 122 (process of storing the generated images in step S304). The stored moving image is read out in step S801, and movement of the subject's eye is detected based on the image in step S802. After movement of the subject's eye is detected, information about moving objects such as vitreous opacities is extracted from the read moving image, etc. in step S804.

(Second Example)
In the first embodiment, the moving image to be captured in the VTM imaging mode was an SLO moving image captured by the SLO optical system shown in FIG. 2 . The ophthalmic imaging apparatus used in the first embodiment also includes an OCT optical system. Therefore, OCT moving images can also be captured in parallel in the VTM imaging mode. In this embodiment, the OCT moving image is used to extract moving objects in the VTM imaging mode. The ophthalmic imaging apparatus used in this embodiment is the same as the ophthalmic imaging apparatus described in the first embodiment, and therefore a description thereof will be omitted here.

<OCT Imaging in VTM Imaging Mode>
The processes performed in capturing and analyzing OCT moving images in the VTM imaging mode are largely the same as those performed in capturing and analyzing SLO moving images described in the first embodiment. Therefore, steps S303 and S304 of the process executed in the flowchart of FIG. 3A , which differ from those in the first embodiment, will be described below with reference to FIGS. 12 and 13 . Note that while FIG. 12 is a flowchart of the processes executed in the second embodiment for preview and device adjustment, the processes executed up to step S317 are the same as those executed in FIG. 3C in the first embodiment. Therefore, a description of the processes up to step S317 will be omitted here. After performing manual focus in step S317, the examiner further prepares for OCT imaging using the VTM examination screen according to this embodiment shown in FIG. 13A .

<S318 Setting OCT Scanning Parameters>
In the VTM examination screen according to this embodiment, in addition to the displays on the VTM examination screen (for SLO moving images) according to the first embodiment shown in Fig. 6, switches and the like for handling OCT moving images are added. Fig. 13(a) shows a VTM examination screen selected by a tab 430 as an example to which such switches are added. Note that a focus adjustment switch 433 and a laser adjustment switch 434 below it are provided on the VTM examination screen, as in the first embodiment. The examiner operates the focus adjustment switch 433 to adjust the focus to the opacified area 432.

After completing the adjustments, check the OCT checkbox 441 to enable the option to perform OCT imaging in parallel. The drop-down menu next to the OCT checkbox 441 allows you to select the number of B-scans; in this embodiment, 1, 3, 5, and 7 are pre-presented. The examiner selects the appropriate number of B-scans from these. The following explanation uses an example where three B-scans are selected.

When the OCT checkbox 441 is checked, a scanning line mark 442 indicating the OCT scanning area is displayed in the SLO moving image 412, and scanning of the fundus with OCT measurement light begins. In this embodiment, the initial values of the scanning information for OCT video imaging are, for example, three scanning lines, a scanning line spacing of 0.5 mm, and the center of the central scanning line located at the center of the SLO moving image 412. Then, scanning with the OCT measurement light is performed in the direction of the eye movement induced by the instruction to induce eye movement.

In addition, in this embodiment, the examiner can adjust the scanning conditions by manipulating the scanning line mark 442. For example, the scan position can be adjusted by dragging the center of the scanning line mark 442, the scan width by dragging the end points, and the scan angle can be adjusted by manipulating an angle adjustment marker (not shown) that appears when the mouse is placed over the scanning line mark 442. By adjusting the scanning conditions in this way, it is possible to set the scanning conditions so that the scanning line mark 442 covers the expected trajectory of the vitreous opacity area 432. The examiner then switches the display switch radio button 443 located next to the anterior segment video image 413 from Anterior to OCT and checks the OCT tomographic image.

<S315 Reference optical path automatic adjustment>
13B shows the VTM examination screen on which the display has been switched from an anterior segment moving image 413 to an OCT tomographic moving image 444 by switching the display switch radio button 443. The tomographic image displayed in the OCT tomographic moving image 444 is a tomographic moving image of a scanning line numbered in a spin box provided below the display switch radio button 443. Furthermore, the opacified vitreous region 432 displayed on the SLO image is displayed on the OCT tomographic moving image 444 as a tomographic image 445 of the opacified vitreous region.

In this embodiment, the scanning line mark 442 on which the tomographic image is displayed may be displayed as a thicker line than the other scanning line marks 442, or in a different color, so that it can be distinguished. While observing the OCT tomographic video image 444, the examiner operates the reference optical path length adjustment unit 416 to adjust the OCT reference optical path length. Performing this operation completes the device adjustment in the preview state, which is the process performed in step S303. After the device adjustment is complete, the flow proceeds to step S304.

<S304 Capture image &check>
The process executed when the record start (REC) button 435 is pressed is similar to the process executed in the first embodiment described above. When the examiner presses the record start (REC) button 435, the instruction generating unit 128 issues an instruction to the subject to induce eye movement. As an example of the instruction, for example, a warning announcement such as "Please blink when a chime sounds" is output, and after the chime prompting the subject to blink, video recording for, for example, 8 seconds begins. At this time, the OCT measurement light is repeatedly scanned sequentially along the scanning lines displayed as the scanning line marks 442, and the tomographic images are also recorded as video images in parallel with the SLO layer images.

After recording is complete, the recorded SLO video images and OCT video images can be viewed by pressing the play button 436a to play back the recording, just as in the first embodiment. In this embodiment, the playback of the OCT video images and SLO video images is synchronized. During playback, the same operations as those available in the first embodiment, such as repeat playback, rewinding, and pausing, can also be performed in this embodiment.

<Processing during recording and playback>
In the first embodiment, when reproducing a moving image, in addition to reproducing the original image as is, it is also possible to perform image processing on the original image so that the examiner can more easily observe the opacified area and then display it. The same applies to the reproduction of OCT moving images in this embodiment. Specifically, in addition to reproducing the original image as is, it is also possible to perform the above-mentioned retinal movement cancellation processing, opacity enhancement processing, and contrast reduction processing to reproduce the image. The retinal movement cancellation processing is a processing that cancels the movement of the background retinal pattern, and the opacity enhancement processing is a processing that emphasizes the vitreous opacity in the original image and displays it. Furthermore, the contrast reduction processing is a processing that reduces the contrast of the background retinal pattern.

In the first embodiment, when displaying the SLO video, the movement of the background retinal pattern is canceled in step S802 in FIG. 8 . In this embodiment, similar processing can be performed on the captured tomographic images in the OCT tomographic video 444 to cancel the movement of the tomographic images. Specifically, the analysis unit 124 first calculates the amount of positional shift of the tomographic images between a series of frames due to eye movement, using one image from a temporally adjacent frame as the reference frame. The amount of positional shift can be determined, for example, by comparing the images while shifting the position of approximately 50% of the central portion of the retinal tomographic section of the target frame (approximately 70% x 70%) with the reference frame, and determining the amount of positional shift that maximizes image correlation. The position of each frame is then adjusted using the determined amount of shift to generate a video in which the movement of the retinal pattern has been canceled. By performing this processing, a video can be generated in which the relative movement of the moving object with respect to the movement of the tomographic image is extracted while reducing the effect of the movement of the tomographic image.

As with the first embodiment, it is advantageous to perform calculations retroactively from the end of recording, when eye movement has settled, to detect positional deviations between frames. The processes performed in steps S803 to S806 in the first embodiment are also performed in the same way for the tomographic video images of this embodiment. That is, an averaging of each frame image is performed to generate a tomographic image that is less affected by moving objects and can serve as background, and each frame of the generated video and the tomographic image are used to generate a labeled image that distinguishes between areas where moving objects exist and retinal patterns that form the background. Similar processes can then be performed for highlighting moving objects and reducing the contrast of background areas. The selection method for video playback is also the same as in the first embodiment.

<Checking and saving video images>
Using the above functions, the recorded video is checked, and if the results are satisfactory, the OK button 438 is pressed to save the video as the examination result. If the expected image is not recorded, the start recording (REC) button can be pressed again, as in the first embodiment. However, in this embodiment, the number of scanning lines is limited in order to perform OCT imaging as a video. For this reason, adjusting the OCT imaging conditions to properly record the target moving object with this limited number of scanning lines is somewhat difficult, and it is undeniable that adjusting the measurement range in particular takes some time.

(Variation 1)
In the above description, the display switching radio button 443 is used to switch from the display of the anterior segment moving image 413 to the OCT tomographic moving image 444. However, even when "Anterior" is selected with the display switching radio button 443 and the anterior segment moving image 413 is displayed, the display may be automatically switched to the OCT tomographic moving image 444 when an operation is better performed while viewing the OCT tomographic image. Examples of such an operation include an operation on the scanning line mark 442 or the reference optical path length adjustment unit 416, or when a moving image is being played back. Furthermore, the display may be automatically switched to the OCT tomographic moving image 444 when appropriate alignment is performed by auto-alignment and the amount of misalignment is within an allowable range. Furthermore, it is also useful to be able to switch the display positions of the SLO moving image and the OCT moving image during playback.

(Variation 2)
The ophthalmic imaging device used in this embodiment has an analysis function for performing movement analysis of moving objects, such as vitreous opacification areas recorded in a video that has been recorded and confirmed, that move differently from the tomographic image of the subject's eye observed as the background. However, it would be useful in eye examinations if simple observation of moving objects in the vitreous body were possible without performing such analysis of vitreous opacification areas, etc. In the following Modification 2, an example focusing on observation of such moving objects will be described.

In the first and second embodiments, post-processing is performed on the recorded video to cancel the movement of the retinal pattern and tomographic image, and even to extract moving objects. In contrast, in this modified example, the process of canceling the movement of the retinal pattern and tomographic image is performed based on the tracking process that is performed during image capture.

In the first embodiment, fundus tracking is performed in step S314 of FIG. 3(b), etc. In step S314, the information extraction unit 126 processes the SLO moving image generated by the image generation unit 121 to calculate the movement of the fundus of the test eye. Then, based on the calculation results, the control unit 123 issues a drive instruction to the OCT scanning means 226 so that the OCT measurement light follows the movement of the test eye E. In this modified example, the control unit 123 controls the SLO scanning means 214 so that not only the OCT measurement light but also the SLO measurement light follows the movement of the test eye E. By applying fundus tracking to the SLO scanning system in this way, the obtained SLO image is an image in which the background retinal pattern does not move. Therefore, without performing post-processing after imaging, images in which the movement of the background retinal pattern is canceled can be obtained over time, making it possible to observe the relative movement of a moving object.

Alternatively, a provisional measurement may be performed before the actual measurement, and a retinal pattern image that can serve as the background may be generated in advance based on that data. In this case, this retinal pattern image and the real-time image captured during the actual imaging are calculated and aligned in real time, and the aligned image is then displayed. This method also makes it possible to perform processing to highlight moving objects and suppress background areas in conjunction with the extraction of moving objects. Note that since performing such real-time calculations of images can result in a large computational load, it is desirable for the control unit to be equipped with a GPU or similar.

As explained above, in this modified example, the examiner can also repeatedly play back and check the behavior of moving objects such as recorded vitreous opacities. Furthermore, by highlighting these, the examiner can observe the behavior of the vitreous opacities independently of eye movement.

<S305 Analysis>
Here, the analysis of moving objects and the display of analysis results in the second embodiment will be described. The results of the analysis process of the stored moving images are displayed on the analysis screen as in the first embodiment, but the analysis process is performed for each B-scan moving image in each OCT scanning line. Figures 14(a), 14(b), and 14(c) show three tomographic images obtained by scanning different scanning lines arranged at arbitrary intervals with measurement light. Furthermore, moving objects 445a, 445b, and 445c shown in each tomographic image indicate moving objects identified in the respective tomographic moving images.

When the width of each scan line is less than a certain width, the identified moving object can be determined as a single moving object with a contour on, for example, three slices, as shown in Figure 14(d). Therefore, the shape can be defined using a polygon model in which the intersections of the contour and a grid of a predetermined pitch on the B-scan cross section are used as contour points. Alternatively, a wireframe model can be formed using, for example, spline interpolation to estimate the approximate shape of the three-dimensional moving object. Furthermore, the estimated shape can be used to calculate the moving object's features. However, when the number of slices is small, these processes are not very useful, and understanding the positional relationship between the moving object and the retina is more important. Furthermore, when analyzing the three-dimensional velocity of moving objects, it is important to treat them as a single moving object rather than as separate moving objects. Other processing performed during analysis is similar to that described in the first embodiment, and therefore will not be described here.

<S814 Video playback, etc.>
In the second embodiment, as in the first embodiment, the examiner can switch between moving objects to be analyzed and check the analysis results for each moving object. In addition, as in the first embodiment, the examiner can freely play back the recorded video to observe the movement of the moving objects in detail.

Figure 15(a) shows an example of how OCT video is displayed on the analysis screen of this modified example. In an examination in which OCT imaging is performed in parallel, a button 501 for opening an OCT video window is displayed to the left of the SLO video image 412 on the analysis screen. When the examiner clicks this button, a pop-up window 502 for displaying the OCT video opens, as shown in Figure 15(b), and a scan line mark 442 indicating the scanning position where OCT imaging was performed is displayed on the SLO video image 412. The playback operation of the OCT video image is always synchronized with the playback operation of the SLO video image, and playback conditions, etc. can be controlled using the playback control button group 493, image processing control checkbox group 494, and seek bar display group 495, etc.

In the OCT video images displayed in the pop-up window 502, for example, an ID may be assigned to the identified moving object 503, and the moving object may be color-coded for each B-scan video in the same color as the scanning line mark 442 and displayed superimposed. Each B-scan video may also be displayed independently (see FIG. 14(e)). Furthermore, the pop-up window 502 may be movable to any position in the application window 400, and the display magnification may be variable. Furthermore, the pop-up window 502 may be made full screen or closed using a button located above.

As described above, in the ophthalmic imaging apparatus according to this embodiment, a moving object can also be extracted using a tomographic image of the subject's eye acquired by the OCT optical system. In this case, in the imaging head 110, to generate a tomographic image, the light-receiving optical system generates interference light between the reference light corresponding to the measurement light and the returned light, and the image generating means (image generating unit 121) generates a tomographic image of the subject's eye using the interference light. In this case, the scanning optical system (OCT scanning means 226) in the OCT optical system preferably scans the measurement light in a direction (scanning direction in this embodiment) determined based on the direction of movement of the subject's eye detected from the generated image.

As described above, the ophthalmic imaging device according to the second embodiment enables simple three-dimensional observation and analysis in addition to two-dimensional observation and analysis of moving objects. Therefore, more detailed information can be obtained to determine the degree of vitreous liquefaction, the behavior of moving objects such as areas of vitreous opacity, and the impact on visual function.

(Other Examples)
The present invention can also be realized by providing a program that realizes one or more functions of the above-described embodiments and modifications to a system or device via a network or a storage medium, and having the computer of the system or device read and execute the program. The computer has one or more processors or circuits, and may include multiple separate computers or a network of multiple separate processors or circuits to read and execute computer-executable instructions.

The processor or circuitry may include a central processing unit (CPU), a microprocessing unit (MPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), or a field programmable gate array (FPGA). The processor or circuitry may also include a digital signal processor (DSP), a data flow processor (DFP), or a neural processing unit (NPU).

The present invention has been described above with reference to examples and modifications, but the present invention is not limited to the above-mentioned examples and modifications. The present invention also includes inventions that have been modified within the scope of the spirit of the present invention, and inventions that are equivalent to the present invention. Furthermore, the above-mentioned examples and modifications can be combined as appropriate within the scope of the spirit of the present invention.

110: Imaging head, 120: Control device, 130: Display device, 140: Input device, 121: Image generation unit, 122: Memory unit, 123: Control unit, 124: Analysis unit, 125: Acquisition unit, 126: Information extraction unit

Claims (18)

an imaging head including a measurement light source that emits measurement light, a scanning optical system that scans the interior of the subject's eye with the measurement light in accordance with predetermined scanning information, and a light receiving optical system that receives return light of the measurement light from the subject's eye and generates a light receiving signal;
an image generating means for generating an image using the scanning information and the light receiving signal;
a storage means for storing a plurality of images of the same type relating to the same subject's eye under a predetermined movement, the images being generated;
means for detecting the predetermined movement based on the stored images;
an analysis means for identifying, using at least two of the stored images, an area where an opacity site exists in the vitreous body of the subject's eye, and which exhibits a movement induced by the predetermined movement and different from the detected predetermined movement, and for calculating information about the movement of the identified opacity site using at least two of the stored images;
An ophthalmic imaging device comprising: The ophthalmologic imaging device of claim 1 further comprises an instruction generating means for issuing an instruction to the subject to induce the predetermined movement in the subject's eye. an imaging head including a measurement light source that emits measurement light, a scanning optical system that scans the interior of the subject's eye with the measurement light in accordance with predetermined scanning information, and a light receiving optical system that receives return light of the measurement light from the subject's eye and generates a light receiving signal;
an image generating means for generating an image using the scanning information and the light receiving signal;
a storage means for storing the generated images of the same type of images of the subject's eye after an instruction generating means has issued an instruction to the subject to induce a predetermined movement in the subject's eye;
a calculation means for identifying, using at least two of the stored images, an area where an opacity site exists within the vitreous body of the subject's eye, and where the opacity site exhibits a movement different from the predetermined movement when induced by the predetermined movement, and for calculating information about the movement of the identified opacity site using at least two of the stored images;
An ophthalmic imaging device comprising: An ophthalmologic imaging device according to claim 2 or 3, wherein the instruction generating means includes an audio output means for generating audio instructions to the subject to induce blinking of the subject's eye. the imaging head further includes a fixation lamp presenting means for presenting a fixation lamp at a predetermined position to guide the fixation of the subject's eye,
4. The ophthalmologic imaging apparatus according to claim 2, wherein the instruction generating means generates an instruction to the fixation lamp presenting means to move the presenting position of the fixation lamp in order to induce a movement of the subject's eye. An ophthalmologic imaging device according to any one of claims 2 to 5, wherein the instruction generating means repeats the instruction a predetermined number of times. The ophthalmologic imaging device according to any one of claims 1 to 6, further comprising a blink detection means for detecting blinks of the subject's eye. the imaging head further includes a means for imaging an anterior segment of the subject's eye,
The ophthalmologic imaging apparatus according to claim 7 , wherein the blink detection means includes the imaging means. the imaging head further includes a means for imaging an anterior segment of the subject's eye,
The ophthalmologic imaging apparatus according to claim 1 , wherein the start of the movement of the subject's eye is detected based on the image acquired by the imaging means. An ophthalmologic imaging device according to any one of claims 1 to 9, wherein the storage means stores the generated image in association with the detected movement of the subject's eye. An ophthalmologic imaging device according to any one of claims 1 to 10, further comprising a display control means for controlling the display means to repeatedly display a moving image generated using multiple images based on received light signals acquired successively over time. An ophthalmologic imaging device according to any one of claims 1 to 11, wherein the same type of image is a front image of the fundus of the subject's eye or a tomographic image of the fundus. the light receiving optical system generates interference light between the reference light corresponding to the measurement light and the return light,
The ophthalmologic imaging apparatus according to claim 1 , wherein the image generating means generates a tomographic image of the subject's eye using the interference light. The ophthalmic imaging device of claim 13, wherein the scanning optical system scans the measurement light in a direction determined based on the direction of movement of the subject's eye. The ophthalmologic imaging device of claim 11, wherein the display control means controls the display means to repeatedly display the moving image, obtained using at least two of the stored images, in which the influence of movement of the retinal pattern of the subject's eye is reduced. A control method for an ophthalmologic imaging device including an imaging head having a measurement light source that emits measurement light, a scanning optical system that scans the interior of a subject's eye with the measurement light in accordance with predetermined scanning information, and a light receiving optical system that receives return light of the measurement light from the subject's eye and generates a light receiving signal, comprising:
generating an image using the scanning information and the received light signal;
storing a plurality of images of the same type for the same subject's eye under a predetermined motion;
detecting the predetermined movement based on the stored images;
a step of identifying, using at least two of the stored images, an area where an opacity site exists in the vitreous body of the subject's eye, the opacity site exhibiting a movement induced by the predetermined movement and different from the detected predetermined movement; and calculating information about the movement of the identified opacity site using at least two of the stored images;
A method for controlling an ophthalmic imaging apparatus, comprising: A control method for an ophthalmologic imaging device including an imaging head having a measurement light source that emits measurement light, a scanning optical system that scans the interior of a subject's eye with the measurement light in accordance with predetermined scanning information, and a light receiving optical system that receives return light of the measurement light from the subject's eye and generates a light receiving signal, comprising:
generating an image using the scanning information and the received light signal;
a step of storing a plurality of images of the same type regarding the subject's eye, which are the generated images, after an instruction generating means has issued an instruction to the subject to induce a predetermined movement in the subject's eye;
a step of identifying, using at least two of the stored images, an area where an opacity site exists in the vitreous body of the subject's eye, the opacity site exhibiting a movement different from the predetermined movement induced by the predetermined movement, and calculating information about the movement of the opacity site using at least two of the stored images;
A method for controlling an ophthalmic imaging apparatus, comprising: A program that, when executed by a computer, causes the computer to execute each step of the method for controlling an ophthalmic imaging device described in claim 16 or 17.