JP7704369B2 - Ophthalmic information processing device, ophthalmic device, operation method of ophthalmic information processing device, and program - Google Patents

Description

The present invention relates to an ophthalmic information processing device, an ophthalmic device, an operation method of an ophthalmic information processing device, and a program.

In recent years, rapid advances in machine learning techniques, such as deep learning, have led to the practical application of artificial intelligence technology in a variety of fields. In particular, in the medical field, deep learning has improved the accuracy of detecting diseased areas or tissue features in diagnostic images, making it possible to perform accurate and rapid high-precision medical diagnoses.

For example, Patent Document 1 discloses a method of performing machine learning using feature points in multiple past time-series fundus images as training data, and using the resulting trained model to diagnose glaucoma from multiple newly acquired time-series fundus images. For example, Patent Document 2 discloses a method of using a neural network to classify the pathology of glaucoma according to the shape of the optic disc.

International Publication No. 2018/211688 JP 2019-5319 A

For example, ocular diseases such as glaucoma, age-related macular degeneration, and diabetic retinopathy are diseases whose symptoms will progress unless appropriate treatment is administered, even if they are discovered early through screening or other methods. In such cases, by properly understanding the pathology of the disease, it is possible to administer appropriate treatment for the disease.

In this way, there is a need to detect fundus diseases early and with high accuracy, while identifying the pathology of the detected fundus disease with high accuracy. This is true not only for fundus diseases, but also for diseases of the examined eye in general.

The present invention was made in consideration of these circumstances, and its purpose is to provide a new technology for providing appropriate treatment for diseases of the examinee's eye at an early stage.

A first aspect of some embodiments is an ophthalmological information processing device that includes an acquisition unit that acquires multiple images of a test eye having different cross-sectional directions, and a disease estimation unit that outputs estimation information for estimating whether the test eye is a diseased eye from the multiple images using multiple trained models obtained by machine learning for each type of the multiple images.

In a second aspect of some embodiments, in the first aspect, the disease estimation unit includes a plurality of estimators that use each of the plurality of trained models for each type of the plurality of images to output feature values or confidence information representing the confidence that the test eye is a diseased eye, and a classifier that uses a classification model obtained by machine learning to output the estimation information from the plurality of feature values or the plurality of confidence information output from the plurality of estimators.

In a third aspect of some embodiments, in the first or second aspect, the plurality of images include a tomographic image or a frontal image of the fundus of the subject eye, and the disease is a fundus disease.

In a fourth aspect of some embodiments, in the third aspect, the plurality of images includes B-scan images in a cross-sectional direction intersecting a line connecting the optic disc and the fovea.

In a fifth aspect of some embodiments, in the third or fourth aspect, the plurality of images includes a B-scan image in a first cross-sectional direction passing through the optic disc, a B-scan image in a second cross-sectional direction passing through the optic disc and intersecting the first cross-sectional direction, and a B-scan image passing through the fovea.

In a sixth aspect of some embodiments, in any of the third to fifth aspects, the front image includes at least one of an en-face image projected from a layer area corresponding to the internal limiting membrane to a deep layer area, and a projection image.

In a seventh aspect of some embodiments, in any of the third to sixth aspects, the fundus disease includes at least one of glaucoma, age-related macular degeneration, and diabetic reticuloretinopathy.

An eighth aspect of some embodiments, in any one of the first to seventh aspects, includes a pathology estimation unit that uses a trained classification model obtained by supervised machine learning using features obtained from the plurality of trained models to output classification information for estimating a classification result of the pathology of the disease from the plurality of images.

In a ninth aspect of some embodiments, in the eighth aspect, the disease includes glaucoma, and the classification information is information for estimating classification results of multiple types of optic disc shapes.

A tenth aspect of some embodiments is the eighth or ninth aspect, which includes a first learning unit that generates the trained model for classification by supervised machine learning using features obtained from the plurality of trained models.

An eleventh aspect of some embodiments is any of the first to tenth aspects, which includes a second learning unit that generates the plurality of trained models by supervised machine learning for each type of the plurality of images.

A twelfth aspect of some embodiments is any of the first to eleventh aspects, which includes an image generating unit that generates at least one of the plurality of images based on three-dimensional OCT data of the subject's eye.

A thirteenth aspect of some embodiments is an ophthalmic device including an OCT unit that performs optical coherence tomography on the subject's eye, an image generating unit that generates at least one of the plurality of images based on the three-dimensional data acquired by the OCT unit, and an ophthalmic information processing device according to any one of the first to eleventh aspects.

A fourteenth aspect of some embodiments is an ophthalmic device including an OCT unit that performs optical coherence tomography on the subject's eye and an ophthalmic information processing device of the twelfth aspect.

A fifteenth aspect of some embodiments is a method for operating an ophthalmological information processing device, which includes an acquisition step of acquiring a plurality of images of a test eye having different cross-sectional directions, and a disease estimation step of outputting estimation information for estimating whether the test eye is a diseased eye from the plurality of images using a plurality of trained models obtained by machine learning for each type of the plurality of images.

In a sixteenth aspect of some embodiments, in the fifteenth aspect, the disease estimation step includes a plurality of estimation steps for outputting feature values or confidence information representing the confidence that the subject eye is a diseased eye, using each of the plurality of trained models for each type of the plurality of images, and a classification step for outputting the estimation information from the plurality of feature values or the plurality of confidence information output in the plurality of estimation steps, using a classification model obtained by machine learning.

In a seventeenth aspect of some embodiments, in the fifteenth or sixteenth aspects, the plurality of images include a tomographic image or a frontal image of the fundus of the subject eye, and the disease is a fundus disease.

In an eighteenth aspect of some embodiments, in the seventeenth aspect, the plurality of images includes B-scan images in a cross-sectional direction intersecting a line connecting the optic disc and the fovea.

In a 19th aspect of some embodiments, in the 17th or 18th aspects, the plurality of images includes a B-scan image in a first cross-sectional direction passing through the optic disc, a B-scan image in a second cross-sectional direction passing through the optic disc and intersecting the first cross-sectional direction, and a B-scan image passing through the fovea.

In a twentieth aspect of some embodiments, in any of the seventeenth to nineteenth aspects, the front image includes at least one of an en-face image projected from a layer area corresponding to the internal limiting membrane to a deep layer area, and a projection image.

In a twenty-first aspect of some embodiments, in any one of the seventeenth to twentieth aspects, the fundus disease includes at least one of glaucoma, age-related macular degeneration, and diabetic reticuloretinopathy.

A twenty-second aspect of some embodiments, in any one of the fifteenth to twenty-first aspects, includes a pathology estimation step of outputting classification information for estimating a classification result of the pathology of the disease from the plurality of images, using a trained classification model obtained by supervised machine learning using features obtained from the plurality of trained models.

In a twenty-third aspect of some embodiments, in the twenty-second aspect, the disease includes glaucoma, and the classification information is information for estimating classification results of multiple types of optic disc shapes.

A twenty-fourth aspect of some embodiments, in the twenty-second or twenty-third aspect, includes a first learning step of generating the trained model for classification by supervised machine learning using features obtained from the plurality of trained models.

A twenty-fifth aspect of some embodiments, in any one of the fifteenth to twenty-fourth aspects, includes a second learning step of generating the plurality of trained models by supervised machine learning for each type of the plurality of images.

A twenty-sixth aspect of some embodiments, in any one of the fifteenth to twenty-fifth aspects, includes an image generating step of generating at least one of the plurality of images based on three-dimensional OCT data of the subject's eye.

A twenty-seventh aspect of some embodiments is a program for causing a computer to execute each step of the method for operating an ophthalmological information processing device according to any one of the fifteenth to twenty-sixth aspects.

The configurations relating to the above-mentioned aspects can be combined in any way.

Some embodiments of the present invention provide a new technology for providing appropriate treatment for diseases of the examinee's eye at an early stage.

1 is a schematic diagram illustrating an example of a configuration of an ophthalmologic system according to an embodiment. 1 is a schematic diagram illustrating an example of a configuration of an ophthalmologic apparatus according to an embodiment. 1 is a schematic diagram illustrating an example of a configuration of an ophthalmologic information processing apparatus according to an embodiment. 1 is a schematic diagram illustrating an example of a configuration of an ophthalmologic information processing apparatus according to an embodiment. 1 is a schematic diagram illustrating an example of a configuration of an ophthalmologic information processing apparatus according to an embodiment. 1 is a schematic diagram illustrating an example of a configuration of an ophthalmologic information processing apparatus according to an embodiment. 1 is a schematic diagram illustrating an example of a configuration of an ophthalmologic information processing apparatus according to an embodiment. 1 is a schematic diagram illustrating an example of a configuration of an ophthalmologic information processing apparatus according to an embodiment. 4 is a schematic diagram illustrating an example of an operation flow of the ophthalmologic information processing apparatus according to the embodiment. FIG. 4 is a schematic diagram illustrating an example of an operation flow of the ophthalmologic information processing apparatus according to the embodiment. FIG. FIG. 11 is a schematic diagram illustrating an operation of an ophthalmologic information processing apparatus according to a comparative example of the embodiment. FIG. 2 is a schematic diagram illustrating the operation of the ophthalmologic information processing apparatus according to the embodiment. FIG. 13 is a schematic diagram illustrating an example of a configuration of an ophthalmologic apparatus according to a modified example of an embodiment.

Examples of an ophthalmological information processing device, an ophthalmological device, an ophthalmological information processing method, and a program according to some embodiments of the present invention will be described in detail with reference to the drawings. Note that matters described in the documents cited in this specification and any publicly known technology may be incorporated into the embodiments.

The ophthalmic information processing device according to the embodiment is capable of performing a predetermined analysis process and a predetermined display process on three-dimensional OCT data of a test eye acquired by an ophthalmic device using optical coherence tomography (OCT). The ophthalmic device according to some embodiments has not only the function of an OCT device for performing OCT, but also at least one function of a fundus camera, a scanning laser ophthalmoscope, a slit lamp microscope, and a surgical microscope. Furthermore, the ophthalmic device according to some embodiments has a function of measuring the optical characteristics of the test eye. Examples of ophthalmic devices having a function of measuring the optical characteristics of the test eye include a refractometer, a keratometer, a tonometer, a wavefront analyzer, a specular microscope, and a perimeter. The ophthalmic device according to some embodiments has a function of a laser treatment device used in laser treatment.

The ophthalmologic information processing device generates multiple images (tomographic images or frontal images) with different cross-sectional directions from the acquired 3D OCT data of the test eye, and outputs estimation information for estimating whether the test eye is a diseased eye from the generated multiple images. Furthermore, the ophthalmologic information processing device outputs classification information for estimating the classification result of the pathological condition of the disease from the generated multiple images (or their features).

The following describes the case where the ophthalmic information processing device, ophthalmic device, ophthalmic information processing method, and program according to the embodiments are applied to fundus diseases. However, the application field of the ophthalmic information processing device, ophthalmic device, ophthalmic information processing method, and program according to the embodiments is not limited to fundus diseases, and can be applied to all diseases of the subject eye.

Ocular fundus diseases include glaucoma, age-related macular degeneration, and diabetic retinopathy.

When the fundus disease is glaucoma, classification information is output for estimating the classification result based on the shape of the optic disc in order to classify the pathology according to the classification by Nicolela et al. (Nicolella classification) as disclosed in Patent Document 2, for example.

When the fundus disease is age-related macular degeneration, for example, classification information is output for estimating a classification result based on the morphology of the retinal pigment epithelium layer (e.g., the morphology of choroidal neovascularization) to classify the disease as atrophic or exudative.

When the fundus disease is diabetic retinopathy, classification information is output to estimate the classification result based on the morphology and distribution of microaneurysms, hemorrhages, white spots, neovascularization, and hemorrhage areas in order to classify the disease as simple diabetic retinopathy, preproliferative diabetic retinopathy, or proliferative diabetic retinopathy.

In this way, by being able to provide doctors and other medical professionals with classification information regarding the pathological classification of fundus diseases, in addition to estimated information regarding fundus diseases, it becomes possible to determine the appropriate treatment for fundus diseases at an early stage. This makes it possible to increase the chances of curing fundus diseases.

For example, in cases where the ocular fundus disease is determined to be glaucoma, an appropriate treatment can be selected from among drug therapy, laser treatment, surgery, etc. according to the pathology of the disease.

For example, in cases where the ocular fundus disease is determined to be age-related macular degeneration, an appropriate treatment for exudative age-related macular degeneration can be selected from among drug dosage, photodynamic therapy, laser coagulation, surgery, etc. according to the pathology of the disease.

For example, in cases where the ocular fundus disease is determined to be diabetic retinopathy, an appropriate treatment method can be selected from among retinal photocoagulation, vitreous surgery, etc. according to the pathology of the disease.

The ophthalmologic information processing device according to the embodiment is capable of estimating whether or not the subject eye has a fundus disease and classifying the pathology of the fundus disease by using a trained model obtained by machine learning for each of multiple images generated from 3D OCT data.

The ophthalmological system according to the embodiment includes an ophthalmological information processing device. The ophthalmological information processing method according to the embodiment is executed by the ophthalmological information processing device. The program according to the embodiment causes a computer to execute each step of the ophthalmological information processing method.

In the following, glaucoma will be used as an example of an ocular fundus disease, but the following embodiment can also be applied to other ocular fundus diseases.

[Ophthalmology System]
1 is a block diagram showing a configuration example of an ophthalmological system according to an embodiment. The ophthalmological system 1 according to the embodiment includes an ophthalmological apparatus 10, an ophthalmological information processing apparatus (ophthalmological image processing apparatus, ophthalmological analysis apparatus) 100, an operation apparatus 180, and a display apparatus 190.

The ophthalmic device 10 performs OCT on the subject's eye and collects three-dimensional OCT data of the subject's eye. In this embodiment, the ophthalmic device 10 collects fundus OCT data by OCT scanning the fundus of the subject's eye. The ophthalmic device 10 is capable of acquiring images of the fundus from the acquired OCT data of the fundus. The images of the fundus include tomographic images and frontal images of the fundus. The tomographic images of the fundus include B-scan images. The frontal images of the fundus include C-scan images, en-face images, shadowgrams, projection images, and the like. The ophthalmic device 10 transmits the acquired OCT data of the subject's eye or the data of the acquired images to the ophthalmic information processing device 100.

In some embodiments, the ophthalmic device 10 and the ophthalmic information processing device 100 are connected via a data communication network. The ophthalmic information processing device 100 according to some embodiments receives the above data from one of a plurality of ophthalmic devices 10 selectively connected via the data communication network.

The ophthalmological information processing device 100 generates first estimation information for estimating whether or not the subject eye is a glaucomatous eye using an individual estimation model (trained model) obtained by machine learning for each of a plurality of images generated from the 3D OCT data. The ophthalmological information processing device 100 generates second estimation information for estimating whether or not the subject eye is a glaucomatous eye from a plurality of first estimation information generated for each of a plurality of images using a classification model (trained model) obtained by machine learning. Furthermore, the ophthalmological information processing device 100 outputs classification information for estimating a pathology classification result from the above-mentioned plurality of images (or its feature amount) using a pathology estimation model (trained model) obtained by machine learning. The pathology estimation model is obtained by performing machine learning using the feature amount of the above-mentioned plurality of individual estimation models (or a plurality of individual estimation models and classification models). As a result, it is possible to obtain highly accurate classification information using a pathology estimation model obtained by machine learning using less training data compared to the case where a pathology estimation model is generated by performing machine learning from natural images, etc.

Below, a case where five types of images are generated from three-dimensional OCT data will be described. The five types of images are a horizontal B-scan image passing through the center (or its vicinity) of the optic disc, a vertical B-scan image passing through the center (or its vicinity) of the optic disc, a vertical B-scan image passing through the fovea (or its vicinity), a projection image, and an en-face image. However, the configuration according to the embodiment is not limited to the type and number of types of images generated from the three-dimensional OCT data. For example, multiple images including a cross-sectional B-scan image intersecting a line connecting the optic disc and the fovea may be generated from the three-dimensional OCT data, and the above-mentioned estimated information or classification information may be obtained based on the multiple images generated.

In some embodiments, the ophthalmological information processing device 100 constructs the above-mentioned trained model and outputs estimated information or classification information using the constructed trained model. In some embodiments, the ophthalmological information processing device 100 outputs estimated information or classification information using an externally constructed trained model. In some embodiments, the ophthalmological information processing device 100 constructs the above-mentioned trained model and outputs the constructed trained model to an external device.

The operation device 180 and the display device 190 provide functions for exchanging information between the ophthalmologic information processing device 100 and its user, such as displaying information, inputting information, and inputting operation instructions, as a user interface unit. The operation device 180 includes operation devices such as a lever, a button, a key, and a pointing device. In some embodiments, the operation device 180 includes a microphone for inputting information by sound. The display device 190 includes a display device such as a flat panel display. In some embodiments, the functions of the operation device 180 and the display device 190 are realized by a device that integrates a device having an input function such as a touch panel display with a device having a display function. In some embodiments, the operation device 180 and the display device 190 include a graphical user interface (GUI) for inputting and outputting information.

[Ophthalmological equipment]
FIG. 2 is a block diagram showing an example of the configuration of an ophthalmologic apparatus 10 according to an embodiment.

The ophthalmic device 10 is provided with an optical system for acquiring OCT data of the subject's eye. The ophthalmic device 10 has a function for performing swept-source OCT, but the embodiment is not limited thereto. For example, the type of OCT is not limited to swept-source OCT, and may be spectral domain OCT or the like. Swept-source OCT is a method of splitting light from a wavelength-swept (wavelength-scanning) light source into measurement light and reference light, causing the return light of the measurement light passing through the object to be measured to interfere with the reference light to generate interference light, detecting this interference light with a balanced photodiode or the like, and forming an image by applying Fourier transform or the like to the detection data collected in response to the wavelength sweep and the scanning of the measurement light. Spectral domain OCT is a method of splitting light from a low-coherence light source into measurement light and reference light, causing the return light of the measurement light passing through the object to interfere with the reference light to generate interference light, detecting the spectral distribution of the interference light with a spectroscope, and applying Fourier transform or the like to the detected spectral distribution to form an image.

The ophthalmic device 10 includes a control unit 11, an OCT unit 12, an OCT data processing unit 13, and a communication unit 14.

The control unit 11 controls each part of the ophthalmic device 10. In particular, the control unit 11 controls the OCT unit 12, the OCT data processing unit 13, and the communication unit 14.

The OCT unit 12 collects three-dimensional OCT data of the test eye by scanning the test eye using OCT. The OCT unit 12 includes an interference optical system 12A and a scanning optical system 12B.

The interference optical system 12A splits light from a light source (wavelength swept light source) into measurement light and reference light, generates interference light by causing return light of the measurement light that has passed through the test eye to interfere with the reference light that has passed through the reference light path, and detects the generated interference light. The interference optical system 12A includes at least a fiber coupler and a photoreceiver such as a balanced photodiode. The fiber coupler splits light from the light source into measurement light and reference light, and generates interference light by causing return light of the measurement light that has passed through the test eye to interfere with the reference light that has passed through the reference light path. The photoreceiver detects the interference light generated by the fiber coupler. The interference optical system 12A may include a light source.

The scanning optical system 12B is controlled by the control unit 11 and changes the incident position of the measurement light on the fundus of the test eye by deflecting the measurement light generated by the interference optical system 12A. The scanning optical system 12B includes, for example, an optical scanner arranged at a position that is optically approximately conjugate with the pupil of the test eye. The optical scanner includes, for example, a first galvanometer mirror that deflects the measurement light in the horizontal direction, a second galvanometer mirror that deflects the measurement light in the vertical direction, and a mechanism for independently driving these. For example, the second galvanometer mirror is configured to further deflect the measurement light deflected by the first galvanometer mirror. This allows the measurement light to be scanned in any direction on the fundus plane.

The detection result (detection signal) of the interference light by the interference optical system 12A is an interference signal that indicates the spectrum of the interference light.

The OCT data processing unit 13 is controlled by the control unit 11 and forms three-dimensional data (image data) of the fundus based on the data of the subject eye collected by the OCT unit 12.

For example, the OCT data processing unit 13 forms a reflection intensity profile for each A-line, and arranges the formed multiple reflection intensity profiles in the B-scan direction (a direction intersecting the A-scan direction (e.g., z-direction), e.g., x-direction) and a direction intersecting the A-scan direction and B-scan direction (e.g., y-direction) to form three-dimensional OCT data (scan data).

Furthermore, for example, the OCT data processing unit 13 can form an A-scan image of the subject's eye E by imaging the reflection intensity profile in the A-line. The OCT data processing unit 13 can form a three-dimensional image by arranging multiple A-scan images formed for each A-line in the B-scan direction and in a direction intersecting the A-scan direction and the B-scan direction.

The processes performed by the OCT data processing unit 13 include noise removal (noise reduction), filtering, FFT (Fast Fourier Transform), and other processes. The image data acquired in this manner is a data set including a group of image data formed by imaging the reflection intensity profiles in multiple A-lines (paths of each measurement light in the test eye). In order to improve image quality, multiple data sets collected by repeating the scan with the same pattern multiple times can be superimposed (averaged).

In some embodiments, the OCT data processing unit 13 performs various data processing (image processing) and analysis processing on the image. For example, the OCT data processing unit 13 performs correction processing such as image brightness correction and dispersion correction. The OCT data processing unit 13 can form volume data (voxel data) of the subject's eye by performing known image processing such as an interpolation process that interpolates pixels between tomographic images. When an image based on the volume data is to be displayed, the OCT data processing unit 13 performs a rendering process on the volume data to form a pseudo three-dimensional image as viewed from a specific line of sight.

Each of the control unit 11 and the OCT data processing unit 13 includes a processor. The processor includes circuits such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), and a programmable logic device (e.g., an SPLD (Simple Programmable Logic Device), a CPLD (Complex Programmable Logic Device), and an FPGA (Field Programmable Gate Array)). The functions of the OCT data processing unit 13 are realized by the data processing processor. In some embodiments, the functions of both the control unit 11 and the OCT data processing unit 13 are realized by a single processor.

The processor realizes the functions according to the embodiment, for example, by reading and executing a program stored in a memory circuit or a storage device. At least a part of the memory circuit or the storage device may be included in the processor. Also, at least a part of the memory circuit or the storage device may be provided outside the processor.

The storage device etc. stores various types of data. Data stored in the storage device etc. includes data acquired by the OCT unit 12 (measurement data, photographed data, etc.) and information on the subject and the examined eye. The storage device etc. may store various computer programs and data for operating each part of the ophthalmic device 10.

The communication unit 14 is controlled by the control unit 11 and performs communication interface processing to transmit and receive information between the ophthalmological information processing device 100.

In some embodiments, the ophthalmic device 10 transmits data of the subject eye formed by the OCT data processing unit 13 to the ophthalmic information processing device 100.

The ophthalmic device 10 according to some embodiments is provided with a fundus camera, a scanning laser ophthalmoscope, and a slit lamp microscope for acquiring an image of the fundus of the examinee's eye. In some embodiments, the fundus image acquired by the fundus camera is a fluorescein fundus angiography image or a fundus autofluorescence examination image.

[Ophthalmology information processing device]
3 to 6 show block diagrams of configuration examples of the ophthalmological information-processing device 100 according to the embodiment. Fig. 3 shows a functional block diagram of the ophthalmological information-processing device 100. Fig. 4 shows a functional block diagram of the analysis unit 200 in Fig. 3. Fig. 5 shows a functional block diagram of the estimation model construction unit 210 in Fig. 3. Fig. 6 shows a functional block diagram of the estimation unit 220 in Fig. 3.

The ophthalmological information processing device 100 includes a control unit 110, a data processing unit 130, and a communication unit 140. In some embodiments, the ophthalmological information processing device 100 includes an image forming unit having similar functions to the OCT data processing unit 13 of the ophthalmological device 10.

The control unit 110 controls each unit of the ophthalmologic information processing device 100. In particular, the control unit 110 controls the data processing unit 130 and the communication unit 140. The control unit 110 includes a main control unit 111 and a memory unit 112.

The control unit 110 controls each part of the ophthalmologic system 1 based on an operation instruction signal corresponding to the user's operation on the operation device 180.

Each of the control unit 110 and the data processing unit 130 includes a processor. The functions of the data processing unit 130 are realized by a data processor. In some embodiments, the functions of the control unit 110 and the data processing unit 130 are realized by a single processor.

The memory unit 112 stores various types of data. The data stored in the memory unit 112 includes data acquired by the ophthalmic device 10 (measurement data, photographed data, etc.), data processing results by the data processing unit 130, information on the subject and the examined eye, etc. The memory unit 112 may store various computer programs and data for operating each part of the ophthalmic information processing device 100.

The communication unit 140 is controlled by the control unit 110 and performs communication interface processing to transmit or receive information between the communication unit 14 of the ophthalmologic information processing device 100.

The data processing unit 130 can generate an image of any specified cross-sectional direction by performing various renderings on the three-dimensional OCT data from the ophthalmic device 10. In this embodiment, the data processing unit 130 generates a plurality of images with different cross-sectional directions. The plurality of images include a B-scan image, a C-scan image, an en-face image, a projection image, a shadowgram, and the like. An image of an arbitrary cross-section such as a B-scan image or a C-scan image is formed by selecting pixels (pixels, voxels) on a specified cross-section from the three-dimensional OCT data (image data). An en-face image is formed by flattening a portion of the three-dimensional OCT data (image data). A projection image is formed by projecting the three-dimensional OCT data in a specified direction (z direction, depth direction, A-scan direction). A shadowgram is formed by projecting a portion of the three-dimensional OCT data (e.g., partial data corresponding to a specific layer) in a specified direction.

In some embodiments, at least one of the cross-sectional position and the cross-sectional direction is specified by the user operating the operation device 180. The user can specify at least one of the cross-sectional position and the cross-sectional direction using the operation device 180 while referring to a frontal image of the fundus generated from three-dimensional OCT data or a fundus image acquired using a fundus camera (not shown).

In some embodiments, the data processing unit 130 identifies at least one of the cross-sectional position and the cross-sectional direction by analyzing the OCT data. In this case, the data processing unit 130 can identify a characteristic region by a known analysis process and identify at least one of the cross-sectional position and the cross-sectional direction so as to pass through the identified characteristic region. Alternatively, the data processing unit 130 can identify a characteristic region by a known analysis process and identify at least one of the cross-sectional position and the cross-sectional direction so as to avoid the identified characteristic region.

The data processing unit 130 performs a predetermined data processing on the formed image of the subject's eye. Like the OCT data processing unit 13, the data processing unit 130 can perform various data processing (image processing) and analysis processing on the formed image. For example, the data processing unit 130 performs correction processing such as image brightness correction and dispersion correction. The data processing unit 130 can form volume data (voxel data) of the subject's eye by performing known image processing such as interpolation processing that interpolates pixels between tomographic images. When an image based on the volume data is to be displayed, the data processing unit 130 performs rendering processing on the volume data to form a pseudo three-dimensional image as viewed from a specific line of sight.

In addition, instead of the data processing unit 130, the OCT data processing unit 13 in the ophthalmic device 10 may generate the above-mentioned image group. In this case, the ophthalmic information processing device 100 acquires the above-mentioned image group from the ophthalmic device 10.

The data processing unit 130 also generates a disease estimation model for estimating whether the subject eye is a glaucomatous eye from multiple images generated from the 3D OCT data. Furthermore, the data processing unit 130 generates a pathology estimation model for estimating a pathology classification result from the multiple images or the feature amounts of each of the multiple images.

The data processing unit 130 uses the above-mentioned disease estimation model and pathology estimation model to output estimation information for estimating whether the test eye is a glaucomatous eye from multiple images of the test eye, and classification information for estimating the classification result of the pathology of glaucoma.

Such a data processing unit 130 includes an analysis unit 200, an estimation model construction unit 210, and an estimation unit 220.

The analysis unit 200 performs a predetermined analysis process on the fundus image data (or fundus image data acquired by the ophthalmologic device 10). The analysis process includes a process for generating a tomographic image in a desired cross-sectional direction at a desired cross-sectional position. The analysis process according to some embodiments includes a process for identifying a specific site such as the optic disc or fovea.

As shown in FIG. 4, the analysis unit 200 includes an image generation unit 201. The image generation unit 201 generates multiple images with different cross-sectional directions from the three-dimensional OCT data acquired by the ophthalmic device 10. The multiple images include a B-scan image, a C-scan image, an en-face image, a projection image, a shadowgram, etc.

The image generating unit 201 can generate at least one image for which a cross-sectional position and a cross-sectional direction are specified by the user from the three-dimensional OCT data. The image generating unit 201 can also generate at least one image for which a cross-sectional position and a cross-sectional direction are specified so as to pass through a specific site identified in the analysis process in the analysis unit 200. The image generating unit 201 can also generate at least one image for which a cross-sectional position and a cross-sectional direction are specified so as to avoid a specific site identified in the analysis process in the analysis unit 200.

In this embodiment, the image generating unit 201 generates five types of images with different cross-sectional directions from the three-dimensional OCT data of the subject's eye. The five types of images include a horizontal B-scan image passing through the center of the optic disc (or its vicinity), a vertical B-scan image passing through the center of the optic disc (or its vicinity), a vertical B-scan image passing through the fovea (or its vicinity), a projection image, and an en-face image. The projection image is an integrated image of all layers in the depth direction. The en-face image is an integrated image of a predetermined depth (e.g., 50 micrometers) in the depth direction from the top layer of the retina (e.g., the internal limiting membrane). In other words, the en-face image is, for example, an image obtained by projecting a layer region from a layer region corresponding to the internal limiting membrane to a deep layer region.

As described above, the estimation model construction unit 210 constructs a disease estimation model (the estimation model and classification model described above) for estimating whether the subject eye is a glaucomatous eye, and a pathology estimation model for estimating the pathology classification result.

As shown in FIG. 5, the estimation model construction unit 210 includes a disease estimation learning unit 211 and a pathology estimation learning unit 212. As shown in FIG. 6, the estimation unit 220 includes a disease estimation unit 221 and a pathology estimation unit 222.

Figure 7 shows a schematic diagram for explaining the operation of the ophthalmological information processing device 100 according to the embodiment. Figure 7 shows a schematic relationship between the estimation model construction unit 210 and the estimation unit 220. Figure 7 also shows the relationship between the disease estimation unit 221 and pathology estimation unit 222 and the disease estimation learning unit 211 and pathology estimation learning unit 212. In Figure 7, parts similar to those in Figure 5 or Figure 6 are given the same reference numerals, and descriptions will be omitted as appropriate.

The disease estimation learning unit 211 generates multiple individual estimation models for estimating whether or not the subject eye is a glaucomatous eye for each of the multiple images generated from the 3D OCT data. The disease estimation learning unit 211 generates multiple individual estimation models by performing supervised machine learning for each of the multiple images. In some embodiments, the disease estimation learning unit 211 generates multiple individual estimation models by transfer learning.

When five types of images are generated from 3D OCT data as described above, the disease estimation learning unit 211 generates an individual estimation model for each type of image.

For example, for a horizontal B-scan image IMG1 passing through the center of the optic disc (or its vicinity), the disease estimation learning unit 211 generates an individual estimation model for the B-scan image IMG1 by performing supervised machine learning using the B-scan images IMG1 of multiple test eyes other than the test eye that is the subject of estimation as training data TR1 and labels representing the judgment results of a doctor or the like as to whether or not each B-scan image has glaucoma as teacher data SD1. The function of the individual estimation model for the B-scan image IMG1 is realized by the first estimator 301 shown in FIG. 7.

For example, for a vertical B-scan image IMG2 passing through the center of the optic disc (or its vicinity), the disease estimation learning unit 211 uses the B-scan images IMG2 of multiple test eyes other than the test eye that is the subject of estimation as training data TR1, and generates an individual estimation model for the B-scan image IMG2 by performing supervised machine learning using labels representing the judgment results of a doctor or the like as to whether or not each B-scan image has glaucoma as teacher data SD1. The function of the individual estimation model for the B-scan image IMG2 is realized by the second estimator 302 shown in FIG. 7.

For example, for a vertical B-scan image IMG3 passing through the fovea (or its vicinity), the disease estimation learning unit 211 uses the B-scan images IMG3 of multiple test eyes other than the test eye that is the subject of estimation as training data TR1, and generates an individual estimation model for the B-scan image IMG3 by performing supervised machine learning using labels representing the judgment results of a doctor or the like as to whether or not each B-scan image has glaucoma as teacher data SD1. The function of the individual estimation model for the B-scan image IMG3 is realized by the third estimator 303 shown in FIG. 7.

For example, for the projection image IMG4, the disease estimation learning unit 211 generates an individual estimation model for the projection image IMG4 by performing supervised machine learning using the projection images IMG4 of multiple test eyes other than the test eye that is the estimation target as training data TR1 and labels representing the judgment results of a doctor or the like as to whether or not each projection image has glaucoma as teacher data SD1. The function of the individual estimation model for the projection image IMG4 is realized by the fourth estimator 304 shown in FIG. 7.

For example, for an en-face image IMG5, the disease estimation learning unit 211 generates an individual estimation model for the en-face image IMG5 by performing supervised machine learning using en-face images IMG5 of multiple test eyes other than the test eye that is the estimation target as training data TR1 and labels representing the judgment results of a doctor or the like as to whether or not each en-face image has glaucoma as teacher data SD1. The function of the individual estimation model for the en-face image IMG5 is realized by the fifth estimator 305 shown in FIG. 7.

The disease estimation learning unit 211 also generates a classification model for estimating whether the subject eye is a glaucomatous eye or not from multiple pieces of estimation information output from the multiple individual estimation models generated. The disease estimation learning unit 211 generates the classification model by performing unsupervised machine learning or supervised machine learning.

For example, the disease estimation learning unit 211 generates a classification model by performing unsupervised machine learning or supervised machine learning using a group of images IMG1 to IMG5 of multiple test eyes other than the test eye to be estimated as training data TR1. When performing supervised machine learning, a label indicating whether the test eye for training is a glaucomatous eye or not is used as the teacher data SR1. The function of this classification model is realized by the classifier 306 shown in FIG. 7.

A disease estimation model is composed of multiple individual estimation models and a classification model for images IMG1 to IMG5. That is, the estimation unit 220 uses the disease estimation model generated by the disease estimation learning unit 211 to output estimation information for estimating whether or not the test eye is glaucomatous from multiple images of the test eye (multiple images generated from OCT data of the test eye). In some embodiments, the estimation information includes information indicating whether or not the test eye is glaucomatous. In some embodiments, the estimation information includes information (certainty information) indicating the degree of certainty that the test eye is glaucomatous (e.g., the probability that it is estimated to be glaucoma).

Each of the multiple individual estimation models may have a similar configuration.

Figure 8 shows a block diagram of an example configuration of the first estimator 301 according to the embodiment. Each of the second estimator 302 to the fifth estimator 305 may have a configuration similar to that shown in Figure 8.

The function of the first estimator 301 is realized, for example, by a convolutional neural network (CNN). That is, the computer operates in accordance with instructions from an individual estimation model (trained model) stored in memory to perform calculations based on trained weighting coefficients and response functions in the convolutional neural network for pixel values of image IMG1 input to the convolutional layer 311 of the feature extractor 310, which is the input layer, and to output a judgment result from the classifier 320, which is the output layer, in accordance with instructions from an individual estimation model (trained model) stored in memory in the computer. The first estimator 301 having such a configuration can extract local correlation patterns while gradually reducing the image resolution, and output a judgment result based on the extracted correlation pattern.

The first estimator 301 includes a feature extractor 310 and a classifier 320. The feature extractor 310 extracts features of the judgment image by repeating feature extraction and downsampling (filtering) for each predetermined image region of the input image IMG1. The classifier 320 outputs estimation information (e.g., confidence level) for estimating whether the subject eye is a glaucomatous eye based on the features extracted by the feature extractor 310.

The feature extractor 310 includes multiple units in which units including a convolution layer and a pooling layer are connected in multiple stages. In each unit, the input of the pooling layer is connected to the output of the convolution layer. The pixel value of the corresponding pixel in the image IMG1 is input to the input of the first stage convolution layer. The input of the subsequent stage convolution layer is connected to the output of the previous stage pooling layer.

In FIG. 8, the feature extractor 310 includes two units connected in two stages. That is, in the feature extractor 310, a unit including a convolution layer 311 and a pooling layer 312 is connected to a unit including a convolution layer 313 and a pooling layer 314 in the latter stage. The output of the pooling layer 312 is connected to the input of the convolution layer 313.

The classifier 320 includes fully connected layers 321 and 322, and the output of the fully connected layer 321 is connected to the input of the fully connected layer 322.

In the feature extractor 310 and the classifier 320, learned weighting coefficients are assigned between the neurons in the two connected layers. Each neuron performs a calculation using a response function on the calculation result that takes into account the weighting coefficients from one or more input neurons, and outputs the obtained calculation result to the neuron in the next stage.

The weighting coefficients are updated by performing known machine learning using two or more previously acquired images IMG1 of the subject's eye (horizontal B-scan images passing through the center of the optic disc (or its vicinity)) as training data and labels assigned to each image by a doctor or the like as teacher data. Existing weighting coefficients are updated by machine learning using two or more previously acquired images as training data. In some embodiments, the weighting coefficients are updated by transfer learning.

The first estimator 301 may have a known layer structure such as VGG16, VGG19, InceptionV3, ResNet18, or ResNet50. The classifier 320 may have a known configuration such as Random Forest or a Support Vector Machine (SVM).

The classifier 306 in the disease estimation unit 221 is generated by performing known machine learning using, for example, two or more images IMG1 to IMG5 of the test eye acquired in the past as training data and labels attached to each image by a doctor or the like as teacher data. The classifier 306 outputs information OUT indicating whether the test eye is a glaucomatous eye or certainty information OUT indicating the certainty that the test eye is a glaucomatous eye based on multiple pieces of estimated information from the first estimator 301 to the fifth estimator 305. In some embodiments, the classifier 306 outputs information OUT indicating whether the test eye is a glaucomatous eye or certainty information OUT indicating the certainty that the test eye is a glaucomatous eye based on multiple feature amounts extracted from each of the predetermined layers of the first estimator 301 to the fifth estimator 305. The classifier 306 may have a known configuration such as Random Forest or a support vector machine, similar to the classifier 320 shown in FIG. 8.

The pathology estimation learning unit 212 generates a pathology estimation model for estimating a classification result of glaucoma from a plurality of images generated by the image generation unit 201. The pathology estimation learning unit 212 generates the pathology estimation model by performing supervised machine learning. In some embodiments, when the subject eye is determined to be a glaucomatous eye from the estimation information output from the classification model, the pathology estimation learning unit 212 generates classification information for estimating a classification result of the pathology of glaucoma.

For example, five types of images IMG1 to IMG5 generated by the image generation unit 201 are used as training data TR2, and labels representing the doctor's or other person's judgment of whether or not the patient has glaucoma are used as teacher data SD2 to perform supervised machine learning to generate a pathology classification estimation model 401.

At this time, the disease estimation learning unit 212 generates a pathology classification estimation model 401 by machine learning using the features (weighting coefficients, etc.) of the individual estimation models for images IMG1 to IMG5 obtained by the disease estimation learning unit 211. In some embodiments, the pathology estimation learning unit 212 generates a pathology classification estimation model 401 by machine learning using the features (weighting coefficients, etc.) of the individual estimation models and classification models for images IMG1 to IMG5.

The pathology estimation unit 222 uses the pathology estimation learning model generated by the pathology estimation learning unit 212 to output classification information OUTa to OUTd for classifying the pathology of glaucoma from five types of images IMG1 to IMG5 of the test eye. In some embodiments, the classification information includes information representing one of the multiple types of pathology of glaucoma. In some embodiments, the classification information includes information representing the certainty of each of the multiple types of pathology of glaucoma (e.g., the probability of being estimated as each type). For example, the classification information OUTa is certainty information representing the certainty that the glaucoma developed in the test eye is focal ischemic type (FI). The classification information OUTb is certainty information representing the certainty that the glaucoma developed in the test eye is myopic type (MY). The classification information OUTc is certainty information representing the certainty that the glaucoma developed in the test eye is age-related sclerosis type (SS). The classification information OUTd is certainty information that indicates the certainty that the glaucoma developed in the test eye is generalized expanding (GE).

In this embodiment, the pathology estimation unit 222 outputs classification information according to the Nicollella classification.

The Nicolela classification defines four types (focal ischemic, myopic, age-related sclerosis, and global enlargement) according to the shape of the optic disc. The focal ischemic type (Focal Ischemia: FI) is a type in which a notch is present in part of the rim and a localized defect in the optic nerve fiber layer is seen. The focal ischemic type is more common in women and is often accompanied by migraines and seizures. The myopic type (MY) is a type in which the optic disc is inclined and crescent-shaped peripapillary chorioretinal atrophy (PPA) with a depression on the temporal side is seen. The myopic type is more common in young people and is often accompanied by myopia. The senior sclerotic type (SS) is a type in which the optic disc is circular, the depression is shallow, and a halo is seen around the optic disc. Age-related sclerosis is common in elderly people and is often accompanied by cardiovascular disorders. Generalized enlargement (GE) is a type that presents a large, deep circular depression. Generalized enlargement is common in subjects with high intraocular pressure.

When the subject eye is a glaucomatous eye, the pathology estimation unit 222 can output classification information indicating the type classified as the glaucoma pathology that has developed in the subject eye from among the multiple types of the Nicollera classification. In addition, the pathology estimation unit 222 can output classification information indicating the degree of certainty of each of the multiple types of the Nicollera classification.

The ophthalmic device 10 or the communication unit 140 that performs the receiving process of the OCT data from the ophthalmic device 10 is an example of an "acquisition unit" according to the embodiment. As the five types of images, a horizontal B-scan image passing through the center (or its vicinity) of the optic disc, a vertical B-scan image passing through the center (or its vicinity) of the optic disc, a vertical B-scan image passing through the fovea (or its vicinity), a projection image, and an en-face image are examples of "multiple images" according to the embodiment. The individual estimation model for the B-scan image IMG1, the individual estimation model for the B-scan image IMG2, the individual estimation model for the B-scan image IMG3, the individual estimation model for the projection image IMG4, and the individual estimation model for the en-face image IMG5 are examples of "multiple trained models" according to the embodiment. The B-scan image is an example of a "tomographic image" according to the embodiment. The projection image or the en-face image is an example of a "front image" according to the embodiment. The disease estimation learning unit 211 is an example of a "second learning unit" according to the embodiment. The pathology estimation learning unit 212 is an example of a "first learning unit" according to the embodiment. The horizontal direction is an example of a "first cross-sectional direction" according to the embodiment. The vertical direction is an example of a "second cross-sectional direction" according to the embodiment.

[Example of operation]
An operation example of the ophthalmological information-processing apparatus 100 according to some embodiments will be described.

Figures 9 and 10 show flow diagrams of an example of the operation of the ophthalmology information processing device 100 according to the embodiment. Figure 9 shows a flow diagram of an example of a generation process of a trained model for disease estimation and pathological condition estimation. Figure 10 shows a flow diagram of an example of a disease estimation process and pathological condition estimation process using a trained model generated by the generation process shown in Figure 9. The memory unit 112 stores a computer program for realizing the processes shown in Figures 9 and 10. The control unit 110 (main control unit 111) is able to execute the processes shown in Figures 9 and 10 by operating in accordance with this computer program.

In FIG. 9, it is assumed that OCT has been performed in advance on the fundus of multiple test eyes in the ophthalmic device 10, and multiple sets of 3D OCT data have been acquired.

(S1: Create a training data set from 3D OCT data)
First, the main controller 111 controls the communication unit 140 to acquire a plurality of pieces of three-dimensional OCT data for a plurality of examinee's eyes acquired by the ophthalmologic apparatus 10 .

Then, the main controller 111 controls the image generator 201 to generate the above five types of images as a training data group from each of the multiple 3D OCT data. That is, the image generator 201 generates a horizontal B-scan image passing through the center of the optic disc (or its vicinity), a vertical B-scan image passing through the center of the optic disc (or its vicinity), a vertical B-scan image passing through the fovea (or its vicinity), a projection image, and an en-face image from the 3D OCT data. A doctor or the like interprets the generated images and attaches a label indicating whether the subject eye is a glaucomatous eye. The main controller 111 associates the interpreted images with the labels and stores them in the memory unit 112.

(S2: Generate a disease estimation model)
Next, the main controller 111 controls the disease estimation learning unit 211 to generate an individual estimation model (disease estimation model) for one of the above five types of images. The disease estimation learning unit 211 generates the individual estimation model by performing supervised machine learning as described above.

(S3: Next?)
Next, the main control unit 111 determines whether or not to perform the generation process of an individual estimation model for the next image. For example, the main control unit 111 determines whether or not to perform the generation process of an individual estimation model for the next image by counting the number of predetermined image types or the number of image types generated in step S1.

When it is determined that the generation process of an individual estimation model is to be performed for the next image (step S3: Y), the operation of the ophthalmological information-processing device 100 proceeds to step S2. When it is not determined that the generation process of an individual estimation model is to be performed for the next image (step S3: N), the operation of the ophthalmological information-processing device 100 proceeds to step S4.

(S4: Generate a pathology estimation model)
In step S3, when it is not determined that the process of generating an individual estimation model is to be performed for the next image (step S3: N), the main controller 111 controls the pathology estimation learning unit 212 to generate a pathology classification estimation model 401 for estimating the classification result of the pathology of glaucoma based on the above five types of images. The pathology estimation learning unit 212 generates the pathology classification estimation model 401 as described above.

This completes the operation of the ophthalmologic information processing device 100 (END).

As shown in FIG. 10, the ophthalmological information processing device 100 uses a trained model generated according to the flow shown in FIG. 9 to generate estimation information for estimating whether the subject eye is a glaucomatous eye, and classification information for estimating the classification result of the pathology of glaucoma.

In FIG. 10, it is assumed that OCT has been performed in advance on the fundus of the test eye, which is the estimation target, in the ophthalmic device 10, and three-dimensional OCT data has been acquired.

(S11: Creating an image group from 3D OCT data)
First, the main controller 111 controls the communication unit 140 to acquire three-dimensional OCT data of the subject's eye, which is the estimation target, acquired by the ophthalmologic apparatus 10 .

Then, the main controller 111 controls the image generator 201 to generate the above five types of images as an image group from the 3D OCT data. That is, the image generator 201 generates, from the 3D OCT data, a horizontal B-scan image passing through the center of the optic disc (or its vicinity), a vertical B-scan image passing through the center of the optic disc (or its vicinity), a vertical B-scan image passing through the fovea (or its vicinity), a projection image, and an en-face image.

(S12: Disease estimation, pathological condition estimation)
Next, the main control unit 111 controls the disease estimation unit 221 to output estimation information for estimating whether or not the subject eye is a glaucomatous eye, using a disease estimation model (individual estimation model for each image) generated by repeating steps S2 to S3 of Figure 9 for the five types of image groups generated in step S11.

The main control unit 111 also controls the pathology estimation unit 222 to output classification information for estimating the classification results of the pathology of glaucoma for the five types of image groups generated in step S11 using the pathology estimation model generated in step S4 of FIG. 9.

In some embodiments, the main control unit 111 operates the disease estimation unit 221 and the pathological condition estimation unit 222 in parallel to output the estimation information and the classification information simultaneously.

This completes the operation of the ophthalmologic information processing device 100 (END).

Here, to evaluate the accuracy of glaucoma diagnosis by the ophthalmologic information processing device 100 according to the embodiment, a comparison with a comparative example of the embodiment will be made. The ophthalmologic information processing device according to the comparative example of the embodiment includes a five-input convolutional neural network that uses five types of images similar to those of the embodiment as input data, and is configured to determine whether or not the subject eye has glaucoma based on five types of images of the fundus of the subject eye by performing known supervised machine learning.

Figure 11 shows an example of the area under the receiver operating characteristic curve (AUROC) obtained by an ophthalmologic information processing device according to a comparative example of an embodiment.

In Figure 11, the results of evaluation using 143 cases of OCT data from normal eyes and 672 cases of OCT data from glaucoma eyes in a 5-fold cross-validation are shown as AUROC. In Figure 11, the AUC (Area Under Curve) is 0.950 ± 0.023.

Figure 12 shows an example of AUROC obtained by the ophthalmological information processing device 100 according to the embodiment.

In Figure 12, the results of evaluation using 5-fold cross-validation using the same OCT data as in the comparative example (143 normal eyes, 672 glaucoma eyes) are shown as AUROC. In Figure 12, the AUC is 0.983 ± 0.009.

As described above, the configuration according to the embodiment makes it possible to improve the accuracy of the determination compared to a case where a convolutional neural network is simply used to determine whether or not a patient has glaucoma.

<Modification>
The configuration according to the embodiment is not limited to the above configuration.

In some embodiments, the ophthalmic device has at least one of the functions of an ophthalmic information processing device 100, the functions of an operation device 180, and the functions of a display device 190 in addition to the functions of the ophthalmic device 10.

Below, we will explain some of the variations of the ophthalmic device according to the embodiment, focusing on the differences from the ophthalmic device according to the embodiment described above.

Figure 13 shows a block diagram of an example of the configuration of an ophthalmic device 10a according to a modified embodiment. In Figure 13, parts similar to those in Figure 2 are given the same reference numerals, and descriptions thereof will be omitted as appropriate.

The configuration of the ophthalmic device 10a according to this modification differs from the configuration of the ophthalmic device 10 according to the above embodiment in that the ophthalmic device 10a has the functions of an ophthalmic information processing device 100, an operation device 180, and a display device 190. The ophthalmic device 10a includes a control unit 11a, an OCT unit 12, an OCT data processing unit 13, an ophthalmic information processing unit 15a, an operation unit 16a, and a display unit 17a.

The control unit 11a controls each part of the ophthalmic device 10a. In particular, the control unit 11a controls the OCT unit 12, the OCT data processing unit 13, the ophthalmic information processing unit 15a, the operation unit 16a, and the display unit 17a.

The ophthalmological information processing unit 15a has a configuration similar to that of the ophthalmological information processing device 100, and has the same functions as the ophthalmological information processing device 100. The operation unit 16a has a configuration similar to that of the operation device 180, and has the same functions as the operation device 180. The display unit 17a has a configuration similar to that of the display device 190, and has the same functions as the display device 190.

According to this modified example, it is possible to obtain, with a compact configuration, estimation information for estimating whether or not the subject eye is a glaucoma eye (diseased eye) and classification information for estimating the classification result of the pathology of glaucoma with high accuracy by machine learning using less training data. This makes it possible to provide a new technology for providing appropriate treatment for the disease of the subject eye at an early stage.

＜Effects＞
Hereinafter, an ophthalmological information processing apparatus, an ophthalmological apparatus, an ophthalmological information processing method, and a program according to some embodiments will be described.

An ophthalmological information processing device (ophthalmological information processing device 100, ophthalmological information processing unit 15a) according to some embodiments includes an acquisition unit (ophthalmological device 10, or a communication unit 140 that performs reception processing of OCT data from the ophthalmological device 10) and a disease estimation unit (221). The acquisition unit acquires multiple images of the subject eye having different cross-sectional directions. The disease estimation unit uses multiple trained models obtained by machine learning for each type of the multiple images to output estimation information for estimating whether the subject eye is a diseased eye from the multiple images.

With this configuration, for each of the images in multiple cross-sectional directions that are different from one another, a trained model obtained by machine learning is used to obtain inferred information for estimating whether the test eye is a diseased eye, making it possible to detect diseases early and with high accuracy. This makes it possible to provide appropriate treatment for the disease in the test eye early.

In some embodiments, the disease estimation unit includes a plurality of estimators (first estimator 301 to fifth estimator 305) that use a plurality of trained models for each of a plurality of image types to output feature quantities or confidence information representing the confidence that the test eye is a diseased eye, and a classifier (306) that uses a classification model obtained by machine learning to output estimation information from the plurality of feature quantities or the plurality of confidence information output from the plurality of estimators.

According to this configuration, a trained model obtained by machine learning is used to output feature values or confidence information for each type of multiple images, and a classification model obtained by machine learning is used to output estimated information from multiple feature values or multiple confidence information, making it possible to detect diseases early and with high accuracy. This makes it possible to provide appropriate treatment for diseases of the examined eye early.

In some embodiments, the plurality of images includes a tomographic image or a frontal image of the fundus of the test eye, and the disease is a fundus disease.

This configuration allows for early detection of fundus diseases with high accuracy. This allows for early and appropriate treatment of fundus diseases in the examined eye.

In some embodiments, the plurality of images includes cross-sectional B-scan images intersecting a line connecting the optic disc and the fovea.

With this configuration, estimated information is obtained using multiple images, including B-scan images in a cross-sectional direction that intersects with a line connecting the optic disc and the fovea, making it possible to detect diseases that affect the morphology of the cross section that intersects with a line connecting the optic disc and the fovea with high accuracy and at an early stage.

In some embodiments, the multiple images include a B-scan image in a first cross-sectional direction (horizontal direction) passing through the optic disc, a B-scan image in a second cross-sectional direction (vertical direction) passing through the optic disc and intersecting the first cross-sectional direction, and a B-scan image passing through the fovea.

This configuration allows for early detection of diseases that affect the cross-sectional morphology of the optic disc or fovea with high accuracy.

In some embodiments, the front image includes at least one of an en-face image projecting a layer region corresponding to the internal limiting membrane to a deep layer region, and a projection image.

This configuration allows for early detection of diseases that affect the frontal morphology of the subject's eye with high accuracy.

In some embodiments, the ocular disease includes at least one of glaucoma, age-related macular degeneration, and diabetic reticuloretinopathy.

This configuration allows for early detection of glaucoma and other diseases with high accuracy. This allows for early and appropriate treatment of the disease in the examined eye.

Some embodiments include a pathology estimation unit (222) that uses a trained classification model (pathology classification estimation model 401) obtained by supervised machine learning using features obtained from multiple trained models to output classification information for estimating a classification result of the pathology of a disease from multiple images.

With this configuration, it becomes possible to estimate the classification results of the pathology of a disease with high accuracy. This makes it possible to provide appropriate treatment for the disease of the subject's eye at an early stage. In addition, it becomes possible to obtain highly accurate classification information through machine learning using less training data compared to the case of using a trained model obtained by performing machine learning from natural images, etc.

In some embodiments, the disease includes glaucoma, and the classification information is information for estimating classification results for multiple types of optic disc shapes.

This configuration makes it possible to estimate the classification results of the pathology of glaucoma with high accuracy. This makes it possible to provide appropriate treatment for glaucoma that has developed in the subject's eye at an early stage.

Some embodiments include a first learning unit (pathology estimation learning unit 212) that generates a trained model for classification by supervised machine learning using features obtained from multiple trained models.

With this configuration, it is possible to provide an ophthalmologic information processing device that can generate a trained classification model for estimating with high accuracy the classification result of the pathology of a disease developed in the subject's eye through machine learning using less training data compared to using a trained model obtained by performing machine learning on natural images, etc.

Some embodiments include a second learning unit (disease estimation learning unit 211) that generates multiple trained models by supervised machine learning for each of multiple image types.

This configuration makes it possible to provide an ophthalmologic information processing device that can generate a trained model for detecting diseases in the subject's eye with high accuracy and at an early stage.

Some embodiments include an image generating unit (201) that generates at least one of a plurality of images based on three-dimensional OCT data of the test eye.

With this configuration, it is possible to provide an ophthalmologic information processing device that can acquire 3D OCT data of the subject's eye and detect diseases of the subject's eye with high accuracy and at an early stage.

An ophthalmic device (10, 10a) according to some embodiments includes an OCT unit (12) that performs optical coherence tomography on the subject's eye, an image generating unit (201) that generates at least one of a plurality of images based on three-dimensional data acquired by the OCT unit, and an ophthalmic information processing device (100, ophthalmic information processing unit 15a) described in any one of the above.

This configuration makes it possible to provide an ophthalmic device that can perform optical coherence tomography on the subject's eye and detect diseases of the subject's eye at an early stage with high accuracy from the obtained OCT data.

The ophthalmic apparatus (10, 10a) according to some embodiments includes an OCT section (12) that performs optical coherence tomography on the subject's eye, and the ophthalmic information processing device (100, ophthalmic information processing section 15a) described above.

This configuration makes it possible to provide an ophthalmic device that can perform optical coherence tomography on the subject's eye and detect diseases of the subject's eye at an early stage with high accuracy from the obtained OCT data.

An ophthalmologic information processing method according to some embodiments includes an acquisition step of acquiring multiple images of a test eye having different cross-sectional directions, and a disease estimation step of outputting estimation information for estimating whether the test eye is a diseased eye from the multiple images using multiple trained models obtained by machine learning for each type of the multiple images.

According to this method, for each of images in a plurality of different cross-sectional directions, a trained model obtained by machine learning is used to obtain inferred information for estimating whether or not the subject eye is a diseased eye, so that diseases can be detected early and with high accuracy. This makes it possible to provide appropriate treatment for the disease of the subject eye early.

In some embodiments, the method includes multiple estimation steps of outputting feature values or confidence information representing the confidence that the test eye is a diseased eye using multiple trained models for each of multiple image types, and a classification step of outputting estimation information from the multiple feature values or the multiple confidence information output in the multiple estimation steps using a classification model obtained by machine learning.

According to this method, a trained model obtained by machine learning is used to output feature values or confidence information for each type of multiple images, and a classification model obtained by machine learning is used to output estimated information from multiple feature values or multiple confidence information, making it possible to detect diseases early and with high accuracy. This makes it possible to provide appropriate treatment for diseases of the examined eye early.

In some embodiments, the plurality of images includes a tomographic image or a frontal image of the fundus of the test eye, and the disease is a fundus disease.

This method allows for highly accurate and early detection of fundus diseases. This makes it possible to provide appropriate treatment for fundus diseases of the examined eye at an early stage.

In some embodiments, the plurality of images includes cross-sectional B-scan images intersecting a line connecting the optic disc and the fovea.

According to this method, estimated information is obtained using multiple images, including B-scan images in a cross-sectional direction that intersects with a line connecting the optic disc and the fovea, making it possible to detect diseases that affect the morphology of the cross section that intersects with a line connecting the optic disc and the fovea with high accuracy and at an early stage.

In some embodiments, the multiple images include a B-scan image in a first cross-sectional direction passing through the optic disc, a B-scan image in a second cross-sectional direction passing through the optic disc and intersecting the first cross-sectional direction, and a B-scan image passing through the fovea.

This method allows for early detection of diseases that affect the cross-sectional morphology of the optic disc or fovea with high accuracy.

In some embodiments, the front image includes at least one of an en-face image projecting a layer region corresponding to the internal limiting membrane to a deep layer region, and a projection image.

This method allows for early detection of diseases that affect the frontal morphology of the subject's eye with high accuracy.

In some embodiments, the ocular disease includes at least one of glaucoma, age-related macular degeneration, and diabetic reticuloretinopathy.

This method allows for early detection of glaucoma and other diseases with high accuracy. This allows for early and appropriate treatment of the disease in the examined eye.

Some embodiments include a pathology estimation step of outputting classification information for estimating a classification result of a pathology of a disease from a plurality of images, using a trained classification model (pathology classification estimation model 401) obtained by supervised machine learning using features obtained from a plurality of trained models.

This method makes it possible to estimate the classification results of the pathology of a disease with high accuracy. This makes it possible to provide appropriate treatment for the disease of the subject's eye at an early stage. Furthermore, compared to using a trained model obtained by performing machine learning from natural images, etc., it becomes possible to obtain highly accurate classification information through machine learning using less training data.

In some embodiments, the disease includes glaucoma, and the classification information is information for estimating classification results for multiple types of optic disc shapes.

This method makes it possible to estimate the classification results of the pathology of glaucoma with high accuracy. This makes it possible to provide appropriate treatment for glaucoma that has developed in the subject's eye at an early stage.

Some embodiments include a first learning step in which features obtained from multiple trained models are used to generate a trained model for classification through supervised machine learning.

According to this method, it is possible to provide an ophthalmologic information processing method that can generate a trained classification model for estimating with high accuracy the classification result of the pathology of a disease developed in a subject's eye by machine learning using less training data compared to using a trained model obtained by performing machine learning on natural images, etc.

Some embodiments include a second learning step in which multiple trained models are generated using supervised machine learning for each of the multiple image types.

This method makes it possible to provide an ophthalmologic information processing method capable of generating a trained model for detecting diseases in the subject's eye with high accuracy and at an early stage.

Some embodiments include an image generation step that generates at least one of a plurality of images based on three-dimensional OCT data of the test eye.

According to this method, it is possible to provide an ophthalmologic information processing method that can obtain three-dimensional OCT data of the test eye, thereby enabling early detection of diseases in the test eye with high accuracy.

A program according to some embodiments causes a computer to execute each step of any of the ophthalmologic information processing methods described above.

According to such a program, for each of a plurality of images in different cross-sectional directions, a trained model obtained by machine learning is used to obtain inferred information for estimating whether or not the subject eye is a diseased eye, making it possible to detect diseases with high accuracy and at an early stage. This makes it possible to provide appropriate treatment for the disease of the subject eye at an early stage.

A program for implementing the ophthalmologic information processing method according to some embodiments can be stored in any non-transitory recording medium that is readable by a computer. The recording medium may be an electronic medium that uses magnetism, light, magneto-optical, semiconductors, or the like. Typically, the recording medium is a magnetic tape, a magnetic disk, an optical disk, a magneto-optical disk, a flash memory, a solid-state drive, or the like.

It is also possible to send and receive computer programs via networks such as the Internet or LAN.

The above-described embodiments are merely examples for implementing the present invention. Anyone who wishes to implement the present invention may make any modifications (omissions, substitutions, additions, etc.) within the scope of the gist of the present invention.

1 Ophthalmology system 10, 10a Ophthalmology device 11, 11a, 110 Control unit 12 OCT unit 12A Interference optical system 12B Scanning optical system 14, 140 Communication unit 15a Ophthalmology information processing unit 16a Operation unit 17a Display unit 100 Ophthalmology information processing device 111 Main control unit 112 Memory unit 130 Data processing unit 180 Operation device 190 Display device 200 Analysis unit 201 Image generation unit 210 Estimation model construction unit 211 Disease estimation learning unit 212 Pathological condition estimation learning unit 220 Estimation unit 221 Disease estimation unit 222 Pathological condition estimation unit

Claims (19)

an acquisition unit that acquires a plurality of images of a subject's eye having cross-sectional directions different from each other;
a plurality of estimators that output, for each type of image, a feature amount or certainty information representing a certainty that the test eye is an eye suffering from age-related macular degeneration, using each of a plurality of trained models obtained by machine learning for each type of image;
a classifier that uses a classification model obtained by machine learning to output inference information for inferring whether the subject's eye is an eye suffering from age-related macular degeneration from the plurality of feature amounts or the plurality of pieces of confidence information output from the plurality of estimators; and
An ophthalmological information processing device comprising: The ophthalmological information processing device according to claim 1, further comprising a pathology estimation unit that uses a trained model for classification obtained by supervised machine learning using features obtained from the plurality of trained models to output classification information for estimating a classification result based on the morphology of the retinal pigment epithelium layer from the plurality of images. an acquisition unit that acquires a plurality of images of a subject's eye having cross-sectional directions different from each other;
a plurality of estimators that output, for each type of image, a feature amount or certainty information representing a certainty that the test eye is an eye with diabetic retinopathy, using each of a plurality of trained models obtained by machine learning for each type of image;
a classifier that uses a classification model obtained by machine learning to output inference information for inferring whether the subject's eye is an eye with diabetic retinopathy from the plurality of feature amounts or the plurality of pieces of confidence information output from the plurality of estimators; and
An ophthalmological information processing device comprising: the pathology estimation unit outputs classification information for estimating a classification result based on at least one of the morphology or distribution of microaneurysm, hemorrhage, white spot, neovascularization, and hemorrhage area from the plurality of images using a trained model for classification obtained by supervised machine learning using feature amounts obtained from the plurality of trained models. The ophthalmologic information processing apparatus according to claim 2 or claim 4 , further comprising a first learning unit configured to generate the trained model for classification by supervised machine learning using feature amounts obtained from the plurality of trained models. The plurality of images include a tomographic image or a front image of the fundus of the subject eye.
6. The ophthalmological information processing apparatus according to claim 1, The ophthalmological information processing device according to any one of claims 1 to 6, further comprising a second learning unit configured to generate the plurality of trained models by supervised machine learning for each type of the plurality of images. The ophthalmologic information processing apparatus according to any one of claims 1 to 7, further comprising an image generating unit configured to generate at least one of the plurality of images based on three-dimensional OCT data of the subject's eye. an OCT unit that performs optical coherence tomography on the subject's eye;
an image generating unit that generates at least one of the plurality of images based on the three-dimensional data acquired by the OCT unit;
An ophthalmological information processing device according to any one of claims 1 to 7 ,
13. An ophthalmic device comprising: an OCT unit that performs optical coherence tomography on the subject's eye;
The ophthalmological information processing apparatus according to claim 8 ;
13. An ophthalmic device comprising: An acquisition step in which a processor acquires a plurality of images of the subject's eye having cross-sectional directions different from each other;
a plurality of estimation steps in which the processor outputs, for each type of image, a feature amount or certainty information representing a certainty that the subject eye is an eye suffering from age-related macular degeneration, using each of a plurality of trained models obtained by machine learning for each type of image;
a classification step in which the processor uses a classification model obtained by machine learning to output inference information for inferring whether or not the subject eye is an eye suffering from age-related macular degeneration from the plurality of feature amounts or the plurality of pieces of confidence information output in the plurality of estimation steps;
A method for operating an ophthalmic information processing device, comprising: The method for operating an ophthalmologic information device according to claim 11, further comprising a pathology estimation step in which the processor uses a trained model for classification obtained by supervised machine learning using features obtained from the plurality of trained models to output classification information for estimating a classification result based on the morphology of the retinal pigment epithelium from the plurality of images. An acquisition step in which a processor acquires a plurality of images of the subject's eye having cross-sectional directions different from each other;
A plurality of estimation steps in which the processor outputs, for each type of image, a feature amount or certainty information representing a certainty that the test eye is an eye with diabetic retinopathy, using each of a plurality of trained models obtained by machine learning for each type of image;
a classification step in which the processor uses a classification model obtained by machine learning to output inference information for inferring whether or not the subject eye is an eye with diabetic retinopathy from the plurality of feature amounts or the plurality of pieces of confidence information output in the plurality of estimation steps; and
A method for operating an ophthalmic information processing device, comprising: The method for operating the ophthalmologic information processing device according to claim 13, further comprising a pathology estimation step of outputting classification information for estimating a classification result based on at least one of the morphology or distribution of microaneurysm, hemorrhage, white spot, neovascularization, and hemorrhage area from the plurality of images, using a trained model for classification obtained by supervised machine learning using features obtained from the plurality of trained models. The method for operating an ophthalmological information processing device described in claim 12 or 14, characterized in that the processor includes a first learning step in which the processor generates the classification trained model by supervised machine learning using features obtained by the multiple trained models. The method for operating an ophthalmologic information processing device according to claim 15, further comprising a second learning step in which the processor generates the plurality of trained models by supervised machine learning for each type of the plurality of images. The plurality of images include a tomographic image or a front image of the fundus of the subject eye.
A method for operating an ophthalmologic information processing apparatus according to any one of claims 11 to 16. The method for operating an ophthalmologic information processing device described in any one of claims 11 to 17, characterized in that the processor includes an image generation step of generating at least one of the plurality of images based on three-dimensional OCT data of the test eye. A program that causes a computer to execute each step of the method for operating an ophthalmologic information processing device according to any one of claims 11 to 18.

Images