JP7706057B2 - Ophthalmic image processing device, ophthalmic image processing program, and ophthalmic image capturing device - Google Patents

Description

The present disclosure relates to an ophthalmic image processing device used to process ophthalmic images of a test eye, an ophthalmic image processing program, and an ophthalmic image capturing device that captures ophthalmic images of a test eye.

In recent years, technology has been proposed to obtain medical data by analyzing ophthalmic images of a subject's eye. For example, the fundus image processing device described in Patent Document 1 obtains detection results of arteries and veins present in at least a part of the fundus image by inputting the fundus image into a mathematical model trained by a machine learning algorithm. In addition, technology has been proposed to obtain analysis results regarding the boundaries of each layer of tissue shown in the ophthalmic image based on the ophthalmic image.

JP 2019-208851 A

For example, when ophthalmic images that are inappropriate for analysis are excluded from the analysis, it is useful to be able to grasp the suitability of an ophthalmic image as an analysis target (hereinafter, sometimes simply referred to as "analysis suitability"). Here, ophthalmic images with poor image quality are often inappropriate for analysis. Therefore, it is possible to use values indicating the image quality of an ophthalmic image (e.g., S/N ratio and contrast, etc.) as the analysis suitability. However, various factors other than image quality can affect the suitability of an ophthalmic image as an analysis target. For example, even if the image quality is good overall, there are ophthalmic images that are inappropriate for analysis due to the presence of a specific disease, the inclusion of unnecessary objects (e.g., eyelids, etc.), or poor shooting range. Conversely, there are also ophthalmic images that are appropriate for analysis because the range required for analysis is properly captured even if the image quality of some parts is poor due to the inclusion of eyelashes, etc.

It is also possible to determine the analysis suitability of an ophthalmic image by using a mathematical model that has been trained on an image basis using both ophthalmic images that are suitable for analysis and ophthalmic images that are not suitable for analysis. However, as described above, various factors affect the analysis suitability of an ophthalmic image. Therefore, when training a mathematical model that determines the analysis suitability, it is necessary to prepare a variety of ophthalmic images with different factors that reduce the analysis suitability as ophthalmic images with low analysis suitability, which is not realistic. In the first place, it is difficult to define the analysis suitability, so learning the analysis suitability itself is difficult. As described above, it was difficult to properly determine the analysis suitability of an ophthalmic image using conventional technology.

A typical objective of the present disclosure is to provide an ophthalmic image processing device, an ophthalmic image processing program, and an ophthalmic image capturing device that are capable of appropriately determining the suitability of an ophthalmic image as a subject for analysis.

An ophthalmic image processing device provided by a typical embodiment of the present disclosure is an ophthalmic image processing device that processes ophthalmic images, which are images of tissues of a test eye, and a control unit of the ophthalmic image processing device executes an image acquisition step of acquiring an ophthalmic image captured by an ophthalmic image capturing device, an image input step of inputting the ophthalmic image into a mathematical model that is trained by a machine learning algorithm and performs an analysis of at least one specific structure and disease of the test eye captured in the input ophthalmic image, thereby outputting a probability distribution in which each of a plurality of classes for detecting at least one of the specific structure and disease captured in the ophthalmic image is a random variable, a certainty information acquisition step of acquiring certainty information indicating the degree of certainty of the analysis performed by the mathematical model on the input ophthalmic image, based on the probability distribution output by the mathematical model, and a determination step of determining the suitability of the ophthalmic image as an analysis target based on the acquired certainty information.

An ophthalmic image processing program provided by a typical embodiment of the present disclosure is an ophthalmic image processing program executed by an ophthalmic image processing device that processes ophthalmic images, which are images of tissues of a test eye, and the ophthalmic image processing program is executed by a control unit of the ophthalmic image processing device to cause the ophthalmic image processing device to execute the following steps: an image acquisition step of acquiring an ophthalmic image captured by an ophthalmic image capturing device; an image input step of inputting the ophthalmic image into a mathematical model that is trained by a machine learning algorithm and performs an analysis of at least one specific structure and disease of the test eye captured in the input ophthalmic image, thereby outputting a probability distribution in which each of a plurality of classes for detecting at least one of the specific structure and disease captured in the ophthalmic image is a random variable; a certainty information acquisition step of acquiring certainty information indicating the degree of certainty of the analysis performed by the mathematical model on the input ophthalmic image, based on the probability distribution output by the mathematical model; and a determination step of determining the suitability of the ophthalmic image as a subject for analysis based on the acquired certainty information.

An ophthalmic image capturing device provided by a typical embodiment of the present disclosure includes an ophthalmic image capturing unit that captures an ophthalmic image, which is an image of the tissue of a test eye, and a control unit that controls the operation of the device, and the control unit executes the following steps: an image capturing step of capturing an ophthalmic image using the ophthalmic image capturing unit; an image input step of inputting the ophthalmic image into a mathematical model that is trained by a machine learning algorithm and performs an analysis of at least one of a specific structure and a disease of the test eye captured in the input ophthalmic image, thereby outputting a probability distribution in which each of a plurality of classes for detecting at least one of the specific structure and the disease captured in the ophthalmic image is a random variable; a certainty information acquisition step of acquiring certainty information indicating the degree of certainty of the analysis performed by the mathematical model on the input ophthalmic image, based on the probability distribution output by the mathematical model; and a determination step of determining the suitability of the ophthalmic image as an analysis target based on the acquired certainty information.

The ophthalmic image processing device, ophthalmic image processing program, and ophthalmic image capturing device according to the present disclosure can appropriately determine the suitability of an ophthalmic image as a subject for analysis.

1 is a block diagram showing a schematic configuration of a mathematical model construction device 1, an ophthalmic image processing device 21, and ophthalmic image capturing devices 11A and 11B. 1A and 1B are diagrams showing an example of an ophthalmological image 30 and blood vessel images 40A and 40B showing blood vessels included in the ophthalmological image 30. 2 is a flowchart of a mathematical model construction process executed by the mathematical model construction device 1. 11 is a graph showing the relationship between confidence information and frequency when a mathematical model analysis is performed on a plurality of ophthalmologic images. FIG. 5 is a diagram showing a list of ophthalmologic images that are ranked within the bottom 15 in terms of certainty in the analysis shown in FIG. 4 . 13 is a flowchart of an analysis appropriateness determination process executed by the ophthalmologic image-processing device 21. 4 is a flowchart of a first photographing control process executed by the ophthalmologic image photographing device 11. 13 is a flowchart of a second photographing control process executed by the ophthalmologic image photographing device 11. 13 is a flowchart of a third photographing control process executed by the ophthalmologic image photographing device 11.

＜Overview＞
The control unit of the ophthalmic image processing device exemplified in the present disclosure executes an image acquisition step, an image input step, a confidence information acquisition step, and a determination step. In the image acquisition step, the control unit acquires an ophthalmic image captured by an ophthalmic image capturing device. In the image input step, the control unit inputs the ophthalmic image to a mathematical model. The mathematical model is trained by a machine learning algorithm. When the ophthalmic image is input to the mathematical model, the mathematical model performs an analysis of at least one of a specific structure and a disease of the subject's eye shown in the input ophthalmic image. In the confidence information acquisition step, the control unit acquires confidence information indicating the confidence of the analysis performed by the mathematical model on the input ophthalmic image. In the determination step, the control unit judges the suitability of the ophthalmic image input to the mathematical model as an analysis target (hereinafter, sometimes simply referred to as "analysis suitability") based on the acquired confidence information.

In a mathematical model trained with multiple ophthalmic images, when an ophthalmic image similar to the ophthalmic image used for training is input, the confidence level of the analysis of the ophthalmic image tends to be high. On the other hand, when an ophthalmic image not similar to the ophthalmic image used for training is input to the mathematical model, the confidence level of the analysis of the ophthalmic image tends to be low. Here, when constructing a mathematical model that analyzes ophthalmic images, ophthalmic images with high analysis suitability are likely to be used to train the mathematical model. Therefore, when an ophthalmic image with high analysis suitability is input to the mathematical model as the analysis target, the confidence level of the analysis is likely to be high. In other words, the correlation between the analysis suitability and confidence level of each ophthalmic image is high.

Based on the above findings, the control unit of the ophthalmic image processing device exemplified in this disclosure determines the analysis suitability of the ophthalmic image based on the confidence information of the analysis performed on the ophthalmic image by the mathematical model. As a result, the analysis suitability, which is influenced by various factors such as image quality and shooting range, is appropriately determined.

The "certainty" may be the degree of certainty of the analysis of the ophthalmic image by the mathematical model, or the inverse of the low certainty (which can also be expressed as uncertainty). For example, if the uncertainty is expressed as x%, the certainty may be a value expressed as (100-x)%. In other words, the suitability of the analysis of the ophthalmic image is appropriately determined not only when the value of the "certainty" indicating the high certainty of the analysis is used, but also when the low certainty (uncertainty) of the analysis is used. Therefore, the term "based on the certainty information" in this disclosure includes not only when the high certainty is used, but also when the low certainty (uncertainty) is used. Note that the "certainty" indicates the degree of certainty of the prediction made using the mathematical model. The certainty is not necessarily proportional to the accuracy of the analysis result.

The confidence level may include the entropy (average information content) of the probability distribution of the output in the automatic analysis by the mathematical model. Entropy indicates the spread (variation) of the probability distribution. When the confidence level in the automatic analysis is at its maximum value, the entropy of the probability distribution is 0. Moreover, the lower the confidence level, the higher the entropy. Therefore, by using the entropy of the probability distribution as the confidence level, the analysis suitability is appropriately determined. Moreover, a value other than the entropy may be adopted as the confidence level. For example, at least one of the standard deviation, the coefficient of variation, the variance, etc., which indicate the degree of dispersion of the probability distribution in the automatic analysis, may be used as the confidence level. The KL divergence, which is a measure of the difference between probability distributions, may be used as the confidence level. The maximum value of the probability distribution may be used as the confidence level. Moreover, when ranking multiple structures, etc. by automatic analysis, the probability of the first place, or the difference between the probability of the first place and the probability of other places (for example, the second place, or the sum of multiple probabilities below the second place, etc.) may be used as the confidence level. In addition, the confidence level may be the variation in the output between multiple mathematical models that use different data or conditions for learning. In this case, the output of the mathematical model may not be a probability distribution.

The specific form of the certainty information acquired in the certainty information acquisition step can be selected as appropriate. For example, the suitability of the analysis may be determined based on information indicating the distribution of certainty (hereinafter sometimes referred to as a "certainty map") of the analysis performed by a mathematical model for each part (e.g., each pixel, etc.) within the image area of the ophthalmic image.

The devices that execute the image acquisition step, image input step, confidence information acquisition step, and judgment step can be selected as appropriate. For example, a control unit of a personal computer (hereinafter referred to as "PC") may execute all of the image acquisition step, image input step, confidence information acquisition step, and judgment step. In other words, the control unit of the PC may acquire an ophthalmic image from an ophthalmic imaging device and judge the analysis suitability of the acquired ophthalmic image. Also, a control unit of the ophthalmic imaging device may execute all of the image acquisition step, image input step, confidence information acquisition step, and judgment step. Also, the control units of multiple devices (e.g., ophthalmic imaging devices and PCs, etc.) may cooperate to execute the image acquisition step, image input step, confidence information acquisition step, and judgment step.

The mathematical model may output a probability distribution for identifying tissue in the ophthalmic image. The confidence information may be obtained based on the probability distribution output by the mathematical model. In this case, the mathematical model for identifying tissue is used to appropriately determine the suitability of the analysis of the ophthalmic image.

The specific content of the analysis that the mathematical model performs on the ophthalmic image can also be selected as appropriate. For example, the mathematical model may output the analysis results of the fundus vascular tissue (the analysis results of the arteries, the analysis results of the veins, or the analysis results of both the arteries and the veins) from the fundus image of the test eye. The mathematical model may output the analysis results of tissues other than the fundus blood vessels (e.g., the optic disc, etc.) from the fundus image of the test eye. The mathematical model may also output the analysis results of at least one of the tissue layers and layer boundaries from the tomographic image of the tissue (e.g., the fundus, etc.) of the test eye. The mathematical model may output the analysis results of the corneal endothelial cells of the test eye.

However, the analysis performed by the mathematical model does not have to be an analysis of tissue. For example, an automatic analysis result indicating the presence or absence of a disease in the subject's eye may be output by the mathematical model. In addition, the probability of the presence of each disease may be output as an analysis result.

In addition, multiple analyses may be performed using a mathematical model. For example, the control unit may use confidence information of a second analysis (e.g., an analysis of the fundus blood vessels) performed by the mathematical model to determine the accuracy of a first analysis (e.g., an analysis of the optic disc) using the mathematical model. In this case, the mathematical model that performs the first analysis and the mathematical model that performs the second analysis may be the same or different.

The analysis performed by the mathematical model to which the ophthalmic image is input in the image input step is defined as the first analysis. The control unit may further perform an analysis step of performing a second analysis on the ophthalmic image in accordance with the determination result of the analysis suitability after the first analysis. In other words, the second analysis may be performed in accordance with the determination result of the analysis suitability, separate from the first analysis performed by the mathematical model before the analysis suitability is determined. In this case, the suitability of the second analysis as an analysis target is determined based on the confidence information regarding the first analysis, and then the second analysis is performed in accordance with the determination result. This makes it easier to perform the second analysis appropriately.

The suitability of the analysis determined in the determination step may be the suitability as a subject of a second analysis using a mathematical model, or it may be the suitability as a subject of a second analysis performed without using a mathematical model (e.g., interpretation by a doctor). In other words, when a second analysis is performed, a mathematical model does not have to be used in the second analysis.

The tissue that is the subject of the first analysis and the tissue that is the subject of the second analysis may be the same. In this case, the analysis suitability in the second analysis is determined by performing the first analysis on the same tissue as the subject tissue of the second analysis. Therefore, the analysis suitability is determined with higher accuracy than when the subject of the first analysis and the subject of the second analysis are different.

The contents of the first analysis and the second analysis may be different. For example, the first analysis may be an analysis of fundus vascular tissue, and the second analysis may be an analysis of the subject's condition (e.g., at least one of the degree of arteriosclerosis, blood pressure, and age). Also, the contents of the first analysis and the second analysis may be the same.

In the determination step, the control unit may exclude the ophthalmic image from the analysis target (e.g., the target of the second analysis described above) if the certainty information about the ophthalmic image input to the mathematical model does not satisfy the condition. In this case, the ophthalmic image that is inappropriate as the analysis target is automatically excluded from the analysis target. Thus, the analysis of the ophthalmic image is performed more appropriately.

The specific method for determining whether the certainty information of the ophthalmic image satisfies the condition can be selected as appropriate. For example, the control unit may refer to a certainty map acquired for the ophthalmic image and determine whether the average value or cumulative value of the certainty in the image area is equal to or greater than a threshold value, thereby determining whether the certainty information of the ophthalmic image satisfies the condition (i.e., whether the analysis suitability is good or not). In this case, the threshold value may be set to a value that can appropriately distinguish whether the analysis of the ophthalmic image (e.g., the second analysis described above) is performed appropriately or not.

The control unit may also determine the analysis suitability of the ophthalmic image based on the certainty information in a part of the attention area of the entire image area of the ophthalmic image. In this case, the accuracy of the determination of the analysis suitability is improved by referring to only the certainty information in the attention area necessary for the analysis out of the entire image area. For example, when the mathematical model identifies a specific tissue, the attention area may be an area detected as a specific tissue (i.e., an area detected as having a higher probability of being a specific tissue than a non-specific tissue). As an example, when the mathematical model detects fundus blood vessels, the attention area may be an area consisting of pixels where "(artery probability + vein probability) > background probability". Areas other than the specific tissue (e.g., background, etc.) are more easily analyzed by the mathematical model (i.e., with high certainty) than the specific tissue. Therefore, the correlation between the analysis suitability of the ophthalmic image and the average certainty is further increased by calculating the average certainty in the area with the area unlikely to be a specific tissue excluded. Therefore, the analysis suitability is determined with higher accuracy.

The control unit may also set the attention area to any area within the image area. The control unit may separately determine the analysis suitability of the ophthalmic image based on the confidence information of each of the multiple attention areas in the entire image area. In this case, the analysis suitability for each area is determined separately, so that a more appropriate analysis result can be obtained. For example, the control unit may perform a second analysis on an area with a high analysis suitability and omit the second analysis on an area with a low analysis suitability. Note that at least one of the position, size, and shape of the attention area may be determined according to an instruction input by the user. However, it goes without saying that the confidence information of the entire image area of the ophthalmic image may be referenced.

However, it is also possible to change the way in which the results of the judgment of the analysis suitability are used. For example, the control unit may acquire multiple ophthalmic images in the image acquisition step. In the judgment step, the control unit may select, from among the multiple ophthalmic images, one or more ophthalmic images whose certainty information satisfies a condition as targets for analysis.

The control unit may also notify the user of the analysis suitability of the ophthalmic image determined in the determination step. In this case, the user can make various appropriate decisions after understanding the analysis suitability of the ophthalmic image.

The control unit may also determine whether to adopt the results of the analysis performed by the mathematical model when acquiring the certainty information (e.g., the results of the first analysis described above) based on the certainty information. In other words, the control unit may determine whether to adopt the results of the analysis performed by the mathematical model depending on whether the certainty information about the ophthalmic image input to the mathematical model satisfies a condition. In this case, the analysis step of performing the second analysis described above may be omitted.

The control unit of the ophthalmic image capturing device exemplified in this disclosure executes an image capturing step, an image input step, a confidence level information acquisition step, and a determination step. In this case, similar to the ophthalmic image processing device described above, the analysis suitability of the ophthalmic image is appropriately determined in the capturing device.

It goes without saying that the ophthalmic image capturing device may also adopt various features described for the ophthalmic image processing device (e.g., the fact that confidence information is obtained based on a probability distribution for identifying tissues, etc.).

The control unit may execute at least one of a notification step and a re-imaging step. In the notification step, the result of the judgment of the analysis suitability in the judgment step is notified. In the re-imaging step, if the analysis suitability judged in the judgment step does not satisfy the condition (or if the certainty information about the ophthalmic image does not satisfy the condition), the same test eye is re-imaged. In this case, the possibility that an ophthalmic image suitable for analysis will be captured by the ophthalmic image capturing device is appropriately improved. Note that, as described above, various methods can be selected as a method for judging whether the analysis suitability satisfies the condition (i.e., whether the certainty information satisfies the condition).

The control unit may repeatedly execute the photographing of a provisional ophthalmic image in the image photographing step, the image input step, the confidence level information acquisition step, and the judgment step, and when the analysis suitability of the provisional ophthalmic image judged in the judgment step satisfies a condition (or when the confidence level information about the ophthalmic image satisfies a condition), the formal ophthalmic image may be photographed by the ophthalmic image photographing unit. In this case, the ophthalmic image is appropriately photographed by the ophthalmic image photographing unit at the time when the condition for photographing an ophthalmic image suitable for analysis is met.

The provisional ophthalmological image and the formal ophthalmological image may be different types of images. For example, the provisional ophthalmological image may be observation images captured continuously by an infrared camera (i.e., images captured in a manner that makes the subject's eye less susceptible to glare), and the formal ophthalmological image may be an image captured using visible light (e.g., a color fundus image captured by a fundus camera). Furthermore, the provisional ophthalmological image and the formal ophthalmological image may be the same type of image.

The control unit may further execute an averaging step in which the same part of the same examinee's eye is continuously photographed in the image photographing step, and the photographed ophthalmic images are averaged to obtain an averaged image. In the image input step, the confidence information acquisition step, and the determination step, the analysis suitability of the averaged image obtained in the averaging step may be determined. The control unit may repeat photographing in the image photographing step and averaging processing in the averaging step until the analysis suitability of the averaged image determined in the determination step satisfies a condition (or until the confidence information about the averaged image satisfies a condition). In this case, photographing of ophthalmic images and averaging processing are repeated until the averaged image becomes an image suitable for analysis. Thus, an averaged image suitable for analysis is appropriately obtained (photographed).

<Embodiment>
(Device configuration)
A typical embodiment of the present disclosure will be described below with reference to the drawings. As shown in FIG. 1, in this embodiment, a mathematical model construction device 1, an ophthalmic image processing device 21, and ophthalmic image capturing devices 11A and 11B are used. The mathematical model construction device 1 constructs a mathematical model by training the mathematical model using a machine learning algorithm. The constructed mathematical model performs an analysis of at least one of a specific structure and a disease of the subject's eye based on an input ophthalmic image. The ophthalmic image processing device 21 obtains an analysis result using the mathematical model, and determines the suitability (analysis suitability) of the ophthalmic image as an analysis target based on the confidence information of the analysis performed by the mathematical model. The ophthalmic image capturing devices 11A and 11B capture ophthalmic images, which are images of the tissues of the subject's eye.

As an example, a personal computer (hereinafter referred to as "PC") is used as the mathematical model construction device 1 of this embodiment. Although details will be described later, the mathematical model construction device 1 trains a mathematical model using data of an ophthalmic image of the subject's eye (hereinafter referred to as "training ophthalmic image") acquired from the ophthalmic image capturing device 11A and data indicating at least one of the structure and disease of the subject's eye from which the training ophthalmic image was captured. As a result, a mathematical model is constructed. However, the device that can function as the mathematical model construction device 1 is not limited to a PC. For example, the ophthalmic image capturing device 11A may function as the mathematical model construction device 1. In addition, the control units of multiple devices (for example, the CPU of the PC and the CPU 13A of the ophthalmic image capturing device 11A) may cooperate to construct a mathematical model.

In addition, a PC is used as the ophthalmic image processing device 21 of this embodiment. However, the device that can function as the ophthalmic image processing device 21 is not limited to a PC. For example, the ophthalmic image capturing device 11B or a server may function as the ophthalmic image processing device 21. When the ophthalmic image capturing device 11B functions as the ophthalmic image processing device 21, the ophthalmic image capturing device 11B can determine the analysis suitability of the captured ophthalmic image while capturing an ophthalmic image. The ophthalmic image capturing device 11B can also perform various processes such as re-capturing based on the determination result of the analysis suitability (these details will be described later). In addition, a mobile terminal such as a tablet terminal or a smartphone may function as the ophthalmic image processing device 21. The control units of multiple devices (for example, the CPU of the PC and the CPU 13B of the ophthalmic image capturing device 11B) may cooperate to perform various processes.

In addition, in this embodiment, a CPU is used as an example of a controller that performs various processes. However, it goes without saying that controllers other than a CPU may be used for at least some of the various devices. For example, a GPU may be used as a controller to speed up processing.

The mathematical model construction device 1 will now be described. The mathematical model construction device 1 is placed, for example, in an ophthalmic image processing device 21 or in a manufacturer that provides an ophthalmic image processing program to a user. The mathematical model construction device 1 includes a control unit 2 that performs various control processes, and a communication I/F 5. The control unit 2 includes a CPU 3 that is a controller that manages control, and a storage device 4 that can store programs, data, and the like. The storage device 4 stores a mathematical model construction program for executing the mathematical model construction process (see FIG. 3) described below. In addition, the communication I/F 5 connects the mathematical model construction device 1 to other devices (for example, an ophthalmic image capturing device 11A and an ophthalmic image processing device 21, etc.).

The mathematical model construction device 1 is connected to an operation unit 7 and a display device 8. The operation unit 7 is operated by a user to input various instructions to the mathematical model construction device 1. For example, at least one of a keyboard, a mouse, a touch panel, etc. can be used for the operation unit 7. Note that a microphone or the like for inputting various instructions may be used together with or instead of the operation unit 7. The display device 8 displays various images. For the display device 8, various devices capable of displaying images (for example, at least one of a monitor, a display, a projector, etc.) can be used. Note that "image" in this disclosure includes both still images and moving images.

The mathematical model construction device 1 can acquire ophthalmic image data (hereinafter, sometimes simply referred to as "ophthalmic image") from the ophthalmic image capture device 11A. The mathematical model construction device 1 may acquire ophthalmic image data from the ophthalmic image capture device 11A by at least one of wired communication, wireless communication, a removable storage medium (e.g., a USB memory), etc.

The ophthalmic image processing device 21 will be described. The ophthalmic image processing device 21 is disposed, for example, in a facility (for example, a hospital or a medical examination facility) where a diagnosis or examination is performed on a subject. The ophthalmic image processing device 21 includes a control unit 22 that performs various control processes, and a communication I/F 25. The control unit 22 includes a CPU 23 that is a controller that manages control, and a storage device 24 that can store programs, data, and the like. The storage device 24 stores an ophthalmic image processing program for executing an analysis suitability determination process (see FIG. 6) described later. The ophthalmic image processing program includes a program for realizing the mathematical model constructed by the mathematical model construction device 1. The communication I/F 25 connects the ophthalmic image processing device 21 to other devices (for example, the ophthalmic image capturing device 11B and the mathematical model construction device 1, etc.).

The ophthalmologic image processing device 21 is connected to an operation unit 27 and a display device 28. As with the operation unit 7 and display device 8 described above, various devices can be used for the operation unit 27 and the display device 28.

The ophthalmic image processing device 21 can acquire ophthalmic images from the ophthalmic image capturing device 11B. The ophthalmic image processing device 21 may acquire ophthalmic images from the ophthalmic image capturing device 11B, for example, by at least one of wired communication, wireless communication, a removable storage medium (e.g., a USB memory), etc. In addition, the ophthalmic image processing device 21 may acquire a program or the like that realizes the mathematical model constructed by the mathematical model construction device 1 via communication, etc.

The ophthalmic image capturing device 11 (11A, 11B) will be described. As an example, in this embodiment, a case will be described in which an ophthalmic image capturing device 11A that provides ophthalmic images to the mathematical model construction device 1 and an ophthalmic image capturing device 11B that provides ophthalmic images to the ophthalmic image processing device 21 are used. However, the number of ophthalmic image capturing devices used is not limited to two. For example, the mathematical model construction device 1 and the ophthalmic image processing device 21 may acquire ophthalmic images from multiple ophthalmic image capturing devices. Furthermore, the mathematical model construction device 1 and the ophthalmic image processing device 21 may acquire ophthalmic images from a single common ophthalmic image capturing device.

The ophthalmic imaging device 11 (11A, 11B) will be described. Various devices that capture images of the tissue of the subject's eye can be used for the ophthalmic imaging device 11. As an example, the ophthalmic imaging device 11 used in this embodiment is a fundus camera that can capture a two-dimensional color front image of the fundus using visible light. However, devices other than a fundus camera (e.g., at least one of an OCT device, a laser scanning ophthalmoscope (SLO), a corneal endothelial cell imaging device, etc.) may also be used. The ophthalmic image may be a two-dimensional front image of the tissue of the subject's eye captured from the front side of the subject's eye, or a three-dimensional image of the tissue.

The ophthalmic image capturing device 11 includes a control unit 12 (12A, 12B) that performs various control processes, and an ophthalmic image capturing section 16 (16A, 16B). The control unit 12 includes a CPU 13 (13A, 13B) that is a controller that controls the control, and a storage device 14 (14A, 14B) that can store programs and data. The ophthalmic image capturing control program executed by the CPU 13 may be stored in the storage device 14. As described above, the ophthalmic image capturing device 11 can also determine the analysis suitability of the ophthalmic image. In this case, the ophthalmic image capturing control program for the ophthalmic image capturing device 11 to execute the capture control process (see Figures 7 to 9) includes a program that realizes the mathematical model constructed by the mathematical model construction device 1. The ophthalmic image capturing control program may be stored in the storage device 14. The ophthalmic image capturing section 16 includes optical members and the like for capturing ophthalmic images of the subject's eye.

(Mathematical model construction process)
2 and 3, a description will be given of a mathematical model construction process executed by the mathematical model construction device 1. The mathematical model construction process is executed by the CPU 3 in accordance with a mathematical model construction program stored in the storage device 4.

In the mathematical model construction process, a mathematical model is constructed that performs an analysis of at least one of a specific structure and a disease of the subject's eye shown in an ophthalmic image. As an example, this embodiment illustrates a mathematical model that analyzes an input ophthalmic image and outputs an analysis result of the fundus blood vessels, which is one of the structures of the subject's eye (in this embodiment, an image of the blood vessels in the input fundus). In other words, the mathematical model of this embodiment detects a specific structure of the subject's eye (fundus blood vessels) shown in the input ophthalmic image. However, as described above, the mathematical model may output an analysis result (e.g., tissue layers and boundaries) different from the analysis result of the fundus blood vessels.

In this embodiment, a convolutional neural network is used for the mathematical model. The mathematical model in this embodiment is trained to output a probability distribution for each pixel constituting an ophthalmic image, with the class (artery pixel, vein pixel, or other pixel) as a random variable. Arteries and veins are detected by determining the class that has the maximum value for each pixel in the output of the mathematical model (in this embodiment, an arterial blood vessel image and a venous blood vessel image are obtained). Note that arterial and venous blood vessel images may be created by extracting the probability corresponding to each artery and vein for each pixel.

As will be described in detail later, certainty information of the analysis performed by the mathematical model is used when determining the suitability of the analysis of an ophthalmic image. In this embodiment, the entropy is calculated for each pixel for the output of the mathematical model, and a certainty map is obtained that indicates the distribution of certainty or uncertainty for each pixel. The average value in a specific region of the certainty map is used as the certainty information for the ophthalmic image input to the mathematical model.

In the mathematical model construction process, a mathematical model is constructed by training the mathematical model using a training data set. The training data set includes input data (input training data) and output data (output training data). As an example, the following describes a case where a two-dimensional color frontal image captured by a fundus camera is input as an input image to the mathematical model, and a blood vessel image is output from the mathematical model.

2 shows an example of input training data and output training data when a vascular image is output to a mathematical model using a two-dimensional color front image as an input image. In the example shown in FIG. 2, an ophthalmic image 30, which is a two-dimensional color front image captured by an ophthalmic image capturing device (fundus camera in this embodiment) 11A, is used as the input training data. In this embodiment, the image area of the ophthalmic image 30 used as the input training data includes both the optic disc 31 and the macula 32 of the subject's eye. In addition, vascular images 40A and 40B, which are images showing at least one of an artery and a vein in the ophthalmic image 30 used as the input training data, are used as the output training data. The output training data may be generated, for example, according to an instruction input by an operator. In this case, the operator may input instructions while referring to the fundus blood vessels shown in the ophthalmic image 30 used as the input training data.

In the example shown in FIG. 2, an arterial vascular image 40A in the ophthalmic image 30 and a venous vascular image 40B in the ophthalmic image 30 are prepared separately. However, a single vascular image showing both an artery and a vein may be used as the training data for output. Furthermore, when only the arterial vascular image is output to the mathematical model, only the arterial vascular image 40A may be used as the training data for output. Similarly, when only the venous vascular image is output to the mathematical model, only the venous vascular image 40B may be used as the training data for output.

The mathematical model construction process will be described with reference to FIG. 3. The CPU 3 acquires the ophthalmic image 30 captured by the ophthalmic image capturing device 11A as input training data (S1). In this embodiment, the data of the ophthalmic image 30 is generated by the ophthalmic image capturing device 11A and then acquired by the mathematical model construction device 1. However, the CPU 3 may also acquire the data of the ophthalmic image 30 by acquiring a signal (e.g., a light receiving signal by a light receiving element, etc.) that is the basis for generating the ophthalmic image 30 from the ophthalmic image capturing device 11A and generating the ophthalmic image 30 based on the acquired signal.

Next, the CPU 3 acquires data indicating at least one of the structures and diseases (in this embodiment, the fundus blood vessels, which are one of the structures) of the test eye from which the ophthalmic image 30 was captured as training data for output (S2). As described above, the training data for output in this embodiment is blood vessel images 40A, 40B indicating at least one of the arteries and veins in the ophthalmic image 30.

Next, the CPU 3 uses a machine learning algorithm to train the mathematical model using the training data set (S3). Commonly known machine learning algorithms include neural networks, random forests, boosting, and support vector machines (SVMs).

Neural networks are a technique that mimics the behavior of biological neural networks. Examples of neural networks include feedforward neural networks, RBF networks (radial basis functions), spiking neural networks, convolutional neural networks, recurrent neural networks (recurrent neural networks, feedback neural networks, etc.), and probabilistic neural networks (Boltzmann machines, Bayesian networks, etc.).

Random forest is a method of generating a large number of decision trees by learning from randomly sampled training data. When using random forest, the branches of multiple decision trees that have been trained in advance as classifiers are traced, and the average (or majority vote) of the results obtained from each decision tree is taken.

Boosting is a technique for generating a strong classifier by combining multiple weak classifiers. A strong classifier is constructed by sequentially training simple weak classifiers.

SVM is a technique for constructing a two-class pattern classifier using linear input elements. For example, SVM learns the parameters of the linear input elements based on the criterion (hyperplane separation theorem) of finding a margin-maximizing hyperplane that maximizes the distance from each data point from the training data.

The mathematical model refers to, for example, a data structure for predicting the relationship between input data (in this embodiment, data of an ophthalmological image similar to the training ophthalmological image) and output data (in this embodiment, data of the analysis results related to the fundus blood vessels). The mathematical model is constructed by training using a training dataset. As described above, the training dataset is a set of input training data and output training data. For example, correlation data (e.g., weights) between each input and output is updated through training.

In this embodiment, a multi-layered neural network is used as the machine learning algorithm. The neural network includes an input layer for inputting data, an output layer for generating data of the analysis result to be predicted, and one or more hidden layers between the input layer and the output layer. A plurality of nodes (also called units) are arranged in each layer. In detail, in this embodiment, a convolutional neural network (CNN), which is a type of multi-layered neural network, is used.

Note that other machine learning algorithms may be used. For example, generative adversarial networks (GAN), which utilize two competing neural networks, may be employed as the machine learning algorithm.

The processes of S1 to S3 are repeated until construction of the mathematical model is completed (S4: NO). When construction of the mathematical model is completed (S4: YES), the mathematical model construction process ends. The program and data that realize the constructed mathematical model are incorporated into the ophthalmic image processing device 21 (in another embodiment, the ophthalmic image capturing device 11B).

(Correlation between confidence and analytical adequacy)
4 and 5, the relationship between the degree of certainty of the analysis of an ophthalmic image performed by a mathematical model and the appropriateness of the analysis of the ophthalmic image will be described. The degree of certainty indicates the degree of likelihood of a prediction made using a mathematical model. As described above, the degree of certainty may be the degree of certainty of the analysis, or may be the inverse of the degree of certainty (uncertainty), etc. Thus, for example, the term "high certainty" is synonymous with the term "low uncertainty".

As described above, the mathematical model used in this embodiment outputs a probability distribution for identifying tissues in an ophthalmic image as the analysis result for the ophthalmic image. By calculating the entropy for each pixel for the output of the mathematical model, a confidence map showing the distribution of the certainty or uncertainty for each pixel is obtained. In other words, the confidence map can be expressed as information showing the distribution of confidence for the analysis performed by the mathematical model for each pixel constituting the ophthalmic image. Below, we consider the relationship between the average confidence for each pixel in the image area of the ophthalmic image and the suitability of the analysis of the ophthalmic image.

The suitability of an ophthalmic image as an analysis subject may be affected not only by the image quality of the ophthalmic image, but also by factors other than image quality. Therefore, it is difficult to obtain high accuracy by determining the analysis suitability of an ophthalmic image based only on a specific parameter of the ophthalmic image (for example, only image quality). Furthermore, as described above, it is unrealistic to construct a mathematical model for determining the analysis suitability of an ophthalmic image. Therefore, it is desirable to be able to more easily and appropriately determine the analysis suitability of an ophthalmic image.

Here, the inventor of the present invention focused on the correlation between the confidence of the analysis of an ophthalmic image performed by a mathematical model and the analysis suitability of the ophthalmic image. In other words, in a mathematical model, when an ophthalmic image that is similar to an ophthalmic image used for training (for example, ophthalmic image 30 shown in FIG. 2) is input, the confidence of the analysis of the ophthalmic image tends to be high. On the other hand, when an ophthalmic image that is not similar to the ophthalmic image used for training is input to the mathematical model, the confidence of the analysis of the ophthalmic image tends to be low. Here, when constructing a mathematical model that analyzes ophthalmic images, ophthalmic images with high analysis suitability are likely to be used for training the mathematical model. Therefore, when an ophthalmic image with high analysis suitability is input to the mathematical model as an analysis target, the confidence of the analysis is likely to be high. Therefore, the correlation between the analysis suitability and the confidence of each ophthalmic image is high.

Figure 4 is a graph showing the relationship between certainty information and frequency when a mathematical model analysis (in this embodiment, fundus blood vessel analysis) is performed on multiple ophthalmological images. In detail, the horizontal axis in Figure 4 shows uncertainty, which is a type of certainty information. In detail, the uncertainty in Figure 4 is the average value of the uncertainty for multiple pixels that are analyzed to have a higher probability of being fundus blood vessel tissue among the pixels that make up the ophthalmological image (multiple pixels for which "(probability of being an artery + probability of being a vein) > probability of being background" in this embodiment). Also, the vertical axis in Figure 4 shows the number (frequency) of ophthalmological images for each uncertainty. Figure 5 is a diagram listing ophthalmological images that were ranked within the bottom 15 in terms of certainty (level of certainty) in the analysis shown in Figure 4.

As shown in Figure 4, the analysis of multiple ophthalmological images revealed a frequency peak in the range of uncertainty from approximately 0.3 to approximately 0.8. When checking the ophthalmological images included in the peak, all of them were suitable for analysis. On the other hand, when checking the ophthalmological images with high analytical uncertainty (i.e., ophthalmological images with low analytical certainty), as shown in the example in Figure 5, all of them were unsuitable for analysis due to various influences such as poor image quality and the inclusion of diseases.

As described above, the degree of certainty of the analysis of an ophthalmic image performed by a mathematical model is highly correlated with the suitability of the analysis of the ophthalmic image. Therefore, in this embodiment, the suitability of the analysis of the ophthalmic image is determined based on the degree of certainty information of the analysis performed on the ophthalmic image by a mathematical model. As a result, the suitability of the analysis, which is influenced by various factors, is appropriately determined. For example, referring to FIG. 4, it is also possible to determine whether an ophthalmic image is suitable for analysis by using about 1.0 (e.g., about 0.9 to about 1.1) as an uncertainty threshold that can appropriately distinguish whether the analysis is performed appropriately.

(Analysis suitability determination process)
An analysis appropriateness determination process executed by the ophthalmological image-processing device 21 will be described with reference to Fig. 6. The analysis appropriateness determination process illustrated in Fig. 6 is executed by the CPU 23 of the ophthalmological image-processing device 21 in accordance with an ophthalmological image processing program stored in the storage device 24.

First, the CPU 23 acquires an ophthalmic image of the subject's eye captured by an ophthalmic image capturing device (a fundus camera in this embodiment) 11B (S1). The ophthalmic image acquired in S1 is the same type of image (i.e., an image captured by the same type of ophthalmic image capturing device) as the ophthalmic image 30 (see FIG. 2) used as input training data when training the mathematical model.

Then, the CPU 23 inputs the ophthalmic image acquired in S1 to the mathematical model trained by the machine learning algorithm (S2). As described above, the mathematical model performs an analysis of at least one of a specific structure and a disease of the test eye shown in the input ophthalmic image. In detail, the mathematical model of this embodiment outputs a probability distribution for identifying tissue in the ophthalmic image. In this embodiment, the mathematical model performs an analysis of the fundus blood vessels of the test eye (the first analysis in this embodiment).

The CPU 23 acquires certainty information associated with the analysis (S3). The certainty information is obtained based on the probability distribution output by the mathematical model. As described above, in this embodiment, the average value in a specific region of the certainty map showing the distribution of certainty or uncertainty of each pixel is calculated as the certainty information. Next, the CPU 23 judges whether the certainty information acquired in S3 satisfies a condition (S4). As an example, in this embodiment, the CPU 23 judges whether the average value of the certainty (level of certainty) in a specific region within the image region is equal to or greater than a threshold (or judges whether the average value of uncertainty is equal to or less than a threshold) to thereby judge whether the certainty information satisfies a condition. The threshold is set to a value that can appropriately distinguish whether the analysis of the ophthalmic image is performed appropriately.

In addition, in S4 of the present embodiment, the analysis suitability of the ophthalmic image is determined based on the certainty information in a part of the attention area (in this embodiment, the average value of the certainty in the attention area) of the entire image area of the ophthalmic image acquired in S1. As a result, only the certainty information in the attention area necessary for the analysis is referenced, so the accuracy of the determination of the analysis suitability is improved. In detail, in this embodiment, the attention area is set to a region consisting of pixels analyzed by a mathematical model to have a higher probability of being a specific tissue (fundus vascular tissue) (pixels where "(probability of being an artery + probability of being a vein) > probability of being background"). As a result, the correlation between the analysis suitability and the average value of the certainty is further increased, since the average value of the certainty in the region is calculated while excluding an area that is easy to analyze (in this embodiment, the background area where no blood vessels exist). However, the attention area may be set to any area in the image area. Also, an arbitrary area in a part of the image area that is analyzed to have a higher probability of being a specific tissue may be set to the attention area. In this case, the accuracy of the determination of the analysis suitability is further improved. Also, the certainty information of the entire image area of the ophthalmic image may be referenced.

If the certainty information about the ophthalmic image acquired in S1 satisfies the condition (S4: YES), the CPU 23 determines that the analysis suitability of the ophthalmic image acquired in S1 is good, and executes a second analysis on the ophthalmic image (S7). That is, the CPU 23 executes a second analysis on the ophthalmic image according to the determination result of the analysis suitability in S4 after the first analysis executed by the mathematical model in S2 (S7). Therefore, the second analysis is executed appropriately on the ophthalmic image with a high analysis suitability.

In this embodiment, the target tissue of the first analysis performed by the mathematical model in S2 is the same as the target tissue of the second analysis performed in S7. In this case, the analysis suitability of the second analysis is determined by performing the first analysis on the same tissue as the target tissue of the second analysis. Therefore, the analysis suitability of the ophthalmic image is determined with higher accuracy than when the target of the first analysis is different from the target of the second analysis.

As an example, in this embodiment, the content of the first analysis (analysis of fundus vascular tissue) is different from the second analysis (analysis of at least one of the subject's degree of arteriosclerosis, blood pressure, and age). However, when the second analysis is performed, the content of the first analysis and the content of the second analysis may be the same.

If the certainty information does not satisfy the condition (i.e., if the analysis suitability of the ophthalmological image is determined to be poor) (S4: NO), the CPU 23 excludes the ophthalmological image acquired in S1 from the analysis target (S8).

It is also possible to change the specific method for determining the analysis suitability of an ophthalmic image. For example, the CPU 23 may determine whether the analysis suitability of an ophthalmic image is good or not in multiple stages (for example, by generating a score indicating the analysis suitability) instead of determining in two stages. The CPU 23 may also determine the analysis suitability of an ophthalmic image separately based on the confidence information of each of a plurality of attention areas in the entire image area. The CPU 23 may perform a second analysis on areas with high analysis suitability and omit the second analysis on areas with low analysis suitability.

As described above, the ophthalmic image capturing device 11 can also determine the analysis suitability of the ophthalmic image. In this case, the program executed by the ophthalmic image capturing device 11 includes a program for realizing the mathematical model constructed by the mathematical model construction device 1. Below, the first to third shooting control processes for the ophthalmic image capturing device 11 to determine the analysis suitability of the ophthalmic image are described.

(First photography control process)
The first imaging control process executed by the ophthalmic image capturing device 11 of the first modified example will be described with reference to Fig. 7. First, the CPU 13 captures an ophthalmic image of the subject's eye using the ophthalmic image capturing unit 16 and acquires data (S11). The CPU 13 inputs the ophthalmic image captured in S11 to a mathematical model trained by a machine learning algorithm (S12). The CPU 13 also acquires confidence factor information associated with the analysis (S13).

Based on the confidence information acquired in S13, the CPU 13 determines the analysis suitability of the ophthalmic image captured in S11 (S14). As described above, the analysis suitability may be determined in two stages or in multiple stages. Next, the CPU 13 notifies the user of the result of the analysis suitability determination performed in S14 (S15). As a result, the user can properly grasp the analysis suitability of the captured ophthalmic image.

The CPU 13 also determines whether the analysis suitability determined in S14 satisfies the condition (or whether the certainty information acquired in S13 satisfies the condition) (S16). If the analysis suitability does not satisfy the condition (for example, if the average certainty in the image area is less than the threshold) (S16: NO), the process returns to S11 and the subject's eye is photographed again. As a result, the possibility of capturing an ophthalmic image suitable for analysis is improved. If the analysis suitability satisfies the condition (for example, if the average certainty is equal to or greater than the threshold) (S16: YES), the process ends.

(Second photography control process)
The second imaging control process executed by the ophthalmic imaging device 11 of the second modified example will be described with reference to Fig. 8. The ophthalmic imaging device 11 executing the second imaging control process can capture both an observation ophthalmic image, which is a provisional ophthalmic image, and an analysis ophthalmic image, which is a formal analysis target, for the tissue of the same subject eye. As an example, the ophthalmic imaging device 11 of the second modified example can continuously capture observation ophthalmic images of the fundus of the subject eye using an infrared camera, and can also capture analysis ophthalmic images of the fundus using visible light.

As shown in FIG. 8, the CPU 13 captures an ophthalmic image for observation of the subject's eye using the ophthalmic image capturing unit 16, and acquires the data (S21). The CPU 13 inputs the ophthalmic image for observation captured in S21 into a mathematical model trained by a machine learning algorithm (S22). The CPU 13 also acquires confidence level information associated with the analysis (S23).

Next, the CPU 13 determines whether the analysis suitability of the observation ophthalmic image captured in S21 satisfies the condition (or whether the certainty information acquired in S23 satisfies the condition) (S24). If the analysis suitability of the observation ophthalmic image does not satisfy the condition (for example, if the average value of the certainty in the image area is less than a threshold value) (S24: NO), it is highly likely that the conditions for capturing an ophthalmic image suitable for analysis are not met. Therefore, the process returns to S21, and the processes of S21 to S24 are repeated. If the analysis suitability of the observation ophthalmic image satisfies the condition (S24: YES), the CPU 13 captures an analysis ophthalmic image of the subject's eye using the ophthalmic image capturing unit 16 (S25). As a result, the analysis ophthalmic image is appropriately captured at the timing when the conditions for capturing an ophthalmic image suitable for analysis are satisfied.

(Third photography control process)
The third photographing control process executed by the ophthalmic image capturing device 11 of the third modified example will be described with reference to FIG. 9. The ophthalmic image capturing device 11 executing the third photographing control process can continuously capture the same part of the same subject's eye and perform averaging of the captured ophthalmic images to obtain an averaged image. In general, the more the number of ophthalmic images used in the averaging process increases, the higher the analysis suitability of the averaged image tends to be. As an example, the ophthalmic image capturing device 11 of the third modified example is an OCT device capable of capturing images of the tissue of the subject's eye (e.g., tomographic images of the fundus tissue, etc.), unlike the fundus camera exemplified in the above embodiment.

As shown in FIG. 9, the CPU 13 captures an ophthalmic image of a specific part of the subject's eye using the ophthalmic image capturing unit 16, and acquires the data (S31). Next, the CPU 13 acquires an average image by averaging multiple ophthalmic images captured of the same specific part (S32). The CPU 13 inputs the average image acquired in S32 to a mathematical model trained by a machine learning algorithm (S33). The CPU 13 also acquires confidence level information associated with the analysis (S34).

Next, the CPU 13 determines whether the analysis suitability of the averaged image acquired in S32 satisfies the condition (or whether the certainty information acquired in S34 satisfies the condition) (S35). If the analysis suitability of the averaged image does not satisfy the condition (for example, if the average certainty in the image area is less than a threshold value) (S35: NO), it is highly likely that an averaged image suitable for analysis has not yet been obtained. Therefore, the process returns to S31, and the processes of S31 to S35 are repeated. If the analysis suitability of the averaged image satisfies the condition (S35: YES), the latest averaged image is stored as an image for analysis (S36), and the averaged process ends. As a result, an averaged image suitable for analysis is appropriately acquired (captured).

The techniques disclosed in the above embodiments and modifications are merely examples. Therefore, it is possible to modify the techniques exemplified in the above embodiments and modifications. For example, it is possible to implement only some of the techniques exemplified in the above embodiments and modifications. It is also possible to implement a combination of the techniques exemplified in each of the above embodiments and modifications.

The processes of acquiring ophthalmic images in S1 of FIG. 6, S11 of FIG. 7, S21 of FIG. 8, and S31 of FIG. 9 are examples of "image acquisition steps". The processes of photographing ophthalmic images in S11 of FIG. 7, S21 of FIG. 8, and S31 of FIG. 9 are examples of "image photographing steps". The processes of inputting ophthalmic images into a mathematical model in S2 of FIG. 6, S12 of FIG. 7, S22 of FIG. 8, and S33 of FIG. 9 are examples of "image input steps". The processes of acquiring confidence information in S3 of FIG. 6, S13 of FIG. 7, S23 of FIG. 8, and S34 of FIG. 9 are examples of "confidence information acquisition steps". The processes of judging the analysis suitability in S4 of FIG. 6, S14 and S16 of FIG. 7, S24 of FIG. 8, and S35 of FIG. 9 are examples of "judging steps". The process of notifying the judgment result in S13 of FIG. 7 is an example of "notifying steps". In S16 of FIG. 7, the process of performing re-imaging when the analysis suitability does not satisfy the conditions is an example of a "re-imaging step." The process of acquiring an arithmetic average image in S32 of FIG. 9 is an example of an "arithmetic average step."

11A, 11B Ophthalmic image capturing device 13A, 13B CPU
14A, 14B Storage device 16A, 16B Ophthalmic image photographing unit 21 Ophthalmic image processing device 23 CPU
24 Storage device 30 Ophthalmological image 40A, 40B Blood vessel image

Claims (7)

An ophthalmic image processing device that processes an ophthalmic image which is an image of tissue of a subject's eye,
The control unit of the ophthalmologic image processing device
an image acquiring step of acquiring an ophthalmic image captured by an ophthalmic image capturing device;
an image input step of inputting an ophthalmic image into a mathematical model that is trained by a machine learning algorithm and that performs an analysis of at least one of a specific structure and a disease of a test eye shown in the input ophthalmic image, thereby outputting a probability distribution in which each of a plurality of classes is a random variable for detecting at least one of the specific structure and the disease shown in the ophthalmic image;
a confidence level information acquisition step of acquiring confidence level information indicating a confidence level of the analysis performed by the mathematical model on the input ophthalmic image based on the probability distribution output by the mathematical model;
a determining step of determining suitability of the ophthalmologic image as an analysis target based on the acquired certainty information;
13. An ophthalmologic image processing apparatus comprising: 2. The ophthalmologic image processing device according to claim 1,
the mathematical model to which the ophthalmic image is input in the image input step outputs a probability distribution for identifying tissues in the ophthalmic image;
The ophthalmologic image processing apparatus according to claim 1, wherein the certainty information is obtained based on the probability distribution output by the mathematical model. 3. The ophthalmologic image processing device according to claim 1,
An ophthalmological image processing device characterized in that, after the first analysis performed by the mathematical model to which the ophthalmological image is input in the image input step, an analysis step is further performed to perform a second analysis on the ophthalmological image depending on the judgment result in the judgment step. 4. The ophthalmologic image processing device according to claim 3,
13. An ophthalmologic image processing apparatus, wherein the tissue that is the subject of said first analysis is the same as the tissue that is the subject of said second analysis. 5. An ophthalmologic image processing apparatus according to claim 1,
The control unit is
2. An ophthalmologic image processing apparatus comprising: an ophthalmologic image processing device, comprising: an ophthalmologic image processing device configured to process an ophthalmologic image including a plurality of ophthalmologic images; An ophthalmic image processing program executed by an ophthalmic image processing device for processing an ophthalmic image which is an image of tissue of a subject's eye,
The ophthalmologic image processing program is executed by a control unit of the ophthalmologic image processing device,
an image acquiring step of acquiring an ophthalmic image captured by an ophthalmic image capturing device;
an image input step of inputting an ophthalmic image into a mathematical model that is trained by a machine learning algorithm and that performs an analysis of at least one of a specific structure and a disease of a test eye shown in the input ophthalmic image, thereby outputting a probability distribution in which each of a plurality of classes is a random variable for detecting at least one of the specific structure and the disease shown in the ophthalmic image;
a confidence level information acquisition step of acquiring confidence level information indicating a confidence level of the analysis performed by the mathematical model on the input ophthalmic image based on the probability distribution output by the mathematical model;
a determining step of determining suitability of the ophthalmologic image as an analysis target based on the acquired certainty information;
An ophthalmologic image processing program causing the ophthalmologic image processing device to execute the above-mentioned. an ophthalmic image capturing unit that captures an ophthalmic image, which is an image of tissue of a subject's eye;
A control unit for controlling the operation of the device;
Equipped with
The control unit is
an image capturing step of capturing an ophthalmologic image by the ophthalmologic image capturing unit;
an image input step of inputting an ophthalmic image into a mathematical model that is trained by a machine learning algorithm and that performs an analysis of at least one of a specific structure and a disease of a test eye shown in the input ophthalmic image, thereby outputting a probability distribution in which each of a plurality of classes is a random variable for detecting at least one of the specific structure and the disease shown in the ophthalmic image;
a confidence level information acquisition step of acquiring confidence level information indicating a confidence level of the analysis performed by the mathematical model on the input ophthalmic image based on the probability distribution output by the mathematical model;
a determining step of determining suitability of the ophthalmologic image as an analysis target based on the acquired certainty information;
An ophthalmologic imaging apparatus comprising:

Images