JP6038438B2 - Fundus image analysis apparatus, fundus image analysis method and program - Google Patents

Description

  The present invention relates to a fundus image analysis technique, and more particularly to a fundus image analysis apparatus that analyzes temporal changes in intensity myopia.

  Popular myopia in ophthalmological diseases is known as severe myopia, a severe illness that causes posterior polar tissue degeneration, eye axis elongation, strong myopia, and sometimes blindness. .

  At present, there are inventions described in the following Patent Document 1 as the radical treatment for intense myopia, but it is still in the research stage. Actually, only symptomatic treatment that suppresses the symptoms and progression of the disease state by pharmacotherapy or surgical treatment is available. There is no current situation. Therefore, early detection is important in the treatment of intense myopia, and basically diagnosis based on continuous captured images by a fundus camera or the like is made.

The following techniques are known as image analysis techniques using images taken by a fundus camera or the like.
(Patent Document 2) “Image superposition method”, NTT
In Patent Document 2, feature points that do not have any problem even if they are used as reference points are prioritized, and feature points that are not suitable for use as reference points are registered in a database for each image modality (eg, fundus camera image). Alignment is performed using several feature points with high priority.
(Patent Document 3) “Personal Authentication Method Using Retina Image”, Iwasaki Gakuen Also, in the retina authentication of Patent Document 3, the nipple is likely to be set as a feature point on the premise that there is little fluctuation in the vicinity of the nipple.

Examples of ophthalmic image processing techniques include the following.
(Patent Document 4) “Ophthalmic Image Processing Method and Storage Medium”, Canon In Patent Document 4, there is a difference in illumination conditions or illumination unevenness by analyzing a low frequency component of an image and performing shading correction. A method for enabling comparison between images is proposed.
(Patent Document 5) “Fundus image processing device”, Nidek In Patent Document 5, as a temporal change analysis of the nipple, a radial distance ratio C / D from the center of the nipple of the cup line where the disc line on the outer edge of the nipple and the blood vessel are recessed A method for calculating the ratio and the R / D ratio (R is the width of DC) is proposed. Even if the position, magnification, and angle of the images are not matched, if they are evaluated by the ratio, they are not affected by the mismatch.

Special table 2007-536272 gazette JP-A-9-192106 JP 2008-206536 A JP 2003-010131 A JP 2011-083555 A

  In the above-mentioned patent document 2, since it is premised on the overlapping of images acquired by the same image modality (fundus camera), it cannot be applied when different cameras are used.

  In the retina authentication of Patent Document 3, there is a high possibility that the nipple is set as a reference feature point. However, depending on the pathological condition such as intense myopia, the variation of the nipple may be the largest, and the reference feature The priorities regarding the selection of points are not fixed, and there is a problem that they are difficult to apply when they change with the progress of the disease state.

  In addition, the method proposed in Patent Document 4 can analyze the degree of growth of the vitiligo that is the main target. However, the consistency of geometric conditions such as position, magnification, and angle does not use high-frequency components. Since they cannot be matched, it is difficult to analyze the time course of ocular blood vessel running and nipple shape.

  It is qualitatively known that when the technique described in Patent Document 5 is applied to intensity myopia, enlargement / reduction / elliptical deformation of the nipple itself occurs. There is a problem that it cannot be analyzed.

  Intensity myopia, since the pathology gradually progresses over a long time span of 20 years or longer, it is desirable to observe the long-term progress using a fundus camera or the like. However, implementation is difficult due to the following two problems.

  According to Article 24, Paragraph 2 of the Physician Law, “… related to medical care performed by a doctor working in a hospital or clinic, is the administrator of the hospital or clinic, and other medical services are related to this doctor for 5 years. In other words, a chart that has passed five years is likely to be discarded from a medical institution, and for medical images (film media such as X-ray images and fundus images). Is not obligated to store for up to 5 years, and due to storage space limitations, it is highly likely that it will be discarded at the previous stage, except for research institutions such as university hospitals.

  Also, medical images have undergone major changes from analog (film media) to digital (computer data) over the past 20 years, and imaging technology has improved dramatically year by year. However, it is difficult to quantitatively compare an image taken 20 years ago with the current image.

  In response to the problem of the previous retention period, the system is going to be changed globally in the direction of centralization of medical records and lifelong electronic medical records (called EHR, which is already in operation in overseas advanced countries). If it is realized, it will be solved in the future.

  On the other hand, a technique for comparing and collating with fundus images taken in the past has been studied not only in the medical field but also as an application to biometric authentication in the IT information security field, starting with Patent Document 3. However, in this field, in principle, comparison of images taken by a camera of the same model with relatively similar ages is not directly applicable to the field of the present application. Further, as disclosed in Japanese Patent Laid-Open No. 9-192106, “Image superposition method” (NTT), a method for examining the degree of progression of arteriosclerosis by comparing past fundus camera images with blood vessel images has been proposed. Since it is aimed at medical examination, the past images that are also used as comparative controls are compared with images one and two years ago and cannot be directly applied to the field of the present application.

  It is an object of the present invention to provide an image analysis technique that matches fundus images with different camera models and imaging conditions, and enables comparison of dimensions of various parts.

  In the present application, for each of a plurality of fundus images of the same subject, a fundus image is roughly cut out to make all the image sizes the same. Then, using the latest image as a reference, linear affine transformation is applied to each of the other images in the order of magnification correction, angle correction, and position correction. The magnification, angle, and position correction parameters are determined through trial and error while interacting with the screen. In order to support this interactive work, each fundus image is monochromeized so that the nipple and blood vessel patterns are clear, and the two images are superimposed so that consistency can be confirmed on the screen. An ophthalmologist or the like can select where the reference is to be made through trial and error.

According to one aspect of the present invention, one among the plurality of fundus image data set based fundus image data, for each fundus image data other than the reference fundus image, magnification, angle, each location set the correction parameter, based on the correction parameters, magnification correction, angle correction, position correction alms, and the fundus image matching section to create a corrected eye fundus image data, the reference fundus image data and the corrected eye fundus image pressure to give a same image processing on the data, the composite image creating unit that creates the reference fundus image data and the previous SL synthetic image obtained by synthesizing the corrected eye fundus image data, created by the composite image creation unit based on the synthesis image, to determine whether the corrected eye fundus image data and the reference fundus image data are matched, if not consistent, repeated alignment process in said fundus image matching unit and the composite image creation unit By Succoth, before Kibai rate, angle, and the correction parameter determination section that determines a respective correction parameter position for each fundus image data, and based on the determined correction parameter, to create at least one corrected eye fundus image data There is provided a fundus image analysis apparatus comprising a corrected fundus image data creation unit.

  Furthermore, an image digitizing unit that creates digital image data from an analog image among a plurality of fundus images, and a region corresponding to the fundus is trimmed for each digital image data, and is configured with a predetermined number of pixels It is preferable to include a fundus image pre-processing unit that performs scaling processing with the same vertical and horizontal magnification parameters so as to become image data, and converts the fundus image data to a uniform size.

  The fundus image pre-processing unit displays the digital image data on a screen, and designates a region corresponding to the fundus on a displayed image with a specific figure, thereby setting the vertical and horizontal same magnification parameters of the scaling process. It is preferable to set.

The fundus image matching unit displays each of the reference fundus image data and the corrected fundus image data on a display screen, displays a figure substantially along the outline of the eyeball, and the size of the area of the figure is the same. It is preferable to set the vertical and horizontal same magnification correction parameters by adjusting so as to be.

  The fundus image matching unit displays the reference fundus image data and the corrected fundus image data on a screen, and displays the first and second images respectively at corresponding positions on both displayed images. It is preferable that the angle rotated by the operation unit is set as an angle correction parameter by instructing the operation unit to rotate so that the parallelism of the reference lines is the same.

  The fundus image matching unit displays the reference fundus image data and the corrected fundus image data on a screen, and displays the first and second images respectively at corresponding positions on both displayed images. It is preferable that the position moved by the operation unit is set as a correction parameter for the position by instructing the operation unit to move the reference point so that the reference point becomes the same position.

  In the above, the composite image creating unit selects any one or two primary colors from among the three RGB primary colors, selects two different primary color sets, and converts one of the selected primary color sets into a monochrome image. The reference fundus image data and the corrected fundus image that have been assigned to the corrected fundus image data converted into a monochrome image and assigned to the reference fundus image data that has been converted into a monochrome image. It is preferable to create the color composite image by combining the data.

  The composite image creating unit preferably performs gray scale conversion, negative / positive inversion, and gradation expansion processing as the predetermined image processing. The fundus image matching unit may perform angle correction after magnification correction.

According to another aspect of the present invention, one among the plurality of fundus image data set based fundus image data, for each fundus image data other than the reference fundus image, magnification, angle, position set each correction parameter, based on the correction parameters, magnification correction, angle correction, position correction alms, and the fundus image matching step of creating the corrected eye fundus image data, the correction fundus and the reference fundus image data in addition the same image processing on the image data is converted into a monochrome image, the composite image creating step of creating a synthetic image obtained by synthesizing the reference fundus image data and the correction fundus image data, creating the synthesized image based on the synthetic image created in step, to determine whether the said reference fundus image data corrected eye fundus image data are matched, if not matching, the fundus image matching stearate By repeating the matching process in said composite image creating step and flop, based pre Kibai rate, angle, the correction parameter and the correction parameter determination step, which is determined for determining the correction parameters of the position for each fundus image data And a corrected fundus image data creating step of creating at least one corrected fundus image data.

  The present invention may be a program for causing a computer to execute the fundus image analysis method described above, or a computer-readable recording medium for recording the program.

  The image matching method proposed in the present invention can be realized by inexpensive general-purpose personal computer software, and does not require the addition of a special graphics board such as a GPU. Also, if the image input system is an inexpensive film scanner, various fundus photographs taken in the past can be input, and images from existing fundus cameras can be captured as they are. Moreover, even if the imaging conditions of images taken in the past are unknown, they can be matched to each other as long as film and image data remain. That is, the present invention can realize a temporal change analysis of “intensity myopia” using fundus photographs of the same patient scattered and stored in various medical institutions without investing in large-scale hardware facilities. .

It is a functional block diagram which shows the example of 1 structure of the image-analysis apparatus (time series collation apparatus) by one embodiment of this invention. It is a figure which shows the outline | summary of the time series image analysis method by the fundus camera image using the image analysis technique by this Embodiment. It is a figure which shows the mode of the unification process of the size of all the images. It is a flowchart figure which shows the flow of the matching process regarding all the process target images on the basis of the reference | standard image, when making a reference | standard image into the image of the newest age. It is a flowchart figure which shows the detail of the matching process of the image by a fundus image matching part, and is a figure which shows the flow of the matching process of the image data of two images. It is a figure which shows the detail of the matching process of an image, the whole eyeball is displayed on the screen, and is a figure which shows the content of the magnification correction process. It is a figure which shows an example of the interactive process of magnification correction. It is a figure which shows an example of the interactive process of magnification correction. It is a figure which shows the example of the content of the angle correction process of the image A and the image C which matched the magnification of the eyeball by the method of FIG. 6 or FIG. 7A and FIG. 7B. It is a figure which shows an example of the interactive type | mold process of angle correction. It is a figure which shows an example of the interactive type | mold process of angle correction. It is a figure which shows the content of the position correction process among the alignment processes of 2 images. It is a figure which shows a mode that position correction is performed so that both position may become the same about reference | standard image A 'and input image 2D'. FIG. 6 is a diagram showing details of overlay display (S3-3-6) in FIG. 5. It is a figure which shows the detail of the overlay display (S3-3-6) of FIG. 5, and is a figure following FIG. It is a figure which shows the detail of a process of the overlay display (step S3-3-6) of 2 images. It is a figure which shows the example of a sample of a time series fundus photograph. A photograph taken with a film camera in 1989 and the latest image of 2010, the top is a raw photo, the bottom is a photo with the magnification and angle corrected based on the latest image of 2010, and the middle photo Is an overlay image. It is a figure which shows the alignment processing operation example of a 2 screen fundus camera image, and is a screen which specifies the numerical value of a magnification and an angle (rotation). It is a figure which shows the alignment processing operation example (2: interactive correction | amendment) of a 2 screen fundus camera image, the 1st image on the left is a 2010 L image, the middle overlay image is a red image, The two images are 1989 L images, and the middle overlay image is a green image. By correcting the magnification and angle, the blood vessels in the middle overlay image can be matched very well, It is a detailed photograph of the state of the change of the part. It is a figure which shows the alignment processing operation example (3: automatic correction) of a 2 screen fundus camera image, and is a figure which shows a mode that the rhombus marking was performed to two feature points corresponding to a left-right image. It is a figure which shows the alignment process operation example (4) of a 2 screen fundus camera image, and is a figure which shows a mode for the automatic position correction based on marking of a rhombus etc. (it may be a cross mark etc.) in two feature points corresponding to a left-right image It is. It is a figure which shows the alignment process operation example (5) of a 2 screen fundus camera image, and is the figure which displayed the overlay image after automatic position correction. It is a figure which shows the example of the time series fundus camera image after completion | finish of a matching process. It is a figure which shows the example of a time series image analysis after completion | finish of matching, and is a figure which shows the photograph which expanded the affected part based on FIG. It is a figure which shows the time-sequential image analysis example (2) after completion | finish of control processing, and demarcates a rhombus figure about the image of 2010 and the image of 1989, and aligns a rhombus as a standard, and it is a middle image FIG. 6 is a diagram illustrating an example of generating an overlay image, and is a diagram in which a Red image and a Green image are overlapped and a time change of a lesion is seen.

  Hereinafter, an image analysis apparatus according to an embodiment of the present invention will be described with reference to the drawings, taking as an example analysis of symptoms of intense myopia based on fundus photographs.

First, the following may be cited as a variation factor in the comparison and collation between fundus images.
(1) Changes in the subject itself (intensity myopia, pathological progression such as glaucoma)
・ Expansion / reduction of the eyeball (magnification of the rear of the eyeball in high myopia)
・ Rotation of the eyeball (rotation around the eye axis)
・ Rotation of the eyeball around the X-axis or Y-axis: Affects the aspect ratio of the photographed image ・ Changes in ocular vascular running Deformation due to arteriosclerosis, changes in blood vessel branch points, generation of new blood vessels ・ Deformation of optic nerve head Change: Influence on the magnification of the photographed image (2) Fluctuation in camera photographing conditions / Difference in camera model (analog / digital method)
・ Magnification difference during shooting ・ Difference in angle during shooting ・ Color temperature variation of illumination light

  On the other hand, for example, when it is desired to analyze the temporal variation of ocular blood vessel running, it is important where the reference point for alignment (a point that does not change) should be set.

  FIG. 1 is a functional block diagram showing an example of the configuration of an image analysis apparatus (time series collation apparatus) according to this embodiment. As shown in FIG. 1, an image analysis apparatus (including a time-series image collation apparatus) 1 includes an image digitizing unit 3 that digitizes a fundus image, and a fundus image that performs trimming of the fundus image, unification of sizes, and the like. In order to match the two images, the processing unit 5, the fundus image matching unit 7 for matching the image of the fundus image serving as a reference and the fundus image to be analyzed, and the composite image creating unit 11 for synthesizing the matched images. A correction parameter determining unit 15 for determining the correction parameters, a corrected fundus image data creating unit 17, and a storage unit (memory) 19.

  This image analysis apparatus 1 includes a time-series collation device for fundus images for analyzing temporal changes of a plurality of fundus images taken in color or monochrome at different times for the same subject using a fundus camera. .

  The image digitizing unit 3 digitizes a fundus image recorded on a film medium among a plurality of fundus images, and creates digital image data.

  The fundus image pre-processing unit 5 trims a region corresponding to the fundus for each digital image data, performs a scaling process with the same vertical and horizontal magnification parameters so as to be image data composed of a predetermined number of pixels, Is converted into unified fundus image data.

  The fundus image matching unit 7 sets at least one of the plurality of digitally converted fundus image data as reference fundus image data, and performs vertical and horizontal magnifications for each fundus image data other than the reference fundus image. Each correction parameter of angle and position is set, and magnification correction, angle correction, and position correction are performed for each aspect, and one or more corrected fundus image data is created.

  The composite image creating unit 11 performs predetermined image processing on each of the reference fundus image data and the one or more corrected fundus image data to convert it into a monochrome image, and the reference fundus image data and one selected corrected fundus A color composite image is created by combining the image data.

  Based on the color composite image created by the composite image creation unit 11, the correction parameter determination unit 15 appropriately repeats the processing of the fundus image matching unit 7 and the composite image creation unit 11, and calculates the magnification, angle, and position for each aspect. Each correction parameter is determined for each fundus image data.

  The corrected fundus image data creation unit 17 creates one or more corrected fundus image data based on the determined correction parameters.

  The image analysis apparatus 1 actually includes a CPU 1a that performs processing, an application program that describes a processing algorithm, a storage unit 19 that stores data such as parameters obtained by the processing, an input unit 21 such as image data, This can be realized by a general personal computer (PC) including the image output unit 23 and the like. In addition, not all the functional units are indispensable. For example, the image digitizing unit can be replaced by another scanner device, and the input unit 21 and the output unit 23 are also provided separately from the main body. Also good.

  FIG. 2 is a diagram showing an overview of a time-series image analysis method using fundus camera images using the image analysis technique according to the present embodiment. In the image analysis method shown in FIG. 2, a plurality of time-series fundus photographs are prepared. Here, the fundus photograph may be a mixture of analog film, digital image data, and the like.

  First, in step S1, a film medium or a digital image printed on the medium (hereinafter referred to as “medium image”) is extracted from the image data of the same fundus photograph. Then, the medium digitizing unit 3 converts this medium image into digital image data using a film scanner or the like. Thereby, all the image data is converted into digital image data.

  Next, in step S2, the fundus image preprocessing unit 5 unifies the vertical sizes of all digital image data.

  FIG. 3 is a diagram showing a state of the unification processing of the sizes of all images. First. It is assumed that an original image (4823 × 3784 pixels) after scanning at the upper left is obtained. However, for example, in film photographs, shooting date information, etc., as indicated by the symbol T, may be reflected. Therefore, the fundus image pre-processing unit 5 performs trimming by excluding the portion other than the fundus photographic part from the original image. Actually, processing that removes the peripheral portion may be performed. Then, a trimmed image as shown in the upper right is obtained (4823 × 3784 pixels). Next, as shown in the lower right diagram, the fundus image preprocessing unit 5 performs image enlargement / reduction processing in order to unify the number of pixels of the image. Here, the vertical length is unified to 1024 pixels. The number of horizontal pixels 1139 changes depending on the unification of the vertical direction. Since there is less noise to be trimmed in the vertical direction than in the horizontal direction, it is preferable to use the vertical direction as a reference.

  Next, in step S <b> 3, the fundus image matching unit 7 performs, for example, matching processing of, for example, all images of the same fundus photograph with reference to the latest age image. Since the image of the latest age is considered to be the most stable technically, the image of the latest age is used as an example as an example, but if it is a stable image, it can be used. An alignment process based on two or more images may be performed. This process will be described later.

  Next, in step S4, the correction parameter determination unit 15 or the like measures and determines the parameter, length, area, and angle (blood vessel, nipple diameter, etc.) of the corresponding characteristic portion of each image. This process will also be described later.

  FIG. 4 is a flowchart showing the flow of matching processing for all the processing target images based on the reference image when the reference image is an image of the latest age.

  First, in step S3-1, time-sequential 1,..., N fundus image data (after unifying the vertical size of the image, the Nth image is set as the latest image) is prepared.

  Next, in step S3-2, first, P = 1, that is, the first image data is taken out. In step S3-3, the fundus image matching unit 7 determines that the N fundus image data and the Pth fundus Performs alignment processing with image data. Next, in step S3-4, P is incremented by 1. In step S3-5, P <N is determined. If No, the process ends (End). If Yes, step S3-3 is reached. Return to and continue processing. In step S3-3, the alignment condition parameter obtained in the alignment process and the P-th corrected (after alignment) image data after the alignment process are output to the output unit 23.

  FIG. 5 is a flowchart showing details of the image matching process (step S3-3) by the fundus image matching unit 7, and shows the flow of the image data matching process of two images. First, the fundus image matching unit 7 performs matching processing using two pieces of image data digitized by the image digitizing unit 3 and pre-processed by the fundus image pre-processing unit 5 as shown in FIG. First, two pieces of input image data are acquired (step S3-3-1), and magnification correction is performed in step S3-3-2.

  Next, angle correction is performed in step S3-3-3. In step S3-3-4, two feature points are designated, and position correction is performed in step S3-3-5. The order of these three types of corrections S3-3-2, S3-3-3, and S3-3-5 greatly affects operability and correction accuracy, and thus must be performed in an optimal order. First, the position correction of S3-3-5 does not cause a significant difference in accuracy at any stage, but it is better to perform it last in terms of operability. On the other hand, S3-3-2 and S3-3-3 have a significant difference in accuracy depending on the order. Factors that require correction include (1) change in the subject itself and (2) change in camera shooting conditions. The correction order is basically (2) correction for changes in camera shooting conditions, (1 ) The order of correction for variations in the subject itself. (2) There are two types of camera shooting condition fluctuations: camera magnification and shooting angle, but it has been found that the influence of the former on the results of the survey is significant. On the other hand, (1) changes in the subject itself include enlargement / reduction of the eyeball, rotation of the eyeball (rotation around the eye axis), and rotation of the eyeball around the X axis or Y axis. In the case of intense myopia, the enlargement / reduction of the eyeball causes an enlargement in the rear part in the direction of the eye axis, so that the magnification and the angle are hardly affected. The rotation of the eyeball around the X axis or the Y axis affects the vertical / horizontal magnification of the captured image, but (2) the influence is small compared to the camera magnification in the variation of the camera imaging conditions. Therefore, (1) The eyeball rotation (rotation of the center of the eye axis) has the greatest influence on the subject itself. From these, (2) magnification correction (caused by the camera magnification) is performed first as a correction for fluctuations in camera shooting conditions, and (1) angle correction (caused by eyeball rotation) is performed as a correction for fluctuations in the subject itself. If the order is to be performed, good results can be obtained both in terms of accuracy and operability.

  Next, in step S3-3-6, the composite image creation unit 11 performs overlay display of two images. In step S3-3-7, it is confirmed whether or not they are matched. If No, the process returns to step S3-3-2. If Yes, the correction parameter determination unit 15 determines a correction parameter, and step S3- In 3-8, the matched image data and magnification / angle / position parameters are output. The corrected image can be created by the corrected fundus image data creation unit 17 based on the original image and the correction parameters.

  FIG. 6 is a diagram showing details of the image matching process (step S3-3-2), in which the entire eyeball is displayed on the screen, and shows the contents of the magnification correction process. Based on the image of the eyeball 21a of the input image 1 (reference image) (A) and the image of the eyeball 21b of the input image 2 (B), magnification correction is performed using the following equation. Here, for example, the eyeball 21c of the input image 2 (C) after correction is indicated by magnification factors sx and sy obtained as 200% in the X direction and 125% in the Y direction. At this timing, the magnification factors sx and sy are stored in the memory together with the image.

  The other example of a magnification correction process is shown. 7A and 7B are diagrams illustrating an example of interactive processing for magnification correction. Here, XY coordinates are defined on the display screen as software, and when a figure or the like is drawn on the display screen, the position coordinates of the figure are stored in the memory by the vertex or the like of the figure. Such software can be realized by general graphic software. Here, the starting image of FIG. 7A is the same as FIG. Here, in the backgrounds 23a and 23b displayed on the screen, two input images (eyeballs) 21a and 21b correspond to two images on the screen as shown in (A ′) and (B ′). Specify two points in the area. Here, a rhombus is designated (A ′, B ′). A rectangular region such as a rectangle can be used instead of the diamond.

  On the screen A ′, the region is displayed on the display screen of the output unit 23 by the input unit 21 of the mouse so that the four vertices of the rhombus 25a prepared as a template along the contour of the oval eyeball substantially coincide with the contour. Specify with. Similarly, on the screen B ', the rhombus 25b template is arranged and specified so as to be slightly larger than the outline of the eyeball in the X and Y directions. The larger dimensions are L1 to L4, respectively. L1 to L4 are stored in the storage unit 19. Here, the reason for setting it to be larger is that there are many cases where a slight reduction occurs in the dimension in the X direction or the Y direction due to angle correction described later. If the angles are apparently different, it is easier to collate by increasing the degree of increase.

  Next, as shown in FIG. 7B, the vertices of the rhombuses 25a and 25b are changed from (X11, Y11) to (X14, Y14), (X21, Y21) to (X24, Y24), and based on the following formula, the coefficient Sx and Sy are obtained. Thereby, X ′ and Y ′ after magnification correction can be obtained. Here, (B ') of the input image 2 is enlarged and displayed as shown in (C) (eyeball outline 21b-1). This figure corresponds to the image 21c after alignment in FIG. The difference from the process of FIG. 6 is that the process of FIG. 6 is automatically enlarged / reduced by the graphic software of the computer software, whereas the process of FIGS. 7A and 7B is a trial and error on the user's screen. The point is that the size is enlarged / reduced to match the reference image by processing.

  In this example, Sx = 200% and Sy = 125%, and these values are stored in the storage unit 19 together with the image.

  FIG. 8 is a diagram showing an example of the content of the angle correction processing of the image A and the image C in which the magnifications of the eyeballs are matched by the method of FIG. 6 or FIGS. 7A and 7B. If the image A is an ellipse, the major axis of the ellipse of the image A is arranged parallel to the x axis. On the other hand, if the major axis of the ellipse shown in image C is not parallel to the x-axis, the angles must be matched. Therefore, the angle θ formed by the major axes of both images A and C is obtained. For example, in this example, θ = 20 degrees.

  Therefore, the angle of the image C is corrected using the following formula, and the coordinates (X ′, Y ′) of the image C can be obtained so that the long axes of the images A and C coincide with each other. Thus, the angle corrected image is obtained. The value of θ is also stored in the storage unit 19.

  Another example of angle correction will be described. 9 and 10 are diagrams illustrating an example of interactive processing for angle correction. The starting image C in FIG. 9 is the same as that in FIG. Next, for the reference image A and the input image C, two graphics corresponding to the two screens are drawn (designated) on the screen by the input unit 21. Here, FIG. 9 shows an example in which a rhombus figure having diagonal lines L5, L6, L5 ′, and L6 ′ is specified on the screen, but straight lines corresponding to the longer diagonal lines L5 and L5 ′ may be used. . These straight lines L5 and L5 'are moved with the mouse so as to be aligned with the long axis of the ellipse, and rhombuses or straight lines are displayed and drawn on the reference image A' and the input image 2C '. Here, since the absolute dimension of the straight line or rhombus specified in the input image 2C ′ is not related to the angle correction, it may be set in any way, but in order to make it easy to see in operation, it is specified larger than the ellipse. Yes.

  Next, as shown in FIG. 10, the diagonal coordinates (X11, Y11), (X12, Y12), (X21, Y21), and (X22) of the rhombus are set so that the intersection angle between the straight line L5 and the straight line L6 is zero. , Y22). Here, for example, the angle difference θ is 20 degrees, and X ′ and Y ′ are obtained by the following equations. The obtained value is stored in the memory.

In this way, by correcting the angle of the input image 2 as shown in the image D, the angle of the major axis is the same as that of the input image 1 (the angle between the major axes is zero). .
At this time, the coordinate value of the corrected image D and the correction angle θ are stored in the memory.

  Next, image position correction processing will be described. FIG. 11 is a diagram illustrating the contents of the position correction process in the matching process of two images.

  When the position of the comparison target image D that is the input image 2 is to be corrected in accordance with the reference image that is the input image 1 (A), first, at least two corresponding feature points are designated in the two images on the screen. Here, the leftmost points (X1, Y1) and (X2, Y2) of the ellipse are designated (A ′, D ′).

  Next, as shown in FIG. 12, the reference image A ′ and the input image 2D ′ are corrected so that the positions of the reference points are the same (in contrast, the positions other than the reference points are not necessarily the same). Not required). In this case, the input image 2D ′ is set to −43 in the X direction and −85 in the Y direction with respect to the reference image A ′, so that the positions of the reference points coincide with each other. That is, the input image 2D ′ can be corrected for position by the following conversion formula.

  Next, as shown in FIGS. 13 and 14, the overlay display (S3-3-6) of FIG. 5 is performed. In FIG. 13, the reference image A and the input image 2 (E) are drawn with different colors and drawn on the overlay image so that they can be distinguished from each other (F). In the reference image A and the input image 2 (E), processing for superimposing the eyeball feature patterns 21a and 21b-5 is performed. In FIG. 13F, for the sake of simplicity, the outline of the ellipse is illustrated as being overlapped. However, actually, as shown in FIG. 17, the feature pattern including the inside of the ellipse is overlapped. Yes. In this case, for example, the former image may be displayed in a form that can be visually identified as red and the latter image as green, that is, a color image so that the degree of overlap can be clearly understood.

  Next, as shown in FIG. 14, it is determined based on the degree of coincidence of the feature pattern of the ellipse whether the consistency is good in the superimposed image F (step S3-3-7). If the consistency is OK, the process proceeds to step S3-3-8. If the consistency is NG, for example, the processing is restarted from the magnification correction of FIG. 7B.

  If the consistency is OK (Yes), the process proceeds to step S3-3-8, and the image data E subjected to the alignment process, the magnification parameter S, the angle parameter θ, and the position parameters X and Y are output. Store parameters in memory. As a result, the process ends (End).

  FIG. 15 is a diagram illustrating details of processing of overlay display of two images (step S3-3-6). Image data 1 (reference image) and image data 2 (input image) are prepared (steps S21 and S31). Next, gray scale conversion (for example, extraction of G component) is performed on each image data (steps S22 and S32). As an example, blood vessels are displayed in black.

For example, when the image size is Xs × Ys, the image data 1 having a value of 0 to 255 is set to Image1 (x, y, c), and the image data 2 is set to Image2 (x, y, c) (x = 0. ,,, .Xs, y = 0,…, Ys, c = 0 (Red), 1 (Green), 2 (Blue))
Images Gray1 (x, y) and Gray2 (x, y) after grayscale conversion are generally

  Next, negative / positive inversion of each grayscale converted image is performed (steps S23 and S33). Here, for example, the defect is whitened and the background is blackened.

である。 Here, the image after negative-positive inversion is

It is.

  Next, gradation expansion conversion is performed in steps S24 and S34. As a result, the defective area is raised. In steps S25 and S26, RGB plane allocation is performed.

  Here, the image after gradation expansion conversion is subjected to the following scaling, and is limited to 0 when it is less than 0, and to 255 when it exceeds 255.

である。 However,

It is.

  In step S25, RGB plane allocation is performed. In step S25, (R) is assigned to the image data 1. In step S35, (G) is assigned to the image data 2. In step S26, the image data 1 and the image data 2 are subjected to RGB plane synthesis and full color conversion. Next, an overlay image (RGB full color image) is generated (step S27). Such an image is shown in FIG.

で表される。 Here, ImageS (x, y, c) is the overlay image data having a value from 0 to 255.

It is represented by

Examples using the image analysis technique according to the above embodiment will be described in detail below.
FIG. 16 is a diagram illustrating an example of a time-series fundus photograph. For example, when fundus photographs collected over a period of more than 20 years from 1989 to 2010 are taken with various cameras (cameras with different image quality, whether analog, digital, or digital), and the magnification and lighting conditions are also different, As shown in FIG. 16, the collation work was difficult.

  FIG. 17 shows, for example, a photograph taken with a film camera in 1989 and the latest image in 2010, with the raw image on the top and the photograph with the magnification and angle corrected based on the latest image in 2010. The middle photo is the overlay image. After the correction, almost the same photograph is obtained, and when the overlay image is viewed, it can be seen that most of the portions are coincident with each other, and only the change of the lesion portion due to intense myopia can be seen.

  FIG. 18 is a diagram showing an example of the matching processing operation of the two-screen fundus camera image, which is a screen for designating numerical values of the magnification 53 and the angle (rotation) 55. Here, it is a figure which shows a mode that the correction parameter which set the magnification X to 95%, the magnification Y to 88%, and set the rotation on XY plane to 85 degree | times is designated. In this way, the correction parameter can be changed while viewing the screen. This method shows an example of a method for enlarging / reducing the size so that the processing of FIGS. 7A and 7B fits the reference image by trial and error processing on the user's screen.

  FIG. 19 is a diagram showing an example of alignment processing operation (2: interactive correction) of a two-screen fundus camera image. The first image on the left is a 2010 L image, and the middle overlay image is a red image. The second image on the right is an L image from 1989, and the middle overlay image is a green image. By correcting the magnification and angle, the blood vessel in the middle overlay image can be matched very well. It can be seen that the change of the lesion site can be seen in detail. As described above, even interactive correction can be performed with high accuracy and images of different ages can be matched.

  FIG. 20 is a diagram showing an example of the alignment processing operation (3: automatic correction) of the two-screen fundus camera image, and is a diagram showing a state in which rhombus marking is performed at two feature points corresponding to the left and right images. Here, a feature point is a portion where blood vessels gather in the nipple (the result branches). By using such a feature point as a reference, processing for aligning two images can be easily performed.

  FIG. 21 is a diagram showing a matching operation operation example (4) of the two-screen fundus camera image. For automatic position correction based on markings such as rhombuses (or a cross mark or the like) at two feature points corresponding to the left and right images. It is a figure which shows a mode. By using such a feature point as a reference, processing for aligning two images can be easily performed. The position can be automatically corrected by the position of the diamond.

  FIG. 22 is a diagram showing an operation example (5) of matching processing for a two-screen fundus camera image, and is a diagram displaying an overlay image after automatic position correction. The middle overlay image is a red image, the second image on the right is an L image from 1989, the middle overlay image is a green image, and by correcting the magnification and angle, It can be seen that the blood vessels can be matched very well, and the state of change of the lesion can be seen in detail. Thus, automatic correction can be performed with high accuracy and images of different eras can be matched.

  FIG. 23 is a diagram illustrating an example of a time-series fundus camera image after the end of the matching process. As a result of correcting and aligning the magnification, angle, and position of six images from 1989 to 2009 based on the 2010 image, despite the transition from analog to digital, the eyeball has been in use for 20 years. It turned out that it rotated (rotated) about 3 degrees clockwise.

  FIG. 24 is a diagram illustrating an example of time-series image analysis after completion of matching, and is a diagram illustrating an enlarged photograph of the affected area based on FIG. 23 and the like. From this figure, it was found that the change in the shape of the nipple and the shape of the blood vessel in the vicinity of the nipple is more prominent than the change in retinal vascular running that was initially assumed. That is, it can be seen that the nipple shape was deformed into an ellipse from 1989 to 1999 and then gradually expanded. Such subtle changes can be compared over time.

  FIG. 25 is a diagram illustrating a time-series image analysis example (2) after the end of the control processing. By defining a rhombus figure for the 2010 image and the 1989 image, and matching the rhombus with the guideline as a guideline It is a figure which shows the example which produced | generated the overlay image like the middle image, and is the figure which looked at the time change of the lesion | superposition by superimposing the Red image and the Green image.

  As described above, after performing preprocessing (trimming, digitization, etc.), position correction / angle correction, etc. for two images taken under different conditions, alignment processing is performed and alignment condition parameters are stored. . By comparing or measuring the form of the corresponding feature location (lesion site, etc.) of each image after correction with this matching condition parameter, etc., the eye disease such as intense myopia changes over time. be able to.

  In the above-described embodiment, the configuration and the like illustrated in the accompanying drawings are not limited to these, and can be appropriately changed within a range in which the effect of the present invention is exhibited. In addition, various modifications can be made without departing from the scope of the object of the present invention. A program for causing a computer to execute the fundus image analysis method and a computer-readable recording medium for recording the program may be used.

  The present invention can be used in a fundus image analysis apparatus.

DESCRIPTION OF SYMBOLS 1 ... Image analysis apparatus, 3 ... Image digitization part, 5 ... Fundus image pre-processing part, 7 ... Fundus image matching part, 11 ... Composite image creation part, 15 ... Correction parameter determination part, 17 ... Correction fundus image data creation part , 19... Storage unit (memory).

Claims (11)

One of a plurality of fundus image data is set as reference fundus image data, and each correction parameter of magnification, angle, and position is set for each fundus image data other than the reference fundus image, and each correction parameter A fundus image matching unit that performs magnification correction, angle correction, position correction, and creates corrected fundus image data,
A composite image creation unit that applies the same image processing to the reference fundus image data and the corrected fundus image data, and generates a composite image obtained by combining the reference fundus image data and the corrected fundus image data;
Based on the composite image created by the composite image creation unit, it is checked whether or not the reference fundus image data and the corrected fundus image data match, and if they do not match, the fundus image matching unit and the composite image A correction parameter determination unit that determines each correction parameter of the magnification, angle, and position for each fundus image data by repeating the matching process with the creation unit;
A fundus image analysis apparatus comprising: a corrected fundus image data generation unit that generates at least one corrected fundus image data based on the determined correction parameter. Furthermore, an image digitizing unit that creates digital image data from an analog image among a plurality of fundus images,
For each of the digital image data, a region corresponding to the fundus is trimmed, and scaling processing is performed with the same vertical and horizontal magnification parameters so that the image data is configured with a predetermined number of pixels. The fundus image analysis apparatus according to claim 1, further comprising a fundus image pre-processing unit that converts the data into data. The fundus image preprocessing unit
The digital image data is displayed on a screen, and an area corresponding to the fundus is indicated by a specific graphic on the displayed image, thereby setting the same vertical / horizontal magnification parameter for the scaling process. Item 3. A fundus image analysis apparatus according to Item 2. The fundus image matching unit includes:
By displaying each of the reference fundus image data and the corrected fundus image data on a display screen, displaying a figure so as to substantially follow the outline of the eyeball, and adjusting the size of the area of the figure to be the same. The fundus image analysis apparatus according to claim 2, wherein the same vertical and horizontal magnification parameters are set. The fundus image matching unit includes:
Each of the reference fundus image data and the corrected fundus image data is displayed on the screen, and the parallelism of the first and second reference lines respectively displayed at corresponding positions on both displayed images is the same. 4. The rotation angle of the operation unit is set as an angle correction parameter by instructing the operation unit to rotate the operation unit. 4. The fundus image analysis apparatus according to the item. The fundus image matching unit includes:
The reference fundus image data and the corrected fundus image data are displayed on the screen, and the first and second reference points respectively displayed at corresponding positions on both displayed images are at the same position. The position moved in the operation unit is set as a correction parameter for the position by instructing the operation unit to move in parallel as described in any one of claims 1 to 4. The fundus image analysis apparatus according to the item. The composite image creation unit
Selecting any one or two primary colors among the three RGB primary colors, selecting two different primary color sets, and assigning one of the selected primary color sets to the reference fundus image data converted into a monochrome image; The other of the selected primary color sets is assigned to the corrected fundus image data converted into a monochrome image, and the reference fundus image data colored with each primary color set and the corrected fundus image data are combined to generate a color composite image. The fundus image analysis apparatus according to claim 1, wherein the fundus image analysis apparatus is created. The composite image creation unit includes gray scale conversion, negative / positive inversion, and predetermined image processing.
The fundus image analysis apparatus according to any one of claims 1 to 7, wherein gradation expansion processing is performed.   9. The fundus image analysis apparatus according to claim 1, wherein the fundus image matching unit performs angle correction after magnification correction. One of a plurality of fundus image data is set as reference fundus image data, and each correction parameter of magnification, angle, and position is set for each fundus image data other than the reference fundus image, and each correction parameter A fundus image matching step for performing correction of the magnification, angle correction, position correction, and generating corrected fundus image data,
Composite image creation for converting the reference fundus image data and the corrected fundus image data into a monochrome image by applying the same image processing to create a composite image by combining the reference fundus image data and the corrected fundus image data Steps,
Based on the composite image created in the composite image creation step, it is checked whether or not the reference fundus image data and the corrected fundus image data match, and if they do not match, the fundus image alignment step and the composite image A correction parameter determination step for determining each correction parameter of the magnification, angle, and position for each fundus image data by repeating the matching process in the creation step;
And a corrected fundus image data creation step of creating at least one corrected fundus image data based on the determined correction parameter.   A program for causing a computer to execute the fundus image analysis method according to claim 10.

Images