JP7623683B2 - Image processing device, image processing program, and image processing method - Google Patents

Description

The present invention relates to an image processing device, an image processing program, and an image processing method.

For early detection of vascular disorders in the brain, medical devices that visualize the inside of the human body, such as CT (Computed Tomography) and MRI (Magnetic Resonance Imaging), are used. However, these medical devices are expensive. This means that the burden of capital investment is large, and only a limited number of hospitals have these medical devices. In addition, medical costs are high, which means that patients bear a large medical expense burden.

The brain is the same organ as the eye in terms of development. Therefore, the state of blood vessels in the brain can be inferred from the state of retinal blood vessels at the fundus. Therefore, fundus examinations, which do not require the aforementioned medical equipment, have been used to diagnose the risk of vascular disorders in the brain. In recent years, with the revision of the third stage of specific health checkups, fundus examinations have been incorporated into internal medicine examination items. However, fundus examinations depend heavily on the experience and skill of the examining physician. For this reason, it is difficult for internists who are not specialists to perform accurate fundus examinations. Therefore, there is a demand for the automation of fundus examinations. To automate fundus examinations, image processing technology is required to accurately extract blood vessels from images of the fundus.

Image processing techniques have been proposed to extract blood vessels from captured images (see, for example, Non-Patent Document 1).

The technology disclosed in Non-Patent Document 1 selects candidate solutions for blood vessel walls by searching using an energy function, and extracts the candidate solution with the greatest similarity to the blood vessel wall as the blood vessel contour. However, the visibility of fundus images is poor, and the accuracy of extraction of blood vessel contours using the technology disclosed in Non-Patent Document 1 is not necessarily good. Therefore, there is a demand for technology to correct blood vessel contours extracted from captured images.

Yuki Okami and 3 others, "Calculation of the diameter of the fundus blood vessels in cataract patients", Proceedings of the 50th Japan Ergonomics Society Chugoku-Shikoku Branch Conference, 2017/12/9, pp. 49-50

The present invention aims to provide a technique for correcting the contours of blood vessels extracted from captured images.

The image processing device according to the present invention is an image processing device that corrects a blood vessel contour line generated based on a captured image in which the blood vessel is captured, the contour line including a first contour line indicating the contour line on one side in the width direction of the blood vessel and a second contour line indicating the contour line on the other side in the width direction, and is characterized by having: a storage unit that stores the captured image and position information of the contour pixels corresponding to the blood vessel contour line captured in the captured image in association with each other; a contour line selection unit that selects one of the first contour line and the second contour line as a target contour line for correction; an extraction unit that extracts multiple contour pixels as extracted pixels from the contour pixels corresponding to the target contour line based on the position information of the contour pixels; a pixel selection unit that selects two extracted pixels having a predetermined positional relationship from the multiple extracted pixels as selected pixels; a correction pixel determination unit that determines a correction pixel corresponding to the target contour line after correction from among the pixels constituting the captured image based on pixel information of each of multiple reference pixels arranged at a predetermined position from the two selected pixels for each of the two selected selected pixels; and a correction contour line identification unit that identifies the target contour line after correction based on the position information of the correction pixels.

According to the present invention, it is possible to correct the contours of blood vessels extracted from captured images.

1 is a network connection diagram showing an embodiment of an image processing apparatus according to the present invention; 1 is a functional block diagram showing an embodiment of an image processing apparatus according to the present invention; 3 is a schematic diagram showing an example of a captured image stored in a storage unit included in the image processing device of FIG. 2; 1 is a flowchart illustrating an embodiment of an image processing method according to the present invention. 5 is a flowchart of a primary correction process included in the image processing method of FIG. 4 . 6A to 6C are schematic diagrams showing captured images at each stage of the primary correction process in FIG. 5 . 6 is a schematic diagram showing a captured image in the middle of the primary correction process of FIG. 5 ; 6 is a schematic diagram showing an example of the luminance value of a reference pixel in the first correction process of FIG. 5 and the amount of change therein; 5 is a flowchart of a secondary correction process included in the image processing method of FIG. 4 . 9A to 9C are schematic diagrams showing captured images at each stage of the secondary correction process in FIG. 8 . 9 is a schematic diagram showing a virtual circle generated by the secondary correction process in FIG. 8 . FIG. 11 is a functional block diagram showing another embodiment of an image processing device according to the present invention. 5 is a flowchart illustrating another embodiment of an image processing method according to the present invention. 13 is a flowchart of a primary correction process included in the image processing method of FIG. 12 . 14A to 14C are schematic diagrams showing captured images at each stage of the primary correction process in FIG. 13 . 14A and 14B are schematic diagrams showing captured images in the middle of the primary correction process of FIG. 13;

Below, with reference to the drawings, we will explain embodiments of an image processing device (hereinafter referred to as "the device"), an image processing program (hereinafter referred to as "the program"), and an image processing method (hereinafter referred to as "the method") according to the present invention.

The present invention corrects the contour line of a blood vessel generated based on a captured image of the blood vessel. Specifically, the present invention determines pixels (hereinafter referred to as "correction pixels") that correspond to the corrected contour line from among the pixels that make up the captured image based on pixel information of the pixels that make up the captured image, and identifies the corrected contour line based on position information of the correction pixels. The present invention also corrects the position information of the correction pixels based on the curvature of each correction pixel.

"Captured image" includes, for example, fundus images captured by a fundus camera, X-ray images captured by an X-ray imaging device, CT images captured by a CT device, as well as images captured by a normal optical camera. In the present invention, the captured image includes at least blood vessels as the subject. As described above, in this embodiment, the captured image is a fundus image.

"Blood vessels" includes not only human blood vessels but also blood vessels of animals other than humans. In this embodiment, the blood vessels are blood vessels in the fundus of a human eye.

A "contour line" is a line that traces the contour of a blood vessel captured in a captured image. In this embodiment, the contour line is generated, for example, by performing known image processing on the captured image. Here, since blood vessels are tubular bodies having a width, the contour line of a single blood vessel captured in a captured image includes a contour line that indicates the contour of one side in the width direction of the blood vessel (hereinafter referred to as a "first contour line"), and a contour line that indicates the contour of the other side in the same width direction (hereinafter referred to as a "second contour line").

"Pixel information" is information for each pixel. In this embodiment, pixel information includes the luminance value of the pixel and the amount of change in luminance value between adjacent pixels (luminance gradient).

"Position information" is information about the position of a pixel, i.e., information indicating the coordinates of the pixel.

●Image processing device (1)●
First, a first embodiment of the present device will be described.

FIG. 1 is a network connection diagram showing an embodiment of the present device.
The figure shows that the device 1 is connected to an information storage device 2 via a network (communication line) N that uses a wired communication method or a wireless communication method.

This device 1 corrects the contours of blood vessels captured in the captured image i1 based on pixel information of the pixels that make up the captured image i1 (see FIG. 3; the same applies below). The specific configuration and operation of this device 1 will be described later.

The information storage device 2 stores the captured image i1 in association with position information of pixels (hereinafter referred to as "contour pixels") corresponding to the contour of the blood vessel captured in the captured image i1. The information storage device 2 is, for example, an information storage server, a NAS, or a personal computer.

The network N is, for example, a communication network such as the Internet, a mobile communication network, a LAN (Local Area Network), a WAN (Wide Area Network), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

Configuration of the Image Processing Device (1) FIG. 2 is a functional block diagram showing an embodiment of the device 1. As shown in FIG.

The present device 1 is realized, for example, by a personal computer. The present program runs on the present device 1, and the present program cooperates with the hardware resources of the present device 1 to realize the present method.

Here, by having a computer (not shown) execute this program, the program can cause the computer to function in the same manner as the device 1, and cause the computer to execute the method.

The device 1 includes a communication unit 11, a memory unit 12, and a control unit 13.

The communication unit 11 acquires the captured image i1 from the information storage device 2 via the network N. The communication unit 11 is composed of, for example, a communication module and an antenna. The captured image i1 is stored in the storage unit 12.

The storage unit 12 stores information necessary for the device 1 to execute the method described below. The storage unit 12 is composed of, for example, a recording device such as a hard disk drive (HDD) or solid state drive (SSD) provided in the device 1, a volatile memory such as a random access memory (RAM), and/or a portable storage medium such as a flash memory. In this embodiment, the storage unit 12 stores the captured image i1 received in advance from the information storage device 2 in association with position information of the contour pixels of the blood vessels captured in the captured image i1.

The control unit 13 controls the operation of the entire device 1 and executes the method described below. The control unit 13 is composed of, for example, a CPU (Central Processing Unit) provided in the device 1, a volatile memory such as a RAM that functions as a working area for the CPU, and a non-volatile memory such as a ROM (Read Only Memory) that stores various information such as the program. The control unit 13 includes a contour line selection unit 131, an extraction unit 132, a pixel selection unit 133, a correction pixel determination unit 134, a secondary correction unit 135, and a correction contour line identification unit 136.

The contour line selection unit 131 selects one of the first contour line and the second contour line from among the contour lines as the contour line to be corrected (hereinafter referred to as the "target contour line"). The specific operation of the contour line selection unit 131 will be described later.

Based on the position information of the contour pixels, the extraction unit 132 extracts multiple contour pixels from among the contour pixels that make up the target contour (corresponding to the target contour) as extracted pixels. The specific operation of the extraction unit 132 will be described later.

The pixel selection unit 133 selects two extracted pixels having a predetermined positional relationship as selected pixels from among the multiple extracted pixels. The specific operation of the pixel selection unit 133 and the predetermined positional relationship will be described later.

For each of the two selected pixels, the correction pixel determination unit 134 determines a correction pixel that corresponds to the target contour after correction from among the pixels constituting the captured image i1, based on pixel information of each of a plurality of pixels (hereinafter referred to as "reference pixels") that are located at positions the same distance from the two selected pixels. The specific operation of the correction pixel determination unit 134 will be described later.

"Positions at the same distance" is an example of a predetermined position in the present invention, and means a position in the captured image i1 where the linear distance from the two selected pixels is the same. Here, since the captured image i1 is composed of multiple pixels, there may not be a single pixel at a position where the linear distance from the two selected pixels is exactly the same. Therefore, pixels arranged at the same distance may include not only pixels arranged at positions where the linear distance from the two selected pixels is the same, but also pixels adjacent to the same position (pixels where the linear distance from the two selected pixels is "almost the same"). That is, for example, if there is an even number of pixels (e.g., 10) between the two selected pixels, a pixel located 6 pixels away from one selected pixel (a pixel located 4 pixels away from the other selected pixel) may be a pixel where the linear distance from the two selected pixels is the same.

The secondary correction unit 135 corrects the position information of the correction pixel. The specific operation of the secondary correction unit 135 will be described later.

The correction contour line specification unit 136 specifies the target contour line after correction based on the position information of the correction pixel. The specific operation of the correction contour line specification unit 136 will be described later.

Operation of the Image Processing Device (1) Next, the operation of the device 1, that is, the embodiment of the method executed by the device 1, will be described with reference to FIG.

In the following description, it is assumed that the device 1 acquires the captured image i1 and position information of the contour pixels corresponding to the contour of the blood vessel captured in the captured image i1 from the information storage device 2 via the network N, and stores them in association with each other in the storage unit 12. It is also assumed that the captured image i1 used in the following description has a contour superimposed thereon. Furthermore, in the following description, it is assumed that the contour is given the symbol "L", the first contour is given the symbol "L1", the second contour is given the symbol "L2", and the target contour is given the symbol "Lt". Furthermore, in the following description, it is assumed that each pixel is given a symbol beginning with "P" in order to distinguish between them.

FIG. 3 is a schematic diagram showing an example of a captured image i1.
For ease of explanation, the figure shows an enlarged view of a portion of one blood vessel, and illustrates that in the captured image i1, a portion of the captured blood vessel is superimposed with a contour line L (a first contour line L1 and a second contour line L2).

Figure 4 is a flowchart illustrating an embodiment of the method.

This method performs a first correction process (S1) and a second correction process (S2).

First, the device 1 executes a first correction process (S1).

FIG. 5 is a flowchart of the primary correction process (S1).
FIG. 6 is a schematic diagram showing a captured image i1 at each stage of the primary correction process (S1).

The "primary correction process (S1)" is a process for correcting the position information of the contour pixels corresponding to the contour line L associated with the captured image i1 based on the pixel information of the pixels constituting the captured image i1.

First, the contour line selection unit 131 selects either the first contour line L1 or the second contour line L2 as the target contour line Lt (S101). That is, for example, the contour line selection unit 131 selects the target contour line Lt based on the positional relationship between the first contour line L1 and the second contour line L2 in the captured image i1 (for example, the distance from the origin (0,0) of the coordinates of the pixels constituting the captured image i1).

The contour line selection unit of the present invention may select as the target contour line a contour line selected by the user of the device using a selection means (not shown), or may select the target contour line based on its relative positional relationship with the blood vessel (such as up, down, left, right, etc. with respect to the blood vessel).

Next, the correction pixel determination unit 134 performs a mask process on pixels included in a predetermined area including a contour line L (in this embodiment, the second contour line L2) that is not selected as the target contour line Lt (S102), and excludes the predetermined area from the target of correction pixels described below. Here, the correction pixel determination unit 134 performs, for example, a known expansion process on the second contour line L2 to define the area (predetermined area) covered by the expanded second contour line L2.

In addition, the specified area that is excluded from the target of correction pixels in the masking process may be a fixed area based on a contour that is not selected as the target contour. Furthermore, the masking process only needs to be performed before the amount of change in the luminance value of the reference pixel is calculated, which will be described later.

Then, based on the position information of the contour pixels corresponding to the target contour line Lt among the pixels constituting the captured image i1, the extraction unit 132 extracts multiple contour pixels from the contour pixels corresponding to the target contour line Lt as extracted pixels P1(n) (S103). That is, for example, the extraction unit 132 extracts a fixed number of contour pixels at approximately equal intervals from the multiple contour pixels as extracted pixels P1(n). In the following description, it is assumed that the extraction unit 132 extracts six extracted pixels P1(1) to P1(6).

In addition, the extraction unit in the present invention may extract all of the contour pixels corresponding to the target contour as extracted pixels.

Next, the pixel selection unit 133 selects two adjacent extracted pixels P1(n) and P1(n+1) from among the extracted pixels P1(1) to P1(6) as selected pixels P2(1) and P2(2) (S104). That is, for example, when the six extracted pixels P1(1) to P1(6) are arranged in numerical order, the pixel selection unit 133 selects the extracted pixels P1(1) and P1(2) as selected pixels P2(1) and P2(2).

Here, in this embodiment, the "adjacent" positional relationship is an example of a predetermined positional relationship in the present invention.

FIG. 7 is a schematic diagram showing a captured image i1 in the middle of the primary correction process (S1).
For ease of explanation, the figure shows an enlarged view of the vicinity of two selected pixels P2(1) and P2(2).

Returning to Figures 5 to 7.
Next, the correction pixel determination unit 134 generates a virtual line segment Lm connecting the two selected pixels P2(1) and P2(2) and generates a virtual perpendicular line Ln perpendicular to the virtual line segment Lm at the midpoint of the virtual line segment Lm (S105). Here, the length of the virtual perpendicular line Ln is preset to a predetermined length (for example, 9 pixels with the midpoint as the center pixel).

The length of the virtual perpendicular line may be set, for example, based on the distance between the first contour line and the second contour line (i.e., the width or caliber of the blood vessel). That is, for example, the length of the virtual perpendicular line may be set to less than twice that distance.

Then, the correction pixel determination unit 134 identifies, among the pixels constituting the captured image i1, a number of pixels corresponding to the virtual perpendicular line Ln as reference pixels P3(n) (S106). That is, for example, if the length of the virtual perpendicular line Ln is nine pixels, nine pixels are identified as reference pixels P3(1) to P3(9). Here, the pixels corresponding to the virtual perpendicular line Ln are pixels located at positions that are approximately the same distance from the two selected pixels P2(1) and P2(2). That is, the pixels corresponding to the virtual perpendicular line Ln are pixels located at predetermined positions in the present invention.

Then, the correction pixel determination unit 134 acquires the luminance value of each of the reference pixels P3(1) to P3(9) from the storage unit 12, and acquires the amount of change in luminance value for each of the two adjacent reference pixels P3(n) and P3(n+1) based on the luminance values of each of the multiple reference pixels P3(1) to P3(9) (S107). That is, for example, the correction pixel determination unit 134 calculates the primary luminance gradient and secondary luminance gradient for each of the reference pixels P3(1) to P3(9).

The correction pixel determination unit in the present invention may calculate only the primary luminance gradient, or may calculate the amount of change in luminance value using other calculations.

Figure 8 is a schematic diagram showing an example of the pixel value of each reference pixel P3(n) and the amount of change (first-order gradient) of the pixel value. The figure shows that the reference pixel P3(6) has the largest amount of change in luminance value.

Returning to Figures 5 to 8.
Next, the correction pixel determination unit 134 determines the reference pixel P3(n) with the largest change in luminance value among the multiple reference pixels P3(1) to P3(9) as the correction pixel P4(n) (S108). Usually, in the captured image i1, the blood vessels, which are the subject of the image, are captured darker than the background. Therefore, the pixels with the largest change in luminance value can be regarded as pixels corresponding to the edge of the blood vessel.

Here, the virtual perpendicular line Ln may pass through the contour L (second contour L2) that is not selected as the target contour Lt. Therefore, the reference pixel P3(n) with the largest change in pixel value may also be located near the second contour L2. As a result, the correction pixel determination unit 134 may determine the reference pixel P3(n) near the second contour L2 as the correction pixel P4(n). However, in this embodiment, in the mask process of the process (S102), the reference pixel P3(n) near the second contour L2 is excluded from the target of the correction pixel P4(n). Therefore, in the process (S108), the pixel with the largest change in luminance value is the single reference pixel P3(n).

The correction pixel determination unit in the present invention does not need to perform the masking process described above. In this case, the correction pixel determination unit in the present invention, for example, determines the pixel closest to the target contour as the correction pixel out of two reference pixels with a large amount of change in pixel value.

Then, the control unit 13 (for example, the correction pixel determination unit 134) determines whether there is an extracted pixel P1(n) that has not been selected as the selected pixel P2(1) or P2(2) (S109). When there is an extracted pixel P1(n) that has not been selected as the selected pixel P2(1) or P2(2) ("Y" in S109), the primary correction process (S1) returns to the process (S103). That is, for example, the pixel selection unit 133 selects, as the next selected pixels P2(1) and P2(2), the extracted pixel P1(3) that is adjacent to the extracted pixel P1(2) that has already been selected among the extracted pixels P1(3) to P1(6) that have not been selected, and the extracted pixel P1(2) that is adjacent to the extracted pixel P1(3) among the extracted pixels P1(1) and P1(2) that have already been selected.

On the other hand, when there is no extracted pixel P1(n) that has not been selected as the selected pixel P2(1) or P2(2) ("N" in S109), the correction pixel determination unit 134 stores the position information of all correction pixels P4(1) to P4(5) in the memory unit 12 as the position information of the correction pixel group in association with the captured image i1 (S110).

The control unit 13 (e.g., the correction pixel determination unit 134) then determines whether or not there is a contour line L that has not been selected as the target contour line Lt (S111). If there is a contour line L that has not been selected as the target contour line Lt ("Y" in S111), the primary correction process (S1) returns to the process (S101). On the other hand, if there is no contour line L that has not been selected as the target contour line Lt ("N" in S111), the primary correction process (S1) ends.

Secondary Correction Process Next, the device 1 executes a secondary correction process (S2).

FIG. 9 is a flowchart of the secondary correction process (S2).
FIG. 10 is a schematic diagram showing the captured image i1 at each stage of the secondary correction process (S2).
FIG. 10 shows a vein crossing an artery such that part of the vein is obscured by part of the artery.

The "secondary correction process (S2)" is a process for correcting the position information of the correction pixel P4(n) determined in the primary correction process (S1). As described above, blood vessels are imaged darker than the background in the captured image i1, so the correction pixel P4(n) determined by the primary correction process (S1), which corrects with the amount of change in luminance value, corresponds approximately exactly to the contour of the blood vessel. However, for example, at the crossing of blood vessels where an artery and a vein cross, the difference between the luminance value of the artery and the luminance value of the vein is small. Therefore, for example, the correction pixel P4(n) of the vein at the crossing may protrude toward the artery or recess into the vein. The secondary correction process (S2) is a process for correcting the position information of the correction pixel P4(n) determined in such a region where the difference in luminance value is small, based on a factor (curvature) different from the luminance value.

First, the secondary correction unit 135 reads out the group of correction pixels to be subjected to secondary correction from the storage unit 12 (S201). In the following explanation, it is assumed that the group of correction pixels is composed of five correction pixels P4(11) to P4(15).

Next, the secondary correction unit 135 calculates the curvature corresponding to each of the three correction pixels P4(12) to P4(14) among the correction pixels P4(11) to P4(15) constituting the correction pixel group, excluding the correction pixel P4(11) serving as the start point and the correction pixel P4(15) serving as the end point. Here, the start point and the end point are each determined in advance, for example, based on the determination order of the correction pixel P4(n).

Specifically, the secondary correction unit 135 selects, from among the correction pixels P4(12) to P4(14), the correction pixel P4(12) that is closest to the starting point and whose curvature has not yet been calculated, as the calculation pixel P5(1) (S202). Next, the secondary correction unit 135 selects the two correction pixels P4(11) and P4(13) that are located on either side of the calculation pixel P5(1) as the calculation auxiliary pixels P5(2) and P5(3) (S203).

Next, the secondary correction unit 135 generates a virtual circle C that connects the calculated pixel P5(1) and the two calculated auxiliary pixels P5(2) and P5(3) (S204).

FIG. 11 is a schematic diagram showing the imaginary circle C. As shown in FIG.
For ease of explanation, FIG. 1 also shows parameters used to calculate the radius "R" of the imaginary circle C.

Returning to Figures 9 to 11.
Next, the secondary correction unit 135 calculates the radius "R" of the imaginary circle C by using the following equation (1) (S205).

ここで、
「||・||」はベクトルのノルムを意味する（以下同じ）。
「ｐ_ｉ－１」は、算出補助画素Ｐ５（２）の座標である。
「ｐ_ｉ＋１」は、算出補助画素Ｐ５（３）の座標である。
「ｑ」は、算出補助画素Ｐ５（２）と仮想円Ｃの中心点「Ｏ」とを結ぶ仮想線Ｌｏと、仮想円Ｃとの交点「Ｑ」の座標である。
「θ」は、算出補助画素Ｐ５（３）と交点「Ｑ」とを結ぶ仮想線Ｌｐと、仮想線Ｌｏと、の間の角度である。

Where:
``||·||'' means the norm of a vector (same below).
"p _i-1 " is the coordinate of the calculated auxiliary pixel P5(2).
"p _i+1 " is the coordinate of the calculated auxiliary pixel P5(3).
"q" is the coordinate of the intersection "Q" between the imaginary line Lo connecting the calculated auxiliary pixel P5(2) and the center point "O" of the imaginary circle C and the imaginary circle C.
"θ" is the angle between the virtual line Lp that connects the calculation auxiliary pixel P5(3) and the intersection point "Q" and the virtual line Lo.

Next, the secondary correction unit 135 calculates the curvature "k _i " of the virtual circle C by using the following equation (2) (S206).

The calculated curvature "k _i " is stored in the storage unit 12 in association with the corresponding calculation pixel P5(1) (correction pixel P4(12)).

Next, the secondary correction unit 135 determines whether or not there is a correction pixel P4(n) for which the curvature "k _i " has not been calculated (S207). When there is a correction pixel P4(n) for which the curvature "k _i " has not been calculated ("Y" in S207), the secondary correction process (S2) returns to the process (S202).

On the other hand, when there is no corrected pixel P4(n) for which the curvature "k _i " has not been calculated ("N" in S207), the secondary correction unit 135 calculates the average value "V _ave. " and standard deviation "σ" of all the curvatures "k _{i "} (S208).

Next, the secondary correction unit 135 determines whether or not there is a correction pixel P4(n) with a relatively large curvature "k _i " based on the calculated average value "V _{ave. "} and standard deviation "σ" (S209). That is, for example, the secondary correction unit 135 determines whether or not there is a correction pixel P4(n) with a curvature "k _i " that deviates from "±2σ" of a normal distribution centered on the average value "V ave." That is, in this embodiment, "the curvature "k _i " is relatively large" means that the curvature "k _i " deviates from "±2σ" of a normal distribution centered on the average value "V _ave. ".

When there is a correction pixel P4(n) with a relatively large curvature "k _i "("Y" in S209), the secondary correction unit 135 selects, from among the correction pixels P4(n), the correction pixel P4(n) that is closest to the starting point and has not been selected as the position correction pixel P6(n), as the position correction pixel P6(1) (S210). In the following description, it is assumed that the correction pixel P4(13) has been selected as the position correction pixel P6(1).

Next, the secondary correction unit 135 selects two correction pixels P4(12) and P4(14) located on either side of the position correction pixel P6(1) as reference pixels P7(1) and P7(2) (S211).

Next, the secondary correction unit 135 calculates the position information of the pixel Po corresponding to the midpoint between the reference pixels P7(1) and P7(2) (hereinafter referred to as the "midpoint pixel") (S212).

Then, the secondary correction unit 135 corrects the position information of the position correction pixel P6(1) to the position information of the midpoint pixel Po (S213). That is, the secondary correction unit 135 moves the position of the correction pixel P4(13) corresponding to the position correction pixel P6(1) to the position of the midpoint pixel Po. That is, the secondary correction unit 135 corrects the position information of the position correction pixel P6(1) based on the position information of the reference pixels P7(1) and P7(2).

Next, the secondary correction unit 135 determines whether or not there is a correction pixel P4(n) that has not been selected as the position correction pixel P6(n) (S214). When there is a correction pixel P4(n) that has not been selected as the position correction pixel P6(n) ("Y" in S214), the secondary correction process (S2) returns to process (S210).

On the other hand, when there is no correction pixel P4(n) with a relatively large curvature "k _i "("N" in S209), or when there is no correction pixel P4(n) that has not been selected as the position correction pixel P6(n) ("N" in S214), the secondary correction unit 135 stores the position information of the correction pixel P4(n) whose position information has not been corrected and the position information of the correction pixel P4(n) whose position information has been corrected in the memory unit 12 as position information of the pixels corresponding to the corrected contour line Lx (S215).

Next, the correction contour line identification unit 136 identifies the corrected contour line Lx based on the position information of the pixels corresponding to the corrected contour line Lx (S216).

In this way, in the first correction process (S1), the device 1 determines a correction pixel P4(n) based on pixel information of an extracted pixel P1(n) extracted from the contour L to correct the position information of the contour L, and in the second correction process (S2), further corrects the position information of the correction pixel P4(n) based on the curvature "k _i " of the correction pixel P4(n). That is, the device 1 corrects the position information of the contour L in two stages based on two different pieces of information, the pixel information and the curvature "k _i ". As a result, the device 1 can correct the contour L of the blood vessel extracted from the captured image i1 with high accuracy.

● Summary (1)
According to the embodiment described above, the device 1 includes a contour selection unit 131 that selects either the first contour line L1 or the second contour line L2 as a target contour line Lt to be corrected; an extraction unit 132 that extracts a plurality of contour pixels as extracted pixels P1(n) from among the contour pixels corresponding to the target contour line Lt based on position information of the contour pixels; a pixel selection unit 133 that selects two extraction pixels P1(n) having a predetermined positional relationship from the plurality of extraction pixels P1(n) as selected pixels P2(1), P2(2); a correction pixel determination unit 134 that determines, for each of the two selected selection pixels P2(1), P2(2), a correction pixel P4(n) corresponding to the corrected target contour line Lt from among the pixels constituting the captured image i1 based on pixel information of each of a plurality of reference pixels P3(n) that are arranged at predetermined positions from the two selected pixels P2(1), P2(2); and a correction contour identification unit 136 that identifies the target contour line Lt after correction based on the position information of the correction pixel P4(n). According to this configuration, the device 1 can determine one correction pixel P4(n) for each of two extracted pixels P1(n) based on the pixel information (amount of change in luminance value) of the reference pixel P3(n).Then, the device 1 can specify the corrected target contour line Lt (contour line L) based on the position information of the determined correction pixel P4(n).In other words, the device 1 can correct the contour line L of the blood vessel extracted from the captured image i1.

According to the embodiment described above, the pixel information includes the luminance value of the reference pixel P3(n). The correction pixel determination unit 134 obtains the amount of change in luminance value between two adjacent reference pixels P3(n) based on the luminance values of each of the reference pixels P3(n), and determines the reference pixel P3(n) with the largest amount of change as the correction pixel P4(n). In general, in the captured image i1, the blood vessels, which are the subject of the image, are captured darker than the background. Therefore, pixels with a large amount of change in pixel value can be considered to be pixels corresponding to the edge of the blood vessel. With this configuration, the device 1 can easily identify pixels corresponding to the edge of the blood vessel (or, if relevant, pixels of the corrected contour line Lx).

Furthermore, according to the embodiment described above, when the first contour line L1 is selected, the correction pixel determination unit 134 excludes pixels included in a predetermined area including the second contour line L2 from the correction pixel P4(n) target, and when the second contour line L2 is selected, the correction pixel determination unit 134 excludes pixels included in a predetermined area including the first contour line L1 from the correction pixel P4(n) target. With this configuration, when calculating the amount of change in luminance value in the virtual perpendicular line Ln, the change in luminance value near the contour line L that is not selected as the target contour line Lt is not affected. As a result, the pixel with the largest amount of change in luminance value becomes the single reference pixel P3(n), and can be easily identified.

Furthermore, according to the embodiment described above, the correction pixel determination unit 134 defines the predetermined area by performing an expansion process on the first contour line L1 or the second contour line L2. With this configuration, the device 1 can easily determine the predetermined area to be masked.

Furthermore, according to the embodiment described above, the device 1 has a secondary correction unit 135 that corrects the position information of the correction pixel P4(n). The secondary correction unit 135 calculates the curvature corresponding to each of the multiple correction pixels P4(n) and corrects the position information of the correction pixel P4(n) based on the curvature of each correction pixel P4(n). With this configuration, the device 1 can further correct the position information of the correction pixel P4(n) in a portion where there is no significant difference in the amount of change in luminance value and where the position information of the correction pixel P4(n) is likely to be erroneous. In other words, the device 1 can correct the contour line L with high accuracy. The device 1 can also identify the correction pixel P4(n) whose position information is to be corrected based on the curvature, i.e., the difference in relative position information with respect to other correction pixels P4(n).

Furthermore, according to the embodiment described above, the secondary correction unit 135 selects the correction pixel P4(n) to be corrected as the position correction pixel P6(n) based on the calculated curvature. According to this configuration, the device 1 can identify the correction pixel P4(n) whose relative position information with respect to other correction pixels P4(n) is significantly shifted based on the curvature, and correct only the position information of the correction pixel P4(n). In addition, the secondary correction unit 135 selects the two correction pixels P4(n) located on both sides of the position correction pixel P6(n) as the reference pixels P7(1) and P7(2), and corrects the position information of the position correction pixel P6(n) based on the position information of the two reference pixels P7(1) and P7(2). According to this configuration, the present device 1 can correct the position information of the position correction pixel P6(n) based on the position information of pixels (reference pixels P7(1), P7(2)) whose position information is not shifted from the blood vessel contour more than the position correction pixel P6(n) and whose position information is closer to the correct position of the position correction pixel P6(n). In other words, the present device 1 can correct the blood vessel contour line L extracted from the captured image i1 with high accuracy.

Furthermore, according to the embodiment described above, the secondary correction unit 135 selects the correction pixel P4(n) with a relatively large curvature as the position correction pixel P6(n). With this configuration, the device 1 can identify only the correction pixel P4(n) whose relative position information with respect to other correction pixels P4(n) is significantly shifted based on the curvature, and correct only the position information of the correction pixel P4(n).

Furthermore, according to the embodiment described above, the secondary correction unit 135 selects, from among the multiple correction pixels P4(n), the correction pixel P4(n) for which the curvature is to be calculated as the calculation pixel P5(1), selects the two correction pixels P4(n) arranged on either side of the calculation pixel P5(1) as the calculation auxiliary pixels P5(2) and P5(3), and calculates the curvature of the calculation pixel P5(1) based on the radius of a virtual circle C that virtually connects the calculation pixel P5(1) and the two calculation auxiliary pixels P5(2) and P5(3). According to this configuration, the device 1 can easily calculate the curvature based only on the position information of the correction pixel P4(n).

●Image processing device (2)●
Next, another embodiment (second embodiment) of the present device will be described, focusing on the differences from the first embodiment described above. The present device of the second embodiment differs from the first embodiment in the pixel selection unit and the correction pixel determination unit of the control unit.

Configuration of the Image Processing Device (2) FIG. 12 is a functional block diagram showing another embodiment of the device.

The device 1A includes a communication unit 11, a memory unit 12, and a control unit 13A.

The control unit 13A controls the operation of the entire device 1A and executes the method described below. The physical configuration of the control unit 13A is the same as that of the control unit 13 in the first embodiment. The control unit 13A includes a contour line selection unit 131, an extraction unit 132, a pixel selection unit 133A, a correction pixel determination unit 134A, a secondary correction unit 135, and a correction contour line identification unit 136.

The pixel selection unit 133A selects two extracted pixels having a predetermined positional relationship as selected pixels from among the multiple extracted pixels. The specific operation of the pixel selection unit 133A and the predetermined positional relationship will be described later.

For each of the two selected pixels, the correction pixel determination unit 134A determines a correction pixel that corresponds to the target contour after correction from among the pixels constituting the captured image i1, based on the pixel information of each of a plurality of pixels (reference pixels) that are positioned at approximately the same distance from the two selected pixels. The specific operation of the correction pixel determination unit 134A will be described later.

Operation of the Image Processing Apparatus (2) Next, the operation of the present apparatus 1A, that is, another embodiment of the present method executed by the present apparatus 1A will be described with reference to FIG.

Figure 13 is a flow chart illustrating another embodiment of the method.

This method performs a first correction process (S1A) and a second correction process (S2). In this embodiment, the second correction process (S2) is the same as the second correction process (S2) in the first embodiment.

First, the device 1A executes a first correction process (S1A).

FIG. 14 is a flowchart of the primary correction process (S1A).
FIG. 15 is a schematic diagram showing a captured image i1 at each stage of the first correction process (S1A).

The "primary correction process (S1A)" is a process that corrects the position information of the contour pixels corresponding to the contour line L associated with the captured image i1 based on the pixel information of the pixels that make up the captured image i1.

First, the contour selection unit 131 selects either the first contour L1 or the second contour L2 as the target contour Lt (S101).

Then, based on the position information of the contour pixels corresponding to the target contour line Lt among the pixels constituting the captured image i1, the extraction unit 132 extracts multiple contour pixels from the contour pixels corresponding to the target contour line Lt as extracted pixels P1(n) (S102A). That is, for example, the extraction unit 132 extracts a fixed number of contour pixels at approximately equal intervals from the multiple contour pixels as extracted pixels P1(n). Here, the process (S102A) is the same as the process (S103) in the first embodiment. In the following description, it is assumed that the extraction unit 132 extracts six extracted pixels P1(1) to P1(6).

Then, the pixel selection unit 133A selects, from among the multiple extracted pixels P1(1) to P1(6), the extracted pixel P1(n) that is a candidate for correction as the candidate pixel P8(n) (S103A). In this method, the candidate pixel P8(n) is selected from among the four extracted pixels P1(2) to P1(5), excluding the extracted pixel P1(1) that is the starting point and the extracted pixel P1(6) that is the end point. Specifically, the pixel selection unit 133A selects, from among the extracted pixels P1(2) to P1(5), the extracted pixel P1(2) that is closest to the starting point and has not been selected as the candidate pixel P8(n), as the candidate pixel P8(1).

Then, the pixel selection unit 133A selects the two extracted pixels P1(1) and P1(3) located on either side of the candidate pixel P8(1) as selected pixels P2A(1) and P2A(2) (S104A). That is, the pixel selection unit 133A selects the two extracted pixels P1(n) and P1(n+2) located on either side of the candidate pixel P8(n) (one pixel apart) from among the multiple extracted pixels P1(1) to P1(6) as selected pixels P2A(1) and P2A(2) (S104A).

Here, in this embodiment, the positional relationship of "both sides of candidate pixel P8(n) (every other pixel)" is an example of a predetermined positional relationship in the present invention.

FIG. 16 is a schematic diagram showing a captured image i1 in the middle of the first correction process (S1A).
For ease of explanation, FIG. 1 shows an enlarged view of the vicinity of two selected pixels P2A(1) and PA2(2).

Returning to Figures 14 to 16.
Next, the correction pixel determination unit 134A generates a virtual line segment Lm connecting the two selected pixels P2A(1) and P2A(2) and generates a virtual perpendicular line Ln perpendicular to the virtual line segment Lm at the midpoint of the virtual line segment Lm (S105). Here, the length of the virtual perpendicular line Ln is preset to a predetermined length (for example, 9 pixels with the midpoint as the center pixel).

Then, the correction pixel determination unit 134A identifies, among the pixels constituting the captured image i1, a number of pixels corresponding to the virtual perpendicular line Ln as reference pixels P3(n) (S106). That is, for example, if the length of the virtual perpendicular line Ln is nine pixels, nine pixels are determined as reference pixels P3(1) to P3(9). Here, the pixels corresponding to the virtual perpendicular line Ln are pixels located at positions that are approximately the same distance from the two selected pixels P2A(1) and P2A(2). That is, the pixels corresponding to the virtual perpendicular line Ln are pixels located at predetermined positions in the present invention.

Then, the correction pixel determination unit 134A acquires the luminance value of each of the reference pixels P3(1) to P3(9) from the storage unit 12, and acquires the amount of change in luminance value for each of the two adjacent reference pixels P3(n) and P3(n+1) based on the luminance values of each of the multiple reference pixels P3(1) to P3(9) (S107). That is, for example, the correction pixel determination unit 134 calculates the primary luminance gradient and secondary luminance gradient for each of the reference pixels P3(1) to P3(9).

Next, the correction pixel determination unit 134A determines, from among the multiple reference pixels P3(1) to P3(9), the reference pixel P3(n) that is closest to the target contour and has the largest amount of change, as the correction pixel P4(n) (S108).

The correction pixel determination unit in the present invention may perform a mask process similar to the method in the first embodiment.

Then, the control unit 13A (for example, the correction pixel determination unit 134A) determines whether or not there is an extracted pixel P1(n) that has not been selected as the candidate pixel P8(n) (S109A). When there is an extracted pixel P1(n) that has not been selected ("Y" in S109A), the primary correction process (S1A) returns to the process (S103A). That is, for example, the pixel selection unit 133A selects the extracted pixel P1(3) that is adjacent to the extracted pixel P1(2) that has already been selected, from among the extracted pixels P1(3) to P1(5) that have not been selected, as the next candidate pixel P8(2).

On the other hand, when there is no unselected extracted pixel P1(n) ("N" in S109A), the correction pixel determination unit 134A corrects the position information of all candidate pixels P8(1) to P8(4) (i.e., extracted pixels P1(2) to P1(5)) to the position information of the corresponding correction pixels P4(1) to P4(4) (S110A). In other words, the correction pixel determination unit 134A moves the position of extracted pixel P1(n) corresponding to candidate pixel P8(n) to the position of correction pixel P4(n).

Next, the correction pixel determination unit 134A stores the position information of all correction pixels P4(2) to P4(5) in the storage unit 12 as position information of the correction pixel group in association with the captured image i1 (S111A).

Next, the control unit 13A (e.g., the correction pixel determination unit 134A) determines whether or not there is an unselected contour line L as the target contour line Lt (S112A). If there is an unselected contour line L ("Y" in S112A), the primary correction process (S1A) returns to process (S101). On the other hand, if there is no unselected contour line L ("N" in S112A), the primary correction process (S1A) ends. Here, the process (S112A) is the same as the process (S111) in the first embodiment.

● Summary (2)
According to the embodiment described above, the pixel selection unit 133 selects, from among a plurality of extracted pixels P1(n), an extracted pixel P1(n) that is a candidate for correction as a candidate pixel P8(n), and selects two extracted pixels P1(n) arranged on both sides of the candidate pixel P8(n) as selected pixels P2A(1) and P2A(2). The correction pixel determination unit 134 corrects the position information of the candidate pixel P8(n) to the position information of the correction pixel P4(n). According to this configuration, the present device 1 can correct the position information itself of the extracted pixel P1(n).

● Summary (Other)
In the above-described embodiments, the apparatus 1, 1A executes the secondary correction process (S2). Alternatively, the apparatus does not need to execute the secondary correction process if the contour line can be corrected with a desired accuracy in the primary correction process.

In addition, in the primary correction process, the present device may combine the selection of selected pixels in the first embodiment with the selection of selected pixels in the second embodiment. That is, for example, for extracted pixels P1(1) to P1(6), the present device may select extracted pixel P1(1) corresponding to the start point and extracted pixel P1(2) adjacent to extracted pixel P1(1) as selected pixels, and then select extracted pixel P1(2) and every other extracted pixel P1(4) as selected pixels.

REFERENCE SIGNS LIST 1 Image processing device 12 Storage unit 131 Contour line selection unit 132 Extraction unit 133 Pixel selection unit 134 Correction pixel determination unit 135 Secondary correction unit 136 Correction contour line specification unit 1A Image processing device 133A Pixel selection unit 134A Correction pixel determination unit

Claims (14)

1. An image processing device that corrects a contour line of a blood vessel generated based on a captured image of the blood vessel, comprising:
The contour line is
A first contour line indicating a contour of one side of the blood vessel in a width direction;
A second contour line indicating the other contour in the width direction;
Including,
a storage unit that stores the captured image and position information of contour pixels corresponding to the contour of the blood vessel captured in the captured image in association with each other;
a contour line selection unit that selects one of the first contour line and the second contour line as a contour line to be corrected;
an extracting unit that extracts, as extracted pixels, a plurality of the contour pixels from among the contour pixels corresponding to the target contour line based on position information of the contour pixels;
a pixel selection unit that selects, from among the plurality of extracted pixels, two extracted pixels having a predetermined positional relationship as selected pixels;
a correction pixel determination unit that determines, for each of the two selected selection pixels, a correction pixel corresponding to the target contour after correction from among pixels constituting the captured image, based on pixel information of each of a plurality of reference pixels that are arranged at predetermined positions from the two selected pixels;
a correction contour specifying unit that specifies the target contour after correction based on position information of the correction pixels;
and
The correction pixel determination unit
When the first contour line is selected, pixels included in a predetermined area including the second contour line are excluded from the pixels to be corrected;
When the second contour line is selected, pixels included in a predetermined area including the first contour line are excluded from the pixels to be corrected.
13. An image processing device comprising: The pixel information is
the luminance value of the reference pixel;
Including,
The correction pixel determination unit
obtaining an amount of change in the luminance value between two adjacent reference pixels based on the luminance values of the plurality of reference pixels;
determining, as the correction pixel, the reference pixel having the largest amount of change among the plurality of reference pixels;
2. The image processing device according to claim 1. The pixel selection unit is
selecting two adjacent extracted pixels from the plurality of extracted pixels as the selected pixels;
3. The image processing device according to claim 2. The pixel selection unit is
selecting, from among the plurality of extracted pixels, the extracted pixel that is a candidate for correction as a candidate pixel, and selecting two extracted pixels disposed on both sides of the candidate pixel as the selected pixels;
3. The image processing device according to claim 2. the correction pixel determination unit corrects position information of the candidate pixel to position information of the correction pixel.
5. The image processing device according to claim 4. a secondary correction unit that corrects position information of the correction pixel;
and
The secondary correction unit is
Calculating a curvature corresponding to each of the plurality of correction pixels;
correcting position information of the correction pixels based on the curvature of each of the correction pixels;
6. The image processing device according to claim 1. the secondary correction unit calculates the curvature corresponding to each of the correction pixels, excluding the correction pixels corresponding to a start point and an end point of the target contour after correction, among the plurality of correction pixels;
7. The image processing device according to claim 6 . The secondary correction unit is
selecting the correction pixel to be corrected as a position correction pixel based on the calculated curvature;
Selecting two of the correction pixels disposed on both sides of the position correction pixel as reference pixels;
correcting the position information of the position correction pixel based on the position information of the two reference pixels;
8. The image processing device according to claim 7 . the secondary correction unit selects the correction pixel having a relatively large curvature as the position correction pixel;
9. The image processing device according to claim 8 . the secondary correction unit corrects position information of the position correction pixel to position information of a pixel corresponding to a midpoint of the two reference pixels;
9. The image processing device according to claim 8 . The secondary correction unit is
selecting the correction pixel for calculating the curvature as a calculation pixel from among the plurality of correction pixels;
selecting two of the correction pixels disposed on both sides of the calculation pixel as calculation auxiliary pixels;
calculating the curvature of the calculation pixel based on a radius of a virtual circle virtually connecting the calculation pixel and the two calculation auxiliary pixels;
7. The image processing device according to claim 6 . A computer is caused to function as the image processing device according to claim 1.
2. An image processing program comprising: 1. An image processing method executed by an image processing device for correcting a contour line of a blood vessel generated based on an image of the blood vessel, the method comprising:
The contour line is
A first contour line indicating a contour of one side of the blood vessel in a width direction;
A second contour line indicating the other contour in the width direction;
Including,
The image processing device includes:
a storage unit that stores the captured image and position information of contour pixels corresponding to the contour of the blood vessel captured in the captured image in association with each other;
Equipped with
The image processing device,
selecting one of the first contour line and the second contour line as a contour line to be corrected;
excluding pixels included in a predetermined area including the contour that is not selected as the target contour from the targets of correction pixels corresponding to the target contour after correction;
extracting, as extracted pixels, a plurality of the contour pixels from among the contour pixels corresponding to the target contour line based on position information of the contour pixels;
selecting, as selected pixels, two of the extracted pixels having a predetermined positional relationship from among the plurality of extracted pixels;
determining, for each of the two selected selection pixels, the correction pixel from among pixels constituting the captured image, based on pixel information of each of a plurality of reference pixels arranged at predetermined positions from the two selected pixels;
identifying the corrected target contour based on position information of the correction pixels;
Including,
13. An image processing method comprising: correcting position information of the correction pixel;
Including,
The image processing method according to claim 13 .

Images