JP6132485B2 - Image processing apparatus, operation method of image processing apparatus, and image processing program - Google Patents

Description

  The present invention relates to an image processing apparatus, an image processing method, and an image processing program that determine the presence or absence of an abnormal portion in an image obtained by imaging the inside of a lumen of a living body.

  As image processing for an intraluminal image (hereinafter also simply referred to as an image) acquired by observing the mucous membrane of the large intestine with a magnifying endoscope, for example, Patent Document 1 discloses the form of a glandular opening (pit) on the mucosal surface. A technique for classifying (referred to as a pit pattern) into a plurality of types is disclosed. Specifically, an m × k dimensional feature vector is calculated by applying a Gabor filter with m frequencies and k phase directions to each pixel of the region of interest set in the image, The average and variance of these feature vectors are calculated. Then, the large intestine pit pattern is classified based on the mean and variance of the feature vectors, and it is determined whether or not the large intestine pit pattern is abnormal.

JP 2007-236956 A

  By the way, in recent years, a capsule endoscope that is introduced into a subject and performs imaging while moving by a peristaltic motion is also used as a means for acquiring an intraluminal image. In the case of this capsule endoscope, since it is difficult to control the relative posture of the imaging unit of the capsule endoscope with respect to the subject (for example, the mucosal surface of the small intestine), An image captured in the opposite direction is not always obtained. For this reason, for example, when the imaging direction with respect to the subject (the angle of the imaging unit) is inclined, the distance in the depth direction (imaging distance) to the subject shown in one intraluminal image varies depending on the region in the image. End up.

  On the other hand, fluff, which is a fine structure on the mucosal surface, particularly abnormal fur in an expanded state, differs in appearance (shape) depending on the imaging distance. For this reason, if the above-described image processing is uniformly performed on the intraluminal image acquired by the capsule endoscope without considering the difference in the imaging distance in the image, the abnormal softness reflected in the image is displayed. The problem is that the shape of the hair changes depending on the imaging distance, making it difficult to distinguish the foam area and mucous membrane contour (groove) from abnormal fur, resulting in overdetection of abnormal parts. was there.

  The present invention has been made in view of the above, and it is possible to accurately determine the presence or absence of abnormal fur even in a case where an image includes an area with a different imaging distance to the subject. An object is to provide an image processing apparatus, an image processing method, and an image processing program.

  In order to solve the above-described problems and achieve the object, an image processing apparatus according to the present invention includes an attention area setting unit that sets an attention area in an image, and a pixel having a pixel value higher than that of the surrounding area in the attention area. A linear convex region extracting unit that extracts a linear region that is continuous for a predetermined number of pixels or more, and an in-region curvature feature that calculates a curvature feature amount based on the curvature of one or more arcs along the linear region An amount determining unit; and an abnormality determining unit configured to determine whether an abnormal portion exists in the region of interest based on the distribution of the curvature feature amount.

  An image processing method according to the present invention includes an attention area setting step for setting an attention area in an image, and extracts a linear area in which pixels having a pixel value higher than the surrounding area are continuous for a predetermined number of pixels in the attention area. Based on the distribution of the curvature feature amount, a linear shape convex region extraction step, an in-region curvature feature amount calculation step for calculating a curvature feature amount based on the curvature of one or more arcs along the linear shape region, An abnormality determination step of determining whether or not an abnormal portion exists in the region of interest.

  An image processing program according to the present invention includes an attention area setting step for setting an attention area in an image, and a line-shaped area in which pixels having a pixel value higher than that of the surrounding area are continuous in a predetermined number of pixels in the attention area. Based on the distribution of the curvature feature amount, a linear shape convex region extraction step, an in-region curvature feature amount calculation step for calculating a curvature feature amount based on the curvature of one or more arcs along the linear shape region, An abnormality determining step for determining whether or not an abnormal portion exists in the region of interest is caused to be executed by a computer.

  According to the present invention, a linear region in which pixels having pixel values higher than the surroundings are continuous for a predetermined number or more is extracted, and a curvature feature amount based on the curvature of one or more arcs along the linear region. And the presence / absence of abnormal fur is determined based on the distribution of the curvature feature value, so even if an image has a different imaging distance to the subject, The presence / absence can be accurately determined.

FIG. 1 is a block diagram showing a configuration of an image processing apparatus according to Embodiment 1 of the present invention. FIG. 2 is a flowchart showing the operation of the image processing apparatus shown in FIG. FIG. 3 is a flowchart showing in detail the processing executed by the attention area setting unit shown in FIG. FIG. 4 is a flowchart showing in detail the processing executed by the mucosa region extracting unit shown in FIG. FIG. 5 is an example of creating a histogram indicating the frequency of the hue average value. FIG. 6 is a schematic diagram for explaining a clustering method in which a valley of a histogram is divided into a plurality of clusters. FIG. 7 is a graph showing the result of clustering the histogram shown in FIG. FIG. 8 is a flowchart showing in detail the processing executed by the linear convex region extraction unit shown in FIG. FIG. 9 is a schematic diagram illustrating the top hat conversion. FIG. 10 is a diagram showing a pattern used for searching for an image in the thinning filter processing. FIG. 11 is a flowchart showing in detail the processing executed by the in-region curvature feature quantity calculator shown in FIG. FIG. 12 is a flowchart showing in detail the processing executed by the size feature amount calculation unit shown in FIG. FIG. 13 is a schematic diagram illustrating a section of a linear region. FIG. 14 is a schematic diagram illustrating a method for calculating the curvature of an arc. FIG. 15 is a schematic diagram illustrating processing executed by the curvature calculation unit illustrated in FIG. FIG. 16A is a schematic diagram showing a normal fur model. FIG. 16B is a schematic diagram illustrating a convex region corresponding to a normal fur model. FIG. 17A is a schematic diagram showing an abnormal fur model. FIG. 17B is a schematic diagram illustrating a linear region corresponding to abnormal fur. FIG. 17C is a graph showing the distribution of curvature of the arc along the line-shaped region corresponding to abnormal fur. FIG. 18A is a schematic diagram showing a bubble model. FIG. 18B is a schematic diagram illustrating a linear region corresponding to the bubble region. FIG. 18C is a graph showing the distribution of the curvature of the arc along the line-shaped region corresponding to the bubble region. FIG. 19 is a schematic diagram illustrating a frequency distribution including two axes of the curvature of the arc and the distance information. FIG. 20 is a block diagram illustrating an image processing apparatus according to Modification 1-1 of Embodiment 1. FIG. 21 is a flowchart showing details of processing executed by the linear convex area extracting unit shown in FIG. FIG. 22 is a schematic diagram showing patterns used in the Hilditch thinning algorithm. FIG. 23 is a schematic diagram showing patterns used in the Hilditch thinning algorithm. FIG. 24 is a block diagram showing a configuration of an image processing apparatus according to Embodiment 2 of the present invention. FIG. 25 is a flowchart showing processing executed by the in-region curvature feature quantity calculation unit shown in FIG. FIG. 26 is a flowchart showing processing executed by the shape feature quantity calculation unit shown in FIG. FIG. 27A is a schematic diagram showing an abnormal fur model. FIG. 27B is a schematic diagram illustrating the curvature of an arc along a linear region corresponding to abnormal fur. FIG. 27C is a graph showing the standard deviation of curvature calculated for the line-shaped region corresponding to abnormal fur. FIG. 28A is a schematic diagram showing a bubble model. FIG. 28B is a schematic diagram showing the curvature of an arc along a linear region corresponding to abnormal fur. FIG. 28C is a graph showing the standard deviation of curvature calculated for the line-shaped region corresponding to the bubble region. FIG. 29 is a block diagram showing a configuration of an image processing apparatus according to Embodiment 3 of the present invention. FIG. 30 is a flowchart showing processing executed by the in-region curvature feature quantity calculation unit shown in FIG. FIG. 31 is a flowchart showing processing executed by the direction feature amount calculation unit shown in FIG. FIG. 32A is a schematic diagram showing an abnormal fur model. FIG. 32B is a schematic diagram illustrating a linear region corresponding to abnormal fur. FIG. 32C is a schematic diagram illustrating the direction of the center of curvature of the arc along the line-shaped region corresponding to abnormal fur. FIG. 32D is a graph showing the frequency of the curvature center direction for each gradient direction in abnormal fur. FIG. 33A is a schematic diagram showing a bubble model. FIG. 33B is a schematic diagram illustrating a linear region corresponding to the bubble region. FIG. 33C is a schematic diagram illustrating the direction of the center of curvature of the arc along the line-shaped region corresponding to the bubble region. FIG. 33D is a graph showing the frequency of the curvature center direction for each gradient direction in the bubble region. FIG. 34 is a block diagram showing a configuration of an image processing apparatus according to Embodiment 4 of the present invention. FIG. 35 is a flowchart showing processing executed by the attention area setting unit shown in FIG. FIG. 36 is a flowchart illustrating the operation of the image processing apparatus according to the modified example 4-1 of the fourth embodiment.

  Hereinafter, an image processing apparatus, an image processing method, and an image processing program according to embodiments of the present invention will be described with reference to the drawings. Note that the present invention is not limited by these embodiments. Moreover, in description of each drawing, the same code | symbol is attached | subjected and shown to the same part.

(Embodiment 1)
FIG. 1 is a block diagram showing an image processing apparatus according to Embodiment 1 of the present invention. As an example, the image processing apparatus 1 according to the first embodiment performs an intraluminal image (hereinafter, also simply referred to as an image) acquired by imaging the inside of the lumen of a living body with a capsule endoscope. It is an apparatus that performs image processing to determine the presence or absence of a lesioned part (abnormal part) in which soft hair, which is a fine structure on the mucosal surface, has expanded. The intraluminal image is usually a predetermined pixel level (for example, 256 gradations) with respect to wavelength components (color components) of R (red), G (green), and B (blue) at each pixel position (pixels). Value).

  As shown in FIG. 1, the image processing apparatus 1 includes a control unit 10 that controls the operation of the entire image processing apparatus 1, an image acquisition unit 20 that acquires image data corresponding to an image captured by an endoscope, and the like. An input unit 30 that receives an input signal input from the outside, a display unit 40 that performs various displays, a recording unit 50 that stores image data acquired by the image acquisition unit 20 and various programs, and image data And an arithmetic unit 100 that executes predetermined image processing.

  The control unit 10 is realized by hardware such as a CPU, and reads various programs recorded in the recording unit 50, thereby according to image data input from the image acquisition unit 20, operation signals input from the input unit 30, and the like. Then, instructions to each unit constituting the image processing apparatus 1 and data transfer are performed, and the overall operation of the image processing apparatus 1 is controlled in an integrated manner.

  The image acquisition unit 20 is appropriately configured according to the mode of the system including the endoscope. For example, when a portable recording medium is used for transferring image data to and from a capsule endoscope, the image acquisition unit 20 detachably mounts the recording medium and records image data of the recorded image. It is comprised with the reader apparatus which reads. Further, when installing a server that stores image data of an image captured by an endoscope, the image acquisition unit 20 includes a communication device connected to the server, and performs image communication with the server to perform image communication. Get the data. Or you may comprise the image acquisition part 20 by the interface apparatus etc. which input an image signal from an endoscope via a cable.

The input unit 30 is realized by an input device such as a keyboard, a mouse, a touch panel, and various switches, and outputs the received input signal to the control unit 10.
The display unit 40 is realized by a display device such as an LCD or an EL display, and displays various screens including intraluminal images under the control of the control unit 10.

  The recording unit 50 is realized by various IC memories such as ROM and RAM such as flash memory that can be updated and recorded, a hard disk built in or connected by a data communication terminal, or an information recording medium such as a CD-ROM and a reading device thereof. The The recording unit 50 operates the image processing apparatus 1 in addition to the image data acquired by the image acquisition unit 20, and uses the program for causing the image processing apparatus 1 to execute various functions, and is used during the execution of this program. Data to be stored. Specifically, the recording unit 50 stores an image processing program 51 for determining the presence or absence of an abnormal portion of fur on the mucosal surface, various information used during execution of this program, and the like.

  The arithmetic unit 100 is realized by hardware such as a CPU, and performs image processing on the intraluminal image by reading the image processing program 51 to determine the presence or absence of abnormal portions of fur on the mucosal surface. Various arithmetic processes are executed.

Next, the configuration of the calculation unit 100 will be described.
As illustrated in FIG. 1, the calculation unit 100 includes a region-of-interest setting unit 110 that sets a region of interest in an intraluminal image to be processed, and a linear shape in which pixels having pixel values higher than the surroundings are equal to or greater than a predetermined number of pixels. And a feature amount based on the curvature of one or more arcs along the line shape region (hereinafter also referred to as a curvature feature amount). An in-region curvature feature amount calculation unit 130 that calculates the distribution and an abnormality determination unit 140 that determines whether or not an abnormal portion exists in the attention region based on the curvature feature amount.

  Among these, the attention area setting unit 110 includes a mucosal area extraction unit 111 that extracts a mucous membrane area by excluding a region other than the mucous membrane such as a residue and a dark part from the image, and sets the extracted mucosal area as the attention area.

The line-shaped convex region extracting unit 120 includes a convex high-frequency component calculating unit 121, an isolated point excluding unit 122, and a thinning unit 123.
The convex high frequency component calculation unit 121 calculates the intensity of a component having a spatial frequency equal to or higher than a predetermined value (hereinafter referred to as a high frequency component) for a pixel region having a pixel value higher than that of the surrounding area. In the following, a pixel region having a pixel value higher than that of the surrounding area is also referred to as a convex region or simply a convex shape, and is a pixel region having a convex shape and a high-frequency component intensity equal to or higher than a predetermined value. Also called a convex high-frequency region. The strength of the high frequency component in the convex high frequency region is also referred to as a convex high frequency component.

The isolated point excluding unit 122 excludes the isolated point as normal soft hair based on the convex high frequency component calculated by the convex high frequency component calculating unit 121. Here, the isolated point is a region in the convex high-frequency region where the number of continuous pixels is less than a predetermined threshold in all directions around the convex high-frequency region.
The thinning unit 123 thins the convex high-frequency region.

  In addition, the in-region curvature feature amount calculation unit 130 calculates, as the size feature amount of the arc, the distance information corresponding to the curvature of one or more arcs along the linear region and the imaging distance to the arc. A calculation unit 131 and a frequency distribution creation unit 132 that creates a frequency distribution of arc size feature values are provided. Among these, the size feature amount calculation unit 131 includes a curvature calculation unit 131a that calculates the curvature of one or more arcs from the linear region, and a curvature representative value that calculates a representative value from the calculated curvatures of the one or more arcs. A calculation unit 131b and a distance information calculation unit 131c that calculates distance information from the imaging position of the intraluminal image (that is, the position of the capsule endoscope) to the convex region having a linear shape.

Next, the operation of the image processing apparatus 1 will be described. FIG. 2 is a flowchart showing the operation of the image processing apparatus 1.
First, in step S <b> 10, the calculation unit 100 reads the image data recorded in the recording unit 50, thereby acquiring an intraluminal image to be processed.

  In subsequent step S20, the attention area setting section 110 sets the attention area for the intraluminal image. FIG. 3 is a flowchart showing in detail the processing executed by the attention area setting unit 110.

  In step S201, the mucosal region extraction unit 111 divides the intraluminal image into a plurality of small regions based on the edge strength of each pixel in the intraluminal image. The process of dividing the intraluminal image into a plurality of small regions will be described in detail with reference to FIG. First, in step S2011, the mucosa region extracting unit 111 creates a G component image using the G component of the pixel values of each pixel in the intraluminal image. The reason why the G component is used here is that the G component most closely represents the structure of the subject, such as the mucous membrane bulge and the boundary of the residue, because it is close to the blood absorption band. In the first embodiment, the G component is used, but instead, other color components, luminance values obtained by converting pixel values (R, G, and B components), and color differences (YCbCr conversion). , Hue, saturation, lightness (HSI conversion), color ratio, and the like may be used.

  In step S2012, the mucous membrane region extraction unit 111 performs filtering on the G component image to calculate edge strength, and creates an edge image having the edge strength as the pixel value of each pixel. As the filter processing for calculating the edge strength, a first-order differential filter such as a Brewit filter or a Sobel filter, a second-order differential filter such as a Laplacian filter, or a LOG (Laplacian of Gaussian) filter is used (reference: CG-). ARTS ASSOCIATION, “Digital Image Processing”, pp. 114-121).

  Furthermore, the mucous membrane region extraction unit 111 performs region division based on the edge strength in the edge image. As an example of the region division method, the method disclosed in International Publication No. 2006/080239 is applied in the first embodiment. That is, in step S2013, the edge image is subjected to a smoothing process to remove noise. In subsequent step S2014, the direction (maximum gradient direction) in which the gradient of the edge intensity is maximum is obtained at each pixel position in the edge image. In step S2015, an edge image is searched from each pixel (starting pixel) along the maximum gradient direction, and a pixel position where the pixel value (edge intensity) is an extreme value is acquired. Further, in step S2016, a label image is created in which starting pixels that are extreme values at the same pixel position are labeled as one region. Here, the label image is an image having a label number (1 to n, n is the number of labels) obtained by labeling as a pixel value of each pixel. A pixel region having the same pixel value in this label image corresponds to each small region obtained by dividing the intraluminal image. Thereafter, the process returns to the main routine.

  As a method for dividing the intraluminal image into regions, a known method other than the method described above may be applied. Specifically, the watershed algorithm (Reference: Luc Vincent and Pierre Soille, “Watersheds in digital spaces: An efficient algorithm based on immersion simulations”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 13, No. 6 , Pp. 583-598, June 1991) may be used. The watershed algorithm is a method of dividing an image so that a boundary is formed between water accumulated in different depressions when water is filled in the terrain where the pixel value information of the image is regarded as altitude.

  In the first embodiment, an area based on an edge is used for the purpose of reducing the influence of isolated pixel noise and facilitating the subsequent abnormality determination by dividing an area along a boundary such as a mucous membrane or a residue. Although the division is performed, the intraluminal image may be divided into rectangular regions of a predetermined size as another region dividing method. In this case, processing such as creation of an edge image and area division based on edge strength is not necessary, and the processing time can be shortened. Alternatively, each pixel may be a processing unit in each step described below without performing region division. In this case, the process of step S201 can be omitted.

  In step S202 following step S201, the mucous membrane region extraction unit 111 calculates an average value of feature amounts for each small region obtained by region division. More specifically, the mucous membrane region extraction unit 111 calculates an average value of each component of R, G, and B from the pixel value of each pixel in the small region. Then, by applying HSI conversion (reference: CG-ARTS Association, “Digital Image Processing”, page 64 (inverse conversion from HSI conversion)) to these average values, the hue average value of a small region is calculated. . Here, the hue average value is calculated because the subject such as the mucous membrane and the contents in the lumen is discriminated to some extent by the color feature amount such as hue from the difference in the light absorption characteristics of the components such as blood and bile. Because it can. Basically, the hues of the gastric mucosa, the intestinal mucosa and the contents change in this order from the red hue toward the yellow hue.

  Note that the hue average value of each small area is calculated by performing HSI conversion on the pixel values (R, G, and B components) of each pixel in the small area and then calculating the hue average value of each pixel. You may obtain | require by calculating. Further, instead of the hue, for example, an average value of feature quantities such as a color difference and a color ratio (G / R, B / G) may be calculated.

  In step S203, the mucous membrane area extracting unit 111 determines whether each small area is a dark area based on the average value of the feature values. More specifically, the mucous membrane region extraction unit 111 first creates a discrimination result list for each small region. Here, the discrimination result list is a list in which the label number in the label image when the intraluminal image is divided into a plurality of small regions and the flag information corresponding to each label number are associated with each other. The size of the discrimination result list corresponds to the number of labels n. Subsequently, the mucous membrane area extracting unit 111 initializes the discrimination result list and assigns a mucosal area flag (0: mucosal area) to all the small areas. Then, the G component that best represents the structure of the subject, such as the mucous membrane bulge and the boundary of the residue, is used as the feature amount, and the dark region flag (1: (Dark area). Note that the threshold value at this time is determined within a range in which the change in the hue average value of the small area or the like maintains linearity with respect to the change in brightness. This is because the linearity is lost in the dark area due to the influence of noise or the like.

  In step S204, the mucous membrane region extraction unit 111 clusters the distribution of the average value (hue average value) of the feature amounts calculated in step S202. Here, clustering is a method of dividing the data distribution in the feature space into chunks called clusters based on the similarity between the data, and is executed by various known methods such as a hierarchical method and a k-means method. (Reference: CG-ARTS Association: “Digital Image Processing”, pages 231 to 232 (clustering)). In the first embodiment, in order to perform processing using a data histogram in the hue space in the subsequent stage, a feature space is set as a one-dimensional space of hue, and a clustering procedure based on the data histogram is shown. Note that clustering is not limited to this, and other methods may be used.

  Hereinafter, processing for clustering the distribution of hue average values will be described in detail with reference to FIGS. First, a histogram is created in which the horizontal axis is the hue average value class interval and the vertical axis is the hue average value frequency corresponding to each class interval. At this time, the hue average value is acquired from the small area to which the mucosal area flag (0: mucosal area) is added in the discrimination result list. The class section is set by dividing the yellow to red hue range that sufficiently includes the hue distribution range in the intraluminal image at predetermined intervals. FIG. 5 is an example of creating a histogram. In FIG. 5, normalization is performed so that the sum of the frequencies becomes 1.

  Subsequently, the distribution of hue average values is divided into a plurality of clusters with the valleys of the histogram as boundaries. FIG. 6 is a schematic diagram illustrating the clustering method. The frequency data graph shown in FIG. 6 simply shows the frequency change of the histogram with a broken line. Here, the class section shown on the horizontal axis of the histogram is indicated by coordinates for convenience. In order to limit the number of clusters clustered, the width of the histogram class section may be changed, or the histogram may be smoothed before clustering.

  For such frequency data, the gradient direction at each coordinate is obtained. Here, the gradient direction is a direction determined based on the difference between the frequency data at the coordinate of interest and the frequency data at the coordinate adjacent to the coordinate of interest, and the direction in which the value of the frequency increases is indicated by an arrow. The coordinates displayed as “pole” are extreme (maximum) coordinates having a frequency higher than any of the adjacent coordinates.

  Further, extreme value coordinates are searched from each coordinate along the gradient direction. In FIG. 6, the change in coordinates when searching for extreme coordinates is indicated by the numerical values of the coordinates from the position of the starting coordinates (each coordinate at n = 0) downward. For example, when the search is started from the coordinate 1, the gradient direction is the right direction at the coordinate 1, and therefore the coordinate 2 adjacent to the right is obtained in the first search step (n = 1). Subsequently, since the gradient direction is also the right direction in the coordinate 2, the coordinate 3 adjacent to the right of the coordinate 2 is obtained in the second search step (n = 2). Thereafter, if the search is continued along the gradient direction, the coordinate 5 which is an extreme value coordinate is finally reached (n = 4). Next, when the search is started from the coordinate 2, the extreme value coordinate of the coordinate 5 which is the extreme value coordinate is finally reached (n = 3). Similarly, when a similar search is performed from all coordinates, when coordinates 1 to 9 are set as the starting coordinates, the coordinate 5 is reached, and when coordinates 10 to 15 are set as the starting coordinates, the coordinates 12 are set. to arrive.

  As a result of such a search, coordinates 1 to 9 where coordinates 5 are obtained as extreme coordinates are set as the first cluster, and coordinates 10 to 15 where coordinates 12 are obtained as extreme coordinates are set as the second cluster. Thus, the data distribution can be divided into clusters with the valleys of the histogram as boundaries.

  FIG. 7 is a graph showing a result of applying the above-described clustering method to the histogram shown in FIG. In the case of FIG. 5, the hue average value can be classified into three of clusters 1 to 3 with the valley of the histogram as a boundary.

  In step S205 subsequent to step S204, the mucous membrane region extraction unit 111 determines an unnecessary region such as a residue based on the distribution of the feature amount average value. Here, in the lumen, generally, an unnecessary area such as a residue has a yellowish color due to a difference in light absorption characteristics between bile and blood. Therefore, among the clusters obtained in step S204, a cluster whose hue is close to yellow is estimated as a cluster composed of hue average value data of small areas that are unnecessary areas (hereinafter referred to as unnecessary area clusters), and unnecessary area clusters are determined. Is identified.

More specifically, first, the center of gravity ω _a bar of each cluster a (a is a cluster number and in the case of FIG. 7, a = 1, 2, 3) (the bar is a symbol “−” indicating an average on ω). ) Using the sum of the hue average values ω _{a in} the cluster a and the number of data belonging to the cluster a (that is, the number of small areas) m _a , Calculated by

また、記号ｉは小領域のラベル番号を示す。記号ｎはラベル数、即ち、画像内の小領域の数を示す。記号ｈ_iは、小領域ｉの色相平均値を示す。記号ａ＿ｍｉｎは、クラスタａにおける色相平均値の最小値を示し、記号ａ＿ｍａｘは、クラスタａにおける色相平均値の最大値を示す。さらに、

In equation (1),

The symbol i indicates the label number of the small area. The symbol n indicates the number of labels, that is, the number of small areas in the image. The symbol h _i indicates the hue average value of the small region i. The symbol a_min indicates the minimum value of the average hue value in the cluster a, and the symbol a_max indicates the maximum value of the average hue value in the cluster a. further,

  In the histogram of hue average values shown in FIG. 7, the yellowish color becomes stronger toward the left of the hue axis (horizontal axis). Therefore, a residue determination threshold value indicating the range of the hue of the residue is calculated from the teacher data acquired in advance, and a cluster whose centroid is located on the left side of the graph with respect to the residue determination threshold value is determined as an unnecessary region cluster. For example, in FIG. 7, cluster 1 is determined as an unnecessary area cluster.

Further, the mucous membrane area extracting unit 111 sets the unnecessary area flag (2: unnecessary) for the small areas belonging to the unnecessary area cluster among the small areas to which the mucosal area flag (0: mucosal area) is assigned in the discrimination result list. Area).
It should be noted that in determining the unnecessary area, the same processing may be performed using the average value of the feature values other than the hue.

  In subsequent step S206, the attention area setting unit 110 acquires coordinate information of the mucosal area (area other than the unnecessary area) in the image based on the flag information of the label image and the determination result list, and coordinates of the mucosal area. Output information as a region of interest. Thereafter, the process returns to the main routine.

  In step S30 following step S20, the line-shaped convex region extraction unit 120 extracts a line-shaped region in which pixels having pixel values higher than the surroundings are continuous for a predetermined number of pixels from the region of interest in the image. FIG. 8 is a flowchart showing in detail the processing executed by the linear convex region extraction unit 120.

  In step S301, the convex high-frequency component calculation unit 121 calculates the intensity of the convex high-frequency component. More specifically, first, the convex high-frequency component calculating unit 121 creates a G component image using the G component of the pixel values of each pixel in the intraluminal image. The reason for using the G component is that, as described above, since the G component is close to the light absorption band of blood, it best represents the structure of the subject. In the first embodiment, the G component is used, but instead, other color components, luminance values obtained by converting pixel values (R, G, and B components), and color differences (YCbCr conversion). , Hue, saturation, lightness (HSI conversion), color ratio, and the like may be used.

Subsequently, the convex high-frequency component calculation unit 121 performs top-hat conversion (reference: Corona, “Morphology”, pages 103 to 104) based on light and shade morphology (morphology) on the G component image. Here, as shown in FIG. 9, the top-hat transform is an output y = f _g (x) obtained by performing low-pass filter processing on the original signal output y = f (x), and the original signal output f This is a process subtracted from (x), and gives an output y = f (x) −f _g (x) of the high-pass filter process. Further, as the low-pass filter process at this time, a morphology opening process is executed. The opening process corresponds to a process of tracing a given signal f (x) from below using a spherical structural element as shown in FIG. At this time, the signal output y = f (x) is accurately traced when the signal is slowly changing, but the sphere (structural element) cannot be entered where there is a positive pulse. Deviate from. Therefore, the output y = f _g (x) from which the positive pulse is removed is obtained by the opening process. By subtracting this output y = f _g (x) from the original signal output y = f (x), as a result, a high-frequency component in the convex region that is the output of the positive pulse can be acquired.
In addition, the intensity of the convex high-frequency component may be calculated using high-pass filter processing using Fourier space, DoG (Difference of Gaussian), or the like.

  In subsequent step S302, the isolated point excluding unit 122 extracts an isolated point based on the intensity of the convex high-frequency component calculated in step S301, and excludes this isolated point as normal soft hair.

  Specifically, the isolated point exclusion unit 122 first creates a binarized image by performing threshold processing on the intensity of the convex high-frequency component. At this time, the isolated point excluding unit 122 gives 0 to a pixel whose intensity of the convex high-frequency component is equal to or lower than a predetermined threshold value, and gives 1 to a pixel higher than the threshold value. As the threshold value, for example, a value of a predetermined ratio with respect to one representative value (maximum value or average value) calculated from the intensity of the convex high-frequency component in the region of interest is set. Alternatively, a fixed value greater than 0 may be set as the threshold value. In the latter case, the calculation of the representative value for the convex high-frequency component and the process of calculating the threshold value from the representative value can be omitted. Note that the binarized image created by this processing is an image from which the convex high-frequency region is extracted.

  Subsequently, the isolated point exclusion unit 122 performs particle analysis and excludes isolated points from the binarized image. In the first embodiment, the particle analysis is performed based on the area of the target region (particle) in the binarized image. Specifically, the isolated point excluding unit 122 creates a label image assigned with label numbers 1 to s (where s is the number of labels) by labeling a region having a pixel value of 1 with respect to the binarized image. To do. Then, the label image is scanned, the number of pixels is counted for each label number, and an area list is created. Here, the area list includes a label number and an area corresponding to each label number, and the size of the area list corresponds to the number of labels s. Further, the isolated point excluding unit 122 performs threshold processing on the area of the label area corresponding to each label number with a predetermined threshold, and determines a label area having a label number whose area is equal to or smaller than the threshold as an isolated point. The label number of the label area determined as an isolated point is set to 0.

  When performing isolated point exclusion processing, in addition to the area of the target area, the perimeter of the target area, the ferret diameter (maximum ferret diameter), and the maximum distance between any two points on the outline of the target area Particle analysis using an absolute maximum length indicating a value, a circle equivalent diameter indicating a diameter of a circle having the same area as the target region, or the like may be performed.

  In step S303, the thinning unit 123 thins the convex high-frequency region. More specifically, the thinning unit 123 performs a thinning process on a region having one or more label numbers in the label image from which the isolated points are excluded by the isolated point excluding unit 122. Therefore, the thinning unit 123 first gives a binary image to a label image by giving a pixel value 0 to an area with a label number 0 and giving a pixel value 1 to an area with a label number 1 or more. create. Subsequently, the thinning unit 123 performs a thinning filter process on the binarized image. The thinning filter process is a process of sequentially searching patterns M1 to M8 of 3 × 3 pixels as shown in FIG. 10 in the binarized image and deleting the center pixel (reference). : University of Tokyo Press, “Image Analysis Handbook”, pages 577 to 578 (thinning and degeneration). Specifically, first, the pattern M1 is searched from the binarized image, and the pixel value of the central pixel is changed to zero. Note that * shown in FIG. 10 indicates a pixel that need not be considered. Next, the pattern M2 is searched from the binarized image, and the pixel value of the center pixel is changed to zero. Such a process is repeated up to the pattern M8 to form one cycle. This cycle is repeated until there are no more points to delete.

  Further, in step S304, the linear shape convex region extraction unit 120 outputs a binarized image (hereinafter also referred to as a linear shape image) that has been subjected to the thinning process. Thereafter, the process returns to the main routine.

  In step S40 subsequent to step S30, the in-region curvature feature amount calculation unit 130 calculates a curvature feature amount based on the curvature of the arc along the linear region. FIG. 11 is a flowchart showing in detail the processing executed by the in-region curvature feature quantity calculation unit 130.

  In step S <b> 401, the size feature amount calculation unit 131 uses the curvature of the arc along the line shape region observed in the line shape image and the imaging position (capsule endoscope) to the arc as the feature size of the arc size. The distance information corresponding to the imaging distance is calculated. The processing executed by the size feature amount calculation unit 131 will be described in detail with reference to FIG.

  In step S4011, the size feature amount calculation unit 131 labels continuous line shape regions in the line shape image. In the first embodiment, labeling is performed for eight connected components. Specifically, the size feature amount calculation unit 131 first performs a raster scan on the line-shaped image, and searches for pixels that have no pixel number among pixels having a pixel value of 1. Then, when a pixel to which no label number is assigned is found, that pixel is set as a target pixel.

  When the pixel above or at the upper left of the target pixel has a label number, the size feature amount calculation unit 131 gives the target pixel the label number of the upper adjacent or upper left pixel. After that, when the label number of the pixel adjacent to the left of the target pixel is different from the label number of the target pixel, the target pixel and the label number of the pixel adjacent to the left belong to the same connected component in the lookup table prepared in advance. Record that.

  Further, when the label number of the upper adjacent pixel and the upper left pixel is 0 (no label number) and the left adjacent pixel has a label number, the label number of the left adjacent pixel is given to the target pixel.

  Further, if no label number is assigned to any of the upper, left, and upper left pixels of the target pixel, the size feature amount calculation unit 131 assigns a new label number to the target pixel.

  The size feature amount calculation unit 131 performs these processes on all the pixels in the line shape image. Finally, the raster scan is performed again, and the smallest label number among the label numbers assigned to the pixel groups is selected and attached to the pixel groups belonging to the same connected component while referring to the lookup table. Correct (Reference: CG-ARTS Association, “Digital Image Processing”, pages 181 to 182).

In step S4012, the curvature calculation unit 131a, as shown in FIG. 13, is a linear region that is delimited by the end point P _t of the linear region L in the linear image G1 and / or the intersection point P _c between the linear regions L. The curvature of one or more arcs is calculated from each section of L (that is, between end points, between cross points, or between end points and cross points).

More specifically, the curvature calculating unit 131a scans the line object image, detects the pixel of the end point P _t or line-shaped region intersection P _c between the line-shaped region. Then, as shown in FIG. 14, to set the pixels of the detected edge point P _t or intersection P _c starting point pixel M (x _s, y _s) to. Further, a pixel arrived by tracing a predetermined number of pixels Δs from the start point pixel M (x _s , y _s ) on the linear region L (that is, on the pixel assigned the same label number) is designated as an end point pixel N ( x _e , y _e ).

Subsequently, the curvature calculation unit 131a calculates the inclination of the tangent line of the start point pixel M and the inclination of the tangent line of the end point pixel N. In the present embodiment, the inclination of each tangent is represented by angles DEG _s and DEG _e formed with the x axis. The angle DEG _i (i = s, e) formed between each tangent and the x axis is two pixels (a predetermined number of pixels away from the start point pixel M (or the end point pixel N) before and after the line shape region). Coordinates (x _a , y _a ), (x _b , y _b )) are set and calculated using the following equation (2).

When the upper left corner of the line shape image is set to coordinates (0, 0) and each row is scanned in the right direction from the upper left corner, the coordinates (x _a , y _a ) are in front of the start point pixel M (or end point pixel N). The coordinates (x _b , y _b ) are the rear coordinates of the start point pixel M (or the end point pixel N). However, when the predetermined number of pixels away from the start point pixel M (or the end point pixel N) deviates from the linear shape area (when it exceeds the end point or the intersection point), the coordinates of the end point or the intersection point of the linear shape area Is set to one of the coordinates (x _a , y _a ) and (x _b , y _b ).

Subsequently, the curvature calculation unit 131a calculates the center angle α of the arc between the start point pixel M and the end point pixel N. This central angle α corresponds to the difference DEG _e −DEG _s between the angle DEG _s and the angle DEG _e .

Further, the curvature calculation unit 131a calculates the curvature κ of the arc between the start point pixel M and the end point pixel N. Here, the curvature κ is the reciprocal of the radius of curvature R of the arc, and the number of pixels Δs corresponding to the length of the arc can be approximated as R × α × (π / 180). Given by (3).

As shown in FIG. 15, the curvature calculation unit 131a shifts the start point pixel M (M ₁ , M ₂ ,...) And the end point pixel N (N ₁ , N ₂ ,. The calculation process of (κ ₁ , κ ₂ ,...) Is repeated. When the end point pixel N exceeds the end point P _{t of} the line-shaped region L or the intersection point P _c between the line-shaped regions L, the curvature calculating unit 131a ends the calculation process of the curvature α, and the linear shape obtained by calculating the curvature α. Label numbers incremented in order from 1 are assigned to the area.

  The curvature calculation unit 131a performs the calculation of the curvature κ and the assignment of the label number to all the line shape regions L detected from the line shape image G1 shown in FIG. The numbers in parentheses shown in FIG. 13 indicate the label numbers given to the respective linear regions L.

  In subsequent step S4013, the curvature representative value calculation unit 131b calculates the representative value of the curvature of the arc for each linear region to which the same label number is assigned by the curvature calculation unit 131a. In the first embodiment, the median value is calculated as the representative value, but the average value, maximum value, minimum value, etc. of the curvature may be calculated as the representative value.

  In subsequent step S4014, the distance information calculation unit 131c calculates distance information in the depth direction from the imaging position to the linear region. Specifically, the distance information calculation unit 131c reduces the absorption of hemoglobin among the pixel values of each pixel from the portion corresponding to the region of interest in the original intraluminal image, and best optimizes the shape of the surface layer of the mucous membrane. An R component value to be expressed is acquired to create an R component image.

  In step S4014, distance information may be acquired from an image created by another method as long as the image has little influence on the fine structure of the fur on the mucous membrane. For example, an image that has been subjected to the opening process using the light and shade morphology described in the process of the convex high-frequency component calculation unit 121 may be used.

  Subsequently, the distance information calculation unit 131c calculates distance information at the position of each arc whose curvature is calculated by the curvature representative value calculation unit 131b. Specifically, the R component value at the pixel position on the arc is acquired, and representative values such as the average value, median value, minimum value, and maximum value of the R component value are acquired for each arc. In the first embodiment, the representative value of the R component value is used as distance information corresponding to the imaging distance. In this case, the larger the distance information value, the shorter the imaging distance, and the smaller the distance information value, the longer the imaging distance. Further, an R component value in a pixel area at the center of curvature of the arc, a circular area composed of the center of curvature of the arc and the radius of curvature, or an inner area of the fan-shaped area may be used as distance information of the arc.

  In step S4015, the size feature amount calculation unit 131 outputs the arc curvature and distance information as the feature size of the arc size. Thereafter, the process returns to the main routine.

  In step S <b> 402 following step S <b> 401, the frequency distribution creating unit 132 creates a frequency distribution of the feature amount output from the size feature amount calculating unit 131. More specifically, a frequency distribution including two axes of the curvature of the arc and the distance information is created. At this time, the frequency distribution creating unit 132 also calculates a statistic such as variance from the created frequency distribution.

  Further, in step S403, the in-region curvature feature amount calculation unit 130 outputs a frequency distribution composed of two axes of the curvature of the arc and the distance information as the curvature feature amount. Thereafter, the process returns to the main routine.

In S50 following Step S40, the abnormality determination unit 140 determines whether or not there is an abnormal part in the attention area based on the curvature feature amount.
Here, the arc-shaped region seen in the intraluminal image includes an abnormal part where the soft hair on the mucosal surface is expanded, a bubble region where the liquid in the lumen is foamed, and a contour of a structure such as a groove in the mucous membrane. (Hereinafter referred to as mucosal contour). Hereinafter, the expanded fur is referred to as abnormal fur.

  FIG. 16A is a schematic diagram showing a normal fur model. Normal fur has a circular shape when viewed from the normal direction of the mucosal surface and has a shape in which the central part protrudes as a whole. The brightness at is the highest. For this reason, when a convex region having a luminance value higher than that of surrounding pixels is extracted from an intraluminal image in which normal fur is reflected, as shown in FIG. 16B, a dot-like convex corresponding to the center of the fur is obtained. Region PR1 is obtained. Such a dot-like convex region PR1 is excluded from the line-shaped image in advance as an isolated point.

  FIG. 17A is a schematic diagram showing an abnormal fur model. The planar shape of the abnormal fur viewed from the normal direction of the mucosal surface is a flat shape such as an ellipse. Therefore, in the intraluminal image, the contour portion (the curvature is high) according to the direction of the fur against the imaging direction. The brightness of the contour portion or the contour portion having a low curvature increases. Therefore, when a convex region is extracted from an intraluminal image in which abnormal fur is reflected, a linear region PR2 having an arc shape with a curvature corresponding to the imaging direction is obtained as shown in FIG. 17B. For this reason, the distribution of the curvature of the arc along the line-shaped region PR2 falls within a certain narrow range as shown in FIG. 17C. Further, when the minimum value of curvature is adopted as the curvature representative value of the arc along each line-shaped region PR2, the curvature is biased to a smaller value (see the solid line). On the other hand, when the maximum value of curvature is adopted as the curvature representative value, the curvature is distributed in a biased manner toward the larger value (see broken line).

FIG. 18A is a schematic diagram showing a bubble model. In the bubble region, each bubble has a substantially spherical shape. In addition, since the entire surface of the bubble is a highly reflective region, the brightness of the outline portion of the bubble is increased in the intraluminal image. Accordingly, when the convex region is extracted from the intraluminal image in which the bubble region is reflected, as shown in FIG. 18B, linear regions PR3 of various sizes having a substantially circular shape are obtained. For this reason, the distribution of the curvature of the arc along the linear region PR3 varies greatly as shown in FIG. 18C.
In addition, the curvature of the arc along the mucous membrane outline generally takes a smaller value than that of abnormal fur or bubble area.

  FIG. 19 is a schematic diagram illustrating a frequency distribution including two axes of the curvature of the arc and the distance information. The curvature of the arc along the line-shaped region indicating the mucous membrane contour is generally small and distributed in a narrow range regardless of the imaging distance. Further, the curvature of the arc along the line-shaped region indicating the bubble region is distributed over a wide range regardless of the imaging distance. On the other hand, the distribution range of the curvature of the arc along the linear region indicating abnormal fur changes depending on the imaging distance. Specifically, when the imaging distance becomes longer (when the value of imaging information becomes smaller), the curvature is distributed in a larger value (where the arc curve is stronger), and when the imaging distance becomes shorter (the value of imaging information becomes larger). ), The curvature is distributed in the smaller value (the one with a gentle arc curve).

Therefore, the abnormality determination unit 140 performs determination as follows.
First, the abnormality judgment unit 140, the variance of curvature at a certain distance information L _arb is the distance when information L greater than a predetermined given threshold value (the dispersion threshold) TH ₁ in accordance with the _arb, foam in the region of interest It is determined that the area is reflected and no abnormal part exists.

Moreover, the abnormality judgment unit 140, when the dispersion of the curvature at the distance information L _arb is below the dispersion threshold value TH _1, when the curvature is distributed in a range smaller than a predetermined threshold value (the curvature threshold) TH _2, the attention It is determined that the region has a mucous membrane outline and no abnormal portion exists.

On the other hand, the abnormality judgment unit 140, when the dispersion of the curvature at the distance information L _arb is below the dispersion threshold value TH _1, when the curvature is distributed to the curvature threshold TH ₂ or more ranges, the abnormality in the region of interest soft Hair is reflected and it is determined that an abnormal part exists.

  In subsequent step S60, operation unit 100 outputs the determination result of the presence or absence of an abnormal part made in step S50. In response to this, the control unit 10 displays the determination result on the display unit 40 and records it in the recording unit 50 in association with the image data of the intraluminal image to be processed.

  As described above, according to the first embodiment, the mucous membrane region extracted from the intraluminal image is set as a region of interest, and the convex region including a pixel group having a pixel value higher than that of surrounding pixels is set from this region of interest. Extract high frequency components from the region. Then, curvature and distance information are calculated as curvature feature amounts for the line-shaped region in which the high-frequency component is thinned, and a subject reflected in the attention region is identified based on the frequency distribution of these feature amounts. Therefore, it suppresses false detection (overdetection) of mucous membrane outlines and bubbles, etc., and supports changes in size according to the imaging distance of the fine structure of fur and changes in shape according to the direction of fur against the imaging direction. Thus, it is possible to accurately identify abnormal fur and determine the presence or absence of an abnormal portion.

(Modification 1-1)
In the first embodiment, the entire remaining area from which the unnecessary area is removed from the intraluminal image is set as one attention area. However, a plurality of areas obtained by dividing the remaining area may be set as the attention areas. good. In this case, it is possible to specify at which position in the intraluminal image the abnormal portion exists. The division method is not particularly limited. For example, the remaining area from which unnecessary areas are removed may be simply divided into a matrix. Alternatively, the distance information corresponding to the imaging distance may be acquired from the pixel values of each pixel in the remaining area from which the unnecessary area is removed, and the area may be divided into hierarchies obtained by dividing the distance information into hierarchies. In this case, the process of step S4014 shown in FIG. 12 can be omitted.

(Modification 1-2)
Next, Modification 1-2 of Embodiment 1 will be described.
FIG. 20 is a block diagram illustrating an image processing apparatus according to Modification 1-1. As illustrated in FIG. 20, the image processing device 1-2 according to the modified example 1-1 includes a calculation unit 100-2 instead of the calculation unit 100 illustrated in FIG.

  The computing unit 100-2 includes a linear convex region extracting unit 150 instead of the linear convex region extracting unit 120 shown in FIG. The line shape convex region extraction unit 150 is isolated from the ridge shape extraction unit 151 that extracts a pixel group whose pixel value changes to a ridge shape as a ridge shape region, and is isolated from the extracted ridge shape region. An isolated point excluding unit 152 that excludes a pixel group as an isolated point, and a thinning unit 153 that thins a ridge-shaped region after excluding the isolated point and extracts a linear convex region.

  The operation of the calculation unit 100-2 is the same as that shown in FIG. 2 as a whole, and only the content of the process executed by the linear convex region extraction unit 150 in step S30 is different. FIG. 21 is a flowchart showing details of processing executed by the linear convex region extraction unit 150.

  First, in step S311, the ridge shape extraction unit 151 extracts a ridge shape region from the region of interest. More specifically, the ridge shape extraction unit 151 sequentially sets the target pixel in the target region, and the maximum direction (hereinafter referred to as the maximum gradient direction) and the minimum direction (hereinafter, the minimum gradient) of the pixel value gradient in each target pixel. Find the direction). Then, the curvature of the shape in which the pixel value changes is calculated for each of the maximum gradient direction and the minimum gradient direction, and the ridge region is detected based on the ratio of these curvatures.

For this purpose, the ridge shape extraction unit 151 first solves the eigen equation by the Hessian matrix H shown in the following equation (4) for the pixel of interest, thereby obtaining the curvature k ₁ in the maximum gradient direction as the eigenvalue and the minimum gradient. The curvature k ₂ in the direction is calculated.
det (H−λE) = 0 (4)

In equation (4), the Hessian matrix H is given by the following equation (5).

Symbol E is a unit matrix, and λE is as in the following equation (6).

Subsequently, the curvature k _1, k _2, calculates the Gaussian curvature K _G and mean curvature A _k is given by the following equation (7-1) and (7-2).
K _G = k ₁ × k ₂ (7-1)
A _k = (k ₁ + k ₂ ) / 2 (7-2)

At this time, a pixel group in which A _k <0 and K _G ≈0, that is, a pixel group in which the absolute value | K _G | is equal to or less than a predetermined threshold is determined to be a ridge-shaped region. The ridge shape extraction unit 151 creates a binarized image by assigning a pixel value of 1 to a pixel group determined to be a ridge shape region and assigning a pixel value of 0 to other pixels.

After calculating the curvatures k ₁ and k ₂ , the ridge-shaped region may be discriminated using a known technique for recognizing an arbitrary three-dimensional surface shape, for example, Shape index and Curvedness (reference: Chitra Dorai, Anil K Jain, “COSMOS-A Representation Scheme for Free-Form Surfaces”).

  In subsequent step S312, the isolated point removal unit 152 extracts isolated points from the ridge-shaped region extracted in step S311 and excludes the isolated points as normal fur. More specifically, a pixel group in which the number of continuous pixels is equal to or smaller than a predetermined value (that is, an area having a size equal to or smaller than the predetermined value) is extracted as an isolated point from the binarized image created in step S311. A pixel value of 0 is assigned to the pixel group extracted as an isolated point.

  In subsequent step S313, the thinning unit 153 thins the ridge-shaped region after the isolated points are excluded. More specifically, the thinning unit 153 performs thinning filter processing using the patterns M1 to M8 illustrated in FIG. 10 on the binarized image created in step S312. The detailed contents of the thinning filter processing are the same as those in the first embodiment.

  Further, in step S314, the linear convex region extraction unit 150 outputs a binarized image that has been subjected to the thinning process. Thereafter, the process returns to the main routine.

(Modification 1-3)
In the first embodiment, various known techniques can be applied in addition to the method described above as a process for thinning the convex high-frequency region extracted from the region of interest (see step S303 in FIG. 8). Hereinafter, the Hilditch thinning algorithm, which is one of known techniques, will be described.

First, as a first step, pixels that satisfy the following six conditions among the pixels _{Pk to be} thinned are sequentially deleted as boundary pixels from the binarized image from which the convex high-frequency region is extracted. Here, k is the pixel number of the pixel in the binarized image (k is a natural number). In addition, the pixel value of the pixel P _k is indicated as B (P _k ) (B (P _k ) = 1). In this case, the first step corresponds to a process of replacing the pixel value B (P _k ) from 1 to −1. Note that the pixel value of the non-target pixel to be thinned is B (P _k ) = 0.

Condition 1: The target pixel is a pixel to be thinned. That is, the following expression (a1) is satisfied.
B (P _k ) = 1 (a1)

Condition 2: One of the pixels adjacent vertically and horizontally with respect to the target pixel is 0. That is, the following expression (a2) is satisfied.

Condition 3: It is not an end point. That is, the following expression (a3) is satisfied.

Condition 4: It is not an isolated point. That is, the following expression (a4) is satisfied. Here, the result obtained by the sequential processing is shown as B (P _k ), and the result at the completion of the previous raster operation is shown as B ^* (P _k ). Note that the pixel value B ^* (P _k ) does not take −1.

Condition 5: Retain connectivity. That is, it applies to any of the patterns M11 to M14 shown in FIG. In FIG. 22, the pixel indicated by * takes a pixel value of 1 or -1. When this condition is expressed by a conditional expression, the following expression (a5) is obtained.

Condition 6: For a line segment with a line width of 2, only one of them is deleted. That is, the result B ^* (P _k ) upon completion of the previous raster operation is applied to any of the patterns M21 to M24 shown in FIG. When this condition is expressed by a conditional expression, the following expression (a6) is obtained.

Next, as a second step, the pixel value of a pixel that is sequentially deleted as a boundary pixel (that is, a pixel in which the pixel value B (P _k ) is replaced from 1 to −1) is set to the pixel value B (P _k ) Replace with 0.
These first step and second step are repeated until there is no replacement with the pixel value in the non-target region. Thereby, the target area is thinned.

(Embodiment 2)
Next, a second embodiment of the present invention will be described.
FIG. 24 is a block diagram illustrating a configuration of the image processing apparatus according to the second embodiment. As illustrated in FIG. 24, the image processing apparatus 2 according to the second embodiment includes a calculation unit 200 instead of the calculation unit 100 illustrated in FIG. The calculation unit 200 includes an attention region setting unit 110, a linear convex region extraction unit 120, an in-region curvature feature amount calculation unit 210, and an abnormality determination unit 220. The configurations and operations of the attention area setting unit 110 and the linear convex area extraction unit 120 are the same as those in the first embodiment. The configuration and operation of each part of the image processing apparatus other than the arithmetic unit 200 are also the same as in the first embodiment.

The in-region curvature feature amount calculation unit 210 calculates the curvature of one or more arcs along the line shape region extracted by the line shape convex region extraction unit 120, and calculates the variation in the curvature as a feature amount . A shape feature amount calculation unit 211 and a frequency distribution creation unit 212 that creates a frequency distribution of feature amounts are provided. More specifically, the shape feature amount calculation unit 211 calculates a curvature of one or more arcs from each section of the linear region that is delimited by the end points of the linear region and / or the intersection of the linear regions. 211a and a curvature standard deviation calculation unit 211b that calculates a standard deviation of the curvature of one or more arcs calculated from each section.
The abnormality determination unit 220 determines the presence or absence of an abnormal part based on the variation in curvature.

Next, the operation of the image processing apparatus 2 will be described.
The operation of the image processing apparatus 2 is generally the same as that shown in FIG. 2. The contents of the processing executed by the in-region curvature feature quantity calculation unit 210 in step S40 and the processing executed by the abnormality determination unit 220 in step S50 are as follows. This is different from the first embodiment.

  FIG. 25 is a flowchart showing processing executed by the in-region curvature feature quantity calculation unit 210. First, in step S421, the shape feature quantity calculation unit 211 calculates the variation in the curvature of the arc along the line shape area as the feature quantity indicating the shape of the line shape area extracted in step S30. The processing executed by the shape feature quantity calculation unit 211 will be described in detail with reference to FIG. Note that steps S4211 and S4212 shown in FIG. 26 correspond to steps S4011 and S4012 shown in FIG. Among these, the process of step S4012 is performed by the curvature calculation unit 211a instead of the curvature calculation unit 131a (see FIG. 1).

In step S4213 following step S4212, the curvature standard deviation calculation unit 211b calculates the standard deviation of the curvature of one or more arcs for each section of the linear region. Specifically, the curvature standard deviation calculation unit 211b calculates the following from m curvatures κ _i (i = 1 to m) calculated for the arcs having the same label number given by the curvature calculation unit 211a. The standard deviation σ given by equation (8) is calculated.

In equation (8), κ bar with a bar on κ is an average value of curvature κ _i and is given by the following equation (9).

  In subsequent step S4214, the shape feature amount calculation unit 211 outputs the standard deviation calculated for each section of the line shape region as a feature amount representing the shape of the line shape region. Thereafter, the process returns to the main routine.

  In step S422 following step S421, the frequency distribution creation unit 212 creates the frequency distribution of the standard deviation output from the shape feature quantity calculation unit 211 as the feature quantity frequency distribution.

  Further, in step S423, the in-region curvature feature quantity calculation unit 210 outputs the frequency distribution of the standard deviation as the curvature feature quantity. Thereafter, the process returns to the main routine.

Next, processing executed by the abnormality determination unit 220 will be described.
The abnormality determination unit 220 determines whether or not there is an abnormal part in the attention area based on the frequency distribution of the standard deviation that is the curvature feature amount.

Here, when abnormal fur exists in the region of interest, as shown in FIG. 27A, a flat line-shaped region PR4 appears in the line shape image created from the region of interest. As shown in FIG. 27B, the curvature of the arc along the linear region PR4 takes various values depending on the position in the linear region PR4. For this reason, the variation in curvature becomes relatively large. Note that the vector v _i (i = 1, 2,...) Shown in FIG. 27B is a vector that represents the magnitude of the curvature of the arc. Therefore, as shown in FIG. 27C, the standard deviation of curvature calculated for the linear region PR4 is distributed in a relatively large range.

  On the other hand, when a bubble area exists in the attention area, as shown in FIG. 28A, a circular line-shaped area PR5 appears in the line shape image created from the attention area. As shown in FIG. 28B, in such a linear region PR5, the curvature of the arc becomes a substantially constant value, and the variation in curvature becomes small. Therefore, as shown in FIG. 28C, the standard deviation of the curvature calculated for the linear region PR5 is distributed in a small range (0 or its vicinity).

  Therefore, the abnormality determination unit 220 determines the presence or absence of an abnormal part based on the frequency distribution of the standard deviation. Specifically, when the frequency distribution of the standard deviation is biased to a range larger than a predetermined threshold, it is determined that an abnormal part exists in the attention area.

  As described above, according to the second embodiment, the mucous membrane region extracted from the intraluminal image is set as the attention region, and the convex region including the pixel group having a pixel value higher than that of surrounding pixels is set from the attention region. Extract high frequency components from the region. Then, a frequency distribution of the standard deviation of the curvature of the arc along the line shape region is calculated for the line shape region in which the high frequency component is thinned, and based on the frequency distribution of the standard deviation, the abnormal portion in the attention region is calculated. Determine presence or absence. Therefore, it is possible to accurately identify the abnormal fur and the bubble region and improve the determination accuracy of the abnormal part.

  Further, according to the second embodiment, since the determination is performed without using the imaging distance (distance information) to the subject in the attention area, the arithmetic processing can be simplified.

(Embodiment 3)
Next, a third embodiment of the present invention will be described.
FIG. 29 is a block diagram illustrating a configuration of an image processing apparatus according to the third embodiment. As illustrated in FIG. 29, the image processing apparatus 3 according to the third embodiment includes a calculation unit 300 instead of the calculation unit 100 illustrated in FIG. The calculation unit 300 includes a region of interest setting unit 110, a linear convex region extraction unit 120, an in-region curvature feature amount calculation unit 310, and an abnormality determination unit 320. The configurations and operations of the attention area setting unit 110 and the linear convex area extraction unit 120 are the same as those in the first embodiment. As for the various parts of the structure and operation of the image processing apparatus 3 other than the arithmetic unit 3 00, it is the same as in the first embodiment.

  The in-region curvature feature amount calculation unit 310 includes a direction feature amount calculation unit 311 that calculates a curvature feature amount indicating the center direction of one or more arcs along the line shape region extracted by the line shape convex region extraction unit 120. A frequency distribution creating unit 312 for creating a frequency distribution of curvature feature quantity. Among these, the direction feature amount calculation unit 311 calculates a direction of curvature center direction calculation unit 311a for calculating a direction from each arc toward the center of curvature (hereinafter referred to as a curvature center direction), and a gradient direction of the subject at the position of each arc. A gradient direction calculation unit 311b. Here, the gradient direction of the subject is a direction in which the subject (specifically, the mucous membrane structure) slopes in the depth direction of the image.

  The abnormality determination unit 320 determines the presence / absence of an abnormal part based on the gradient direction of the mucous membrane structure and the distribution of the curvature center direction.

Next, the operation of the image processing apparatus 3 will be described.
The operation of the image processing apparatus 3 is generally the same as that shown in FIG. 2, and the contents of the process executed by the in-region curvature feature quantity calculation unit 310 in step S40 and the process executed by the abnormality determination unit 220 in step S50 are implemented. This is different from the first form.

  FIG. 30 is a flowchart showing processing executed by the in-region curvature feature quantity calculation unit 310. First, in step S431, the directional feature value calculation unit 311 calculates the curvature center direction and gradient direction of the arc as a feature value indicating the center direction of the arc. The processing executed by the direction feature amount calculation unit 311 will be described in detail with reference to FIG.

  In step S4311, the curvature center direction calculation unit 311a calculates the curvature center direction of the arc along the linear region. Specifically, first, similarly to step S4012 shown in FIG. 12, the curvature of each arc is calculated, and further, a curvature radius R that is the reciprocal of the curvature is calculated.

Subsequently, the curvature center direction calculation unit 311a creates an approximate circle by executing a hough transformation using the position coordinates (x, y) and the curvature radius R on the arc (reference: CG-ARTS). Association, “Digital Image Processing”, pp. 213-214). Here, the detection of the approximate circle by the Hough transform is performed by voting a circle having a radius R passing through the position coordinates (x, y) on the arc to a parameter space including the center coordinates (a, b) and the radius R of the circle, the voting results can be Ri creation by the examining.

  Further, the curvature center direction calculation unit 311a calculates a direction vector from an arbitrary point on the arc (for example, the center point) toward the center of the approximate circle, and outputs this direction vector as the curvature center direction.

  In subsequent step S4312, the gradient direction calculation unit 311b calculates the gradient direction of the subject at the position of the arc along the linear region. Specifically, the gradient direction calculation unit 311b reduces the absorption of hemoglobin among the pixel values of each pixel from the portion corresponding to the attention area to be processed in the original intraluminal image, and the surface layer of the mucous membrane. An R component value closest to the shape is acquired to create an R component image.

  In step S4312, the gradient direction may be acquired from an image created by another method as long as the image has little influence on the fine structure of the fur on the mucous membrane. For example, an image may be used by the opening process using the light and shade morphology described in the process of the convex high-frequency component calculation unit 121.

  Subsequently, the gradient direction calculation unit 311b calculates a gradient direction at the position of the arc from which the curvature center direction is calculated by the curvature center direction calculation unit 311a from the R component image. Specifically, first-order differential filter processing (Prewit filter, Sobel filter, etc.) which is filter processing for calculating edge strength (Reference: CG-ARTS Association, “Digital Image Processing, pages 114-117”) Execute.

  In step S <b> 4313, the direction feature amount calculation unit 311 outputs the curvature center direction and the gradient direction of the arc as feature amounts. Thereafter, the process returns to the main routine.

  In step S432 following step S431, the frequency distribution creating unit 312 creates the frequency distribution of the feature amount output from the direction feature amount calculating unit 311. Specifically, a frequency distribution in the direction of the center of curvature of the arc is created for each gradient direction of the arc.

  In step S433, the in-region curvature feature amount calculation unit 310 outputs the frequency distribution of the arc in the curvature center direction as the curvature feature amount.

Next, processing executed by the abnormality determination unit 320 will be described.
The abnormality determination unit 320 determines whether or not there is an abnormal part in the region of interest based on the frequency distribution in the curvature center direction that is the curvature feature amount.

Here, when abnormal fur exists in the region of interest, as shown in FIG. 32A, when imaging is performed from one imaging direction with respect to the gradient of the subject (mucosal structure), as shown in FIG. A linear image in which the region PR6 is directed substantially in one direction is obtained. In this case, as shown in FIG. 32C, the curvature center direction d in the line-shaped region PR6 is substantially in a fixed direction. Therefore, as shown in FIG. 32D, the distribution in the direction of the curvature center for each gradient direction is biased.

On the other hand, when a bubble area exists in the attention area, as shown in FIG. 33A, when imaging is performed from one imaging direction with respect to the gradient of the subject (mucosal structure), as shown in FIG. 33B, a linear area PR7 is obtained. A linear image having a circular shape corresponding to the outline of the bubble region is obtained. In this case, as shown in FIG. 33C, the curvature center direction d in the linear region PR7 faces all directions. Therefore, as shown in FIG. 33D, the direction of the center of curvature for each gradient direction is widely and substantially uniformly distributed.

  Therefore, the abnormality determination unit 320 detects an abnormality in the attention area when the curvature center direction is biased in the frequency distribution of the curvature center direction for each gradient direction, that is, when the variance in the curvature center direction is equal to or less than a predetermined threshold. It is determined that the part exists.

  As described above, according to the third embodiment, the mucous membrane region extracted from the intraluminal image is set as the attention region, and the convex region including the pixel group having a pixel value higher than the surrounding pixels is set from the attention region. Extract high frequency components from the region. Then, with respect to the line-shaped region in which the high-frequency component is thinned, the curvature center direction and the gradient direction of the arc along the line-shaped region are calculated, and based on the frequency distribution of the curvature center direction for each gradient direction, Determine whether there is an abnormal part. Therefore, it is possible to accurately identify the abnormal fur and the bubble region according to the direction of the fur changing depending on the imaging direction, and to improve the discrimination accuracy of the abnormal part.

(Embodiment 4)
Next, a fourth embodiment of the present invention will be described.
FIG. 34 is a block diagram illustrating a configuration of an image processing apparatus according to the fourth embodiment. As shown in FIG. 34, the image processing apparatus 4 according to Embodiment 4 includes a calculation unit 400 instead of the calculation unit 100 shown in FIG. The calculation unit 400 includes a region of interest setting unit 410, a linear convex region extraction unit 120, an in-region curvature feature amount calculation unit 420, and an abnormality determination unit 430. The configuration and operation of the linear convex region extraction unit 120 are the same as those in the first embodiment. The configuration and operation of each unit of the image processing apparatus 4 other than the calculation unit 400 are the same as those in the first embodiment.

  The attention area setting unit 410 excludes a region other than the mucous membrane such as a residue or a dark part from the intraluminal image to be processed, and extracts a mucosal region, and further extracts the extracted mucosal region into a plurality of regions. And an area dividing unit 411 that divides the area into regions of interest. The operation of the mucous membrane region extraction unit 111 is the same as that in the first embodiment.

  The in-region curvature feature amount calculation unit 420 includes a size feature amount calculation unit 421 including a curvature calculation unit 131a and a curvature representative value calculation unit 131b, and a frequency distribution creation unit 132. The operations of the curvature calculation unit 131a, the curvature representative value calculation unit 131b, and the frequency distribution creation unit 132 are the same as those in the first embodiment.

  The abnormality determination unit 430 determines whether or not there is an abnormal part in each region of interest based on the curvature feature value calculated by the in-region curvature feature value calculation unit 420.

Next, the operation of the image processing device 4 will be described.
The operation of the image processing apparatus 4 is the same as that shown in FIG. 2 as a whole, and the contents of the processing executed in each of steps S20 to S60 are different from those in the first embodiment.

  FIG. 35 is a flowchart showing the processing executed by the attention area setting unit 410 in step S20. Note that steps S201 to S205 in FIG. 35 correspond to FIG.

  In step S241 following step S205, the attention area setting unit 410 divides the mucous membrane area remaining after removing the unnecessary area into a plurality of areas each having a predetermined size or less. The division method is not particularly limited, and in the fourth embodiment, the mucous membrane area is divided into rectangular areas. In addition, the size of one divided region is set in advance so that the difference in the imaging distance to the subject within the divided region is within a predetermined range.

  In subsequent step S242, attention area setting section 410 outputs the coordinate information of each divided area as the attention area.

  In step S <b> 30, the linear shape convex area extraction unit 120 extracts a linear area from each of the attention areas output in step S <b> 20.

  In step S40, the in-region curvature feature amount calculation unit 420 calculates a curvature feature amount based on the curvature of the arc along the linear region for each region of interest output in step S20. As the curvature feature quantity, the frequency distribution of the representative value of the curvature of one or more arcs calculated from each section of the linear region is calculated as in the first embodiment. In contrast, imaging information is not calculated.

  In step S50, the abnormality determining unit 430 determines, for each attention area, whether or not there is an abnormal part in the attention area based on the frequency distribution of the representative value of the curvature of the arc that is the curvature feature amount. Specifically, first, when the curvature is smaller than a predetermined threshold value (curvature threshold value), it is determined that the mucous membrane outline is reflected in the region of interest and no abnormal portion exists.

  In addition, when the curvature is larger than the curvature threshold and the variance of the curvature is larger than a predetermined threshold (dispersion threshold), the abnormality determination unit 430 shows a bubble region in the attention region, and there is no abnormal portion. Is determined. This is because even if the imaging distance is constant, the bubble region inherently includes bubbles with various curvatures.

  On the other hand, when the curvature is greater than the curvature threshold and the variance of the curvature is equal to or less than a predetermined threshold (dispersion threshold), the abnormality determination unit 430 indicates that abnormal fur is reflected in the region of interest and an abnormal portion exists Determine. This is because abnormal soft hairs in the vicinity region originally have similar shapes, so that the size of the attention region is small, and the abnormal soft hair can be ignored when the difference in imaging distance in one attention region can be ignored. This is because the curvature of the arc corresponding to the contour of the hair is uniform.

  In step S60, the calculation unit 400 outputs the result of the determination made for each region of interest in step S50 together with the coordinate information of the region of interest. In response to this, the control unit 10 causes the display unit to display the determination result and records it in the recording unit 50 in association with the image data of the intraluminal image to be processed. At this time, the control unit 10 may cause the intraluminal image displayed on the display unit 40 to display a mark or the like indicating the position of the region of interest determined to have an abnormal part.

  As described above, according to the fourth embodiment, a plurality of regions of interest are set by dividing the mucosal region extracted from the intraluminal image, and the pixel value of each region of interest is higher than that of surrounding pixels. A high-frequency component is extracted from a convex region composed of high pixel groups. Then, a curvature frequency distribution is calculated as a curvature feature amount for the line-shaped region in which the high-frequency component is thinned, and a subject reflected in the attention region is determined based on the feature amount. Therefore, it is possible to accurately identify the abnormal fur, the bubble region, and the mucous membrane outline and determine the presence or absence of the abnormal portion.

  Further, according to the fourth embodiment, the mucous membrane region extracted from the intraluminal image is divided so that the difference in imaging distance to the subject can be ignored, and each divided region is set as a region of interest. Since each process for determining the presence or absence of an abnormal part is performed, the calculation process of the distance information and the determination process of the abnormal part according to the distance information can be omitted, and the calculation process can be simplified.

  Further, according to the fourth embodiment, since the presence / absence of an abnormal part is determined for each region of interest, it is possible to specify the existence range of the abnormal part in the intraluminal image.

(Modification 4-1)
Next, Modification 4-1 of Embodiment 4 will be described.
The size of the attention area set by the attention area setting section 410 may be variable. Hereinafter, the operation of the image processing apparatus 4 when the size of the attention area is variable will be described. FIG. 36 is a flowchart showing the operation of the image processing apparatus 4 in Modification 4-1.

In step S20, the attention area setting unit 410 sets the remaining mucosal area from which the unnecessary area is removed as one attention area.
Processing contents in subsequent steps S30 to S50 are the same as those in the fourth embodiment.

  In step S71 following step S50, the calculation unit 400 determines whether or not a determination result indicating that an abnormal part exists in any of the set attention areas has been obtained. In the first round, there is one region of interest.

  When the determination result that the abnormal part exists is obtained (step S71: Yes), the process proceeds to step S60. Note that the processing content in step S60 is the same as in the fourth embodiment.

On the other hand, when the determination result that an abnormal part exists is not obtained (step S71: No), the arithmetic part 400 determines whether or not the size of the attention area currently set is equal to or smaller than a predetermined size ( Step S72). Note that, as the predetermined size at this time, a size that can ignore the difference in imaging distance with respect to the subject in the attention area is set.
If the size of the attention area is equal to or smaller than the predetermined size (step S72: Yes), the process proceeds to step S60.

  On the other hand, when the size of the attention area is larger than the predetermined size (step S72: No), the attention area setting unit 410 makes the attention area smaller than the currently set attention area and sets the attention area with respect to the mucous membrane area. Reset is performed (step S73). Thereafter, the process proceeds to step S30.

  As described above, according to the present modification example 4-1, the presence / absence of an abnormal portion in each region of interest is determined while gradually reducing the size of the region of interest. Here, by reducing the size of the region of interest, the difference in imaging distance to the subject in each region of interest is reduced, so that each time the processing from step S30 to step S73 is completed, the abnormal portion for each region of interest is detected. The discrimination accuracy will increase. In this way, by changing the discrimination accuracy of the abnormal part with respect to the intraluminal image from the lower side to the higher side, it is possible to improve the efficiency of the abnormal part discrimination process and suppress the discrimination of the abnormal part.

  The image processing apparatuses according to the first to fourth embodiments and the modifications described above can be realized by executing an image processing program recorded on a recording medium by a computer system such as a personal computer or a workstation. . Further, such a computer system may be used by being connected to other computer systems, servers, or other devices via a public network such as a local area network, a wide area network (LAN / WAN), or the Internet. good. In this case, the image processing apparatuses according to the first to fourth embodiments acquire image data of intraluminal images via these networks, or various output devices (viewers and viewers) connected via these networks. The image processing result may be output to a printer or the like, or the image processing result may be stored in a storage device (such as a recording medium and its reading device) connected via these networks.

  The present invention is not limited to the first to fourth embodiments and the modifications thereof, and various inventions can be made by appropriately combining a plurality of constituent elements disclosed in the embodiments and modifications. Can be formed. For example, some constituent elements may be excluded from all the constituent elements shown in each embodiment or modification, or may be formed by appropriately combining the constituent elements shown in different embodiments or modifications. May be.

DESCRIPTION OF SYMBOLS 1, 1-2, 2-4 Image processing apparatus 10 Control part 20 Image acquisition part 30 Input part 40 Display part 50 Recording part 51 Image processing program 100, 100-2, 200, 300, 400 Calculation part 110, 410 Attention area Setting unit 111 Mucosal region extraction unit 120 Linear convex region extraction unit 121 Convex shape high frequency component calculation unit 122, 152 Isolated point exclusion unit 123, 153 Thinning unit 130, 210, 310, 420 Intra-region curvature feature amount calculation unit 131 Size Feature amount calculation unit 131a, 211a Curvature calculation unit 131b Curvature representative value calculation unit 131c Distance information calculation unit 132, 212, 312 Frequency distribution generation unit 140, 220, 320, 430 Abnormality determination unit 150 Linear shape convex region extraction unit 151 Ridge shape Extraction unit 211 Shape feature amount calculation unit 211b Curvature standard deviation calculation unit 311 Direction feature The amount calculating section 311a curvature center direction calculating unit 311b gradient direction calculating section 411 area dividing unit 421 Size feature amount calculation unit

Claims (11)

An attention area setting section for setting an attention area in the intraluminal image;
In the region of interest, a line-shaped convex region extracting unit that extracts a line-shaped region in which pixels having pixel values higher than the surroundings are continuous for a predetermined number of pixels;
Curvature of one or more arcs along the linear region in each section of the linear region delimited by an end point of the linear region or an intersection of the linear regions, and in the lumen An in-region curvature feature amount calculation unit having a size feature amount calculation unit for calculating distance information from the imaging position of the image to the linear region;
In the case where the dispersion of the curvature is equal to or less than a dispersion threshold determined according to the distance information, and the curvature is distributed in a range equal to or greater than the curvature threshold determined according to the distance information, the region of interest An abnormality discriminating unit for discriminating that an abnormal part of fur is present;
An image processing apparatus comprising: An attention area setting section for setting an attention area in the intraluminal image;
In the region of interest, a line-shaped convex region extracting unit that extracts a line-shaped region in which pixels having pixel values higher than the surroundings are continuous for a predetermined number of pixels;
A shape for calculating variation in curvature of one or more arcs along the line-shaped region in each section of the line-shaped region delimited by an end point of the line-shaped region or an intersection of the line-shaped regions. An in-region curvature feature quantity calculation unit having a feature quantity calculation unit;
If the variation is greater than a predetermined value, an abnormality determination unit that determines that there is an abnormal portion of fur in the region of interest;
An image processing apparatus comprising: An attention area setting section for setting an attention area in the intraluminal image;
In the region of interest, a line-shaped convex region extracting unit that extracts a line-shaped region in which pixels having pixel values higher than the surroundings are continuous for a predetermined number of pixels;
A direction from each of one or more arcs along the line-shaped region toward the center of curvature in each section of the line-shaped region delimited by an end point of the line-shaped region or an intersection of the line-shaped regions And an in-region curvature feature quantity calculation unit having a direction feature quantity calculation unit for calculating a direction in which the subject is inclined in the depth direction of the image at each position of the one or more arcs;
An abnormality discriminating unit that discriminates that an abnormal part of fur is present in the region of interest when the variance of the frequency in the direction toward the center of curvature created for each direction in which the subject is inclined is equal to or less than a predetermined threshold;
An image processing apparatus comprising: The size feature amount calculation unit includes:
From each section of the area of the line shape delimited by the intersection between the regions of the end point or the line shape of the region of the line shape, a curvature calculation unit for calculating a curvature of one or more arcs,
A curvature representative value calculating unit that calculates a representative value from the curvature of the one or more arcs;
A distance information calculation unit for calculating distance information from the imaging position of the intraluminal image to the linear region;
The image processing apparatus according to claim 1, further comprising:   The attention area setting unit, when it is determined that the abnormal part does not exist with respect to the set attention area, reduces the size of the attention area and resets the attention area. The image processing apparatus according to claim 1. An operation method of an image processing apparatus,
A region of interest setting step in which the region of interest setting unit sets a region of interest in the intraluminal image;
A line-shaped convex region extracting step, wherein the line-shaped convex region extracting unit extracts a line-shaped region in which the pixels having a pixel value higher than the surroundings in the region of interest continues for a predetermined number of pixels;
The in-region curvature feature quantity calculation unit includes one or more along the line-shaped region in each section of the line-shaped region divided by an end point of the line-shaped region or an intersection of the line-shaped regions. An in-region curvature feature amount calculating step for calculating the curvature of the arc and the distance information from the imaging position of the intraluminal image to the linear region;
When the abnormality determination unit distributes the curvature in a range equal to or greater than the curvature threshold determined according to the distance information when the curvature dispersion is equal to or less than a dispersion threshold determined according to the distance information , An abnormality determination step for determining that an abnormal portion of fur is present in the region of interest;
A method for operating an image processing apparatus, comprising : An operation method of an image processing apparatus,
A region of interest setting step in which the region of interest setting unit sets a region of interest in the intraluminal image;
A line-shaped convex region extracting step, wherein the line-shaped convex region extracting unit extracts a line-shaped region in which the pixels having a pixel value higher than the surroundings in the region of interest continues for a predetermined number of pixels;
The in-region curvature feature quantity calculation unit includes one or more along the line-shaped region in each section of the line-shaped region divided by an end point of the line-shaped region or an intersection of the line-shaped regions. An in-region curvature feature amount calculating step for calculating variation in curvature of the arc;
An abnormality determining step, when the variation is larger than a predetermined value, an abnormality determining step for determining that a fur abnormal portion exists in the region of interest;
A method for operating an image processing apparatus, comprising: An operation method of an image processing apparatus,
A region of interest setting step in which the region of interest setting unit sets a region of interest in the intraluminal image;
Line shape protruding region extraction unit, in the region of interest, a line-shaped convex region extraction step of extracting the line-shaped region in which pixels higher pixel value than surrounding or more consecutive predetermined number of pixels,
The in-region curvature feature quantity calculation unit includes one or more along the line-shaped region in each section of the line-shaped region divided by an end point of the line-shaped region or an intersection of the line-shaped regions. An in-region curvature feature amount calculating step of calculating a direction from each arc toward the center of curvature and a direction in which the subject is inclined in the depth direction of the image at each of the one or more arcs;
An abnormality determination unit that determines that there is an abnormal portion of fur in the region of interest when a variance in the frequency of the direction toward the center of curvature created for each direction in which the subject is inclined is equal to or less than a predetermined threshold value Steps,
A method for operating an image processing apparatus, comprising: A region of interest setting step for setting a region of interest in the intraluminal image;
In the region of interest, a line-shaped convex region extraction step for extracting a line-shaped region in which pixels having a pixel value higher than that of the surrounding region continue for a predetermined number of pixels;
Curvature of one or more arcs along the linear region in each section of the linear region delimited by an end point of the linear region or an intersection of the linear regions, and in the lumen An in-region curvature feature amount calculating step for calculating distance information from an image capturing position to the linear region;
In the case where the dispersion of the curvature is equal to or less than a dispersion threshold determined according to the distance information, and the curvature is distributed in a range equal to or greater than the curvature threshold determined according to the distance information, the region of interest An abnormality determination step for determining that an abnormal part of fur is present;
An image processing program for causing a computer to execute. A region of interest setting step for setting a region of interest in the intraluminal image;
In the region of interest, a line-shaped convex region extraction step for extracting a line-shaped region in which pixels having a pixel value higher than that of the surrounding region continue for a predetermined number of pixels;
An area for calculating a variation in curvature of one or more arcs along the line-shaped area in each section of the line-shaped area divided by an end point of the line-shaped area or an intersection of the line-shaped areas An internal curvature feature amount calculating step;
When the variation is larger than a predetermined value, an abnormality determination step for determining that an abnormal portion of fur is present in the region of interest;
An image processing program for causing a computer to execute. A region of interest setting step for setting a region of interest in the intraluminal image;
In the region of interest, a line-shaped convex region extraction step for extracting a line-shaped region in which pixels having a pixel value higher than that of the surrounding region continue for a predetermined number of pixels;
A direction from each of one or more arcs along the line-shaped region toward the center of curvature in each section of the line-shaped region delimited by an end point of the line-shaped region or an intersection of the line-shaped regions And an in-region curvature feature amount calculating step for calculating a direction in which the subject is inclined in the depth direction of the image at each position of the one or more arcs;
An abnormality determination step of determining that there is an abnormal portion of fur in the region of interest when the variance of the frequency in the direction toward the center of curvature created for each direction in which the subject is inclined is equal to or less than a predetermined threshold;
An image processing program for causing a computer to execute.

Images