JP7637683B2 - Processor device and method for operating the processor device - Google Patents

Description

The present invention relates to a processor device and a method of operating the processor device for obtaining analysis results from endoscopic images captured by an endoscope.

In the medical field, diagnosis using medical images is widely practiced. For example, an endoscopic system equipped with a light source device, an endoscope, and a processor device is widely known as an apparatus that uses medical images. In an endoscopic system, an endoscopic image is obtained as a medical image by irradiating an observation object with illumination light and capturing an image of the observation object illuminated by the illumination light. The endoscopic image is used for diagnosis, etc. by being displayed on a display, etc.

Furthermore, recent endoscope systems analyze endoscopic images to detect and highlight areas of interest, such as lesion areas, to assist users in making diagnoses. The following Patent Document 1 describes a configuration that detects candidates for areas of interest, calculates a reliability for each detected candidate that indicates the likelihood that the candidate is an area of interest, and varies the manner in which each candidate area of interest is highlighted depending on the calculated reliability.

Patent No. 5802364

However, conventional devices have a problem in that they are sometimes unable to perform accurate analysis. In other words, conventional devices perform their analysis under the assumption that the endoscopic images being analyzed are appropriate. However, some endoscopic images are blurred or out of focus, making them less than appropriate for analysis, and analyzing such endoscopic images does not produce accurate analysis results.

The present invention has been made in consideration of the above background, and aims to provide a processor device and a method for operating a processor device that can obtain more accurate analysis results.

In order to achieve the above objective, the processor device of the present invention obtains the analysis results of an endoscopic image captured by an endoscope. In the processor device, an image processing processor determines whether the endoscopic image satisfies specified criteria required for analysis, and if it is determined that the specified criteria are not satisfied, issues a warning regarding the reliability of the analysis.

The amount of blur in the endoscopic image may be set as a specified criterion, and if the amount of blur is equal to or greater than the specified amount, it may be determined that the specified criterion is not met.

The degree of focus of the endoscopic image may be set as a specified criterion, and if the degree of focus is equal to or lower than the specified degree of focus, it may be determined that the specified criterion is not met.

The brightness of the endoscopic image may be set as a specified criterion, and if the brightness is equal to or lower than the specified criterion, it may be determined that the specified criterion is not met.

As a specified criterion, the percentage of the halation area with the maximum brightness gradation value is set, and if the percentage of the halation area is equal to or greater than the specified percentage, it may be determined that the specified criterion is not met.

As a specified criterion, the percentage of the area of adhesions in which adhesions adhering to the imaging optical system are imaged may be set, and if the percentage of the area of adhesions is equal to or greater than the specified percentage, it may be determined that the specified criterion is not met.

When calculating features related to biometric information through analysis, if the features are below a predetermined amount, it may be determined that the specified criteria are not met.

When detecting an area of interest from an endoscopic image through analysis, if the area of interest is more than a specified distance away from the center of the endoscopic image, it may be determined that the area does not satisfy a specified criterion.

When analyzing multiple endoscopic images capturing a common observation target, the specified criteria may consist of a primary criterion and a secondary criterion relating to the proportion of endoscopic images that satisfy the primary criterion, and if the proportion of endoscopic images that satisfy the primary criterion is equal to or lower than the secondary criterion, it may be determined that the specified criteria are not satisfied.

When performing normal imaging, in which normal light is irradiated onto the object to be observed and a normal image is captured as an endoscopic image, and special imaging, in which special light having a different emission spectrum from normal light is irradiated onto the object to be observed and a special image is captured as an endoscopic image, the normal image and the special image may be compared to determine whether they satisfy a specified standard.

In the comparison, the analysis results of normal images and special images may be compared.

When calculating features related to biometric information through analysis, if the feature obtained through analysis of a normal image is equal to or greater than a predetermined standard amount and the feature obtained through analysis of a special image is less than the predetermined standard amount, it may be determined that the predetermined standard is not met.

In addition, in order to achieve the above-mentioned objective, the processor device of the present invention, in a method of operating the processor device, obtains analysis results of an endoscopic image captured by an endoscope, in which the image processing processor determines whether the endoscopic image satisfies specified criteria required for analysis, and if it is determined that the specified criteria are not satisfied, issues a warning regarding the reliability of the analysis.

According to the present invention, more accurate analysis results can be obtained.

FIG. 1 is an external view of an endoscope system. 2 is a block diagram showing the functions of the endoscope system. FIG. 4 is a block diagram showing the functions of a detailed observation mode image processing unit. FIG. 13 is a flowchart showing a procedure for performing a determination process and a warning process. 13 is a flowchart illustrating an example of a determination process. 13 is a flowchart illustrating an example of a determination process. 13 is a flowchart illustrating an example of a determination process. 13 is a flowchart illustrating an example of a determination process. 13 is a flowchart illustrating an example of a determination process. 13 is a flowchart illustrating an example of a determination process. 13 is a flowchart illustrating an example of a determination process. 13 is a flowchart illustrating an example of a determination process. 13 is a flowchart illustrating an example of a determination process. 13 is a flowchart showing a flow of a determination process, a warning process, and an advice process;

1, the endoscope system 10 includes an endoscope 12, a light source device 14, a processor device 16, a display 18, and a user interface 19. The endoscope 12 is optically connected to the light source device 14 and electrically connected to the processor device 16.

The endoscope 12 has an insertion section 12a that is inserted into the body of an object to be observed, an operating section 12b provided at the base end of the insertion section 12a, and a bending section 12c and a tip section 12d provided at the tip side of the insertion section 12a. An angle knob 12e is provided on the operating section 12b, and by operating the angle knob 12e, the bending section 12c is bent so that the tip section 12d can be oriented in a desired direction.

The operation unit 12b is also provided with a mode changeover switch 12f used to change the observation mode, a still image acquisition instruction unit 12g used to instruct acquisition of a still image of the observation target, and a zoom operation unit 12h used to operate the zoom lens 43 (see FIG. 2). The endoscope system 10 has, as observation modes, a normal observation mode and a detailed observation mode for observing in more detail than in the normal observation mode, and the observation mode is switched by operating the mode changeover switch 12f.

The light source device 14 is equipped with a light source unit 20 (see FIG. 2) that emits illumination light for illuminating an observation object. The illumination light from the light source unit 20 is guided by a light guide 25 (see FIG. 2) and irradiated from the tip 12d toward the observation object. The observation object illuminated by the illumination light from the light source unit 20 is imaged by an image sensor 44 (see FIG. 2) built into the tip 12d.

The processor unit 16 is electrically connected to the display 18 and the user interface 19. The display 18 outputs and displays an image of the object to be observed and information associated with the image of the object to be observed. The user interface 19 is a keyboard, mouse, and/or touchpad, and has a function of accepting input operations such as function settings. An external recording unit (not shown) for recording images and image information may be connected to the processor unit 16.

In FIG. 2, the light source device 14 comprises the light source unit 20 described above and a light source processor 21 that controls the light source unit 20, and the emission timing and emission spectrum of the illumination light emitted from the light source unit 20 are controlled by the light source processor 21.

In this embodiment, the light source device 14 emits normal light and special light that have different emission spectra. The normal light is, for example, white light. The white light includes, for example, purple light in the wavelength band of 380 to 420 nm, blue light in the wavelength band of 420 to 500 nm, green light in the wavelength band of 480 to 600 nm, and red light in the wavelength band of 600 to 650 nm.

For example, the special light emits a greater amount of violet light in the wavelength band of 380 nm to 420 nm, which has a high absorption coefficient for hemoglobin in blood vessels, than normal light. As a result, the special image (an endoscopic image captured by irradiating the object of observation with special light) described below has higher resolution of blood vessel and duct structures than other structures. Note that the type of special light is not limited to one, and a configuration may be used in which multiple types of special light are used to capture multiple types of special images.

Illumination light from the light source unit 20 is incident on the aforementioned light guide 25 via the optical path coupling unit 23, which is composed of a mirror, a lens, etc. The light guide 25 is built into the endoscope 12 and the universal cord (a cord that connects the endoscope 12 with the light source device 14 and the processor device 16). The light guide 25 propagates the light from the optical path coupling unit 23 to the tip 12d of the endoscope 12.

The tip 12d of the endoscope 12 is provided with an illumination optical system 30a and an imaging optical system 30b. The illumination optical system 30a has an illumination lens 32, and the illumination light propagated by the light guide 25 is irradiated onto the observation object via the illumination lens 32. The imaging optical system 30b has an objective lens 42, a zoom lens 43, and an imaging sensor 44. Light from the observation object irradiated with the illumination light is incident on the imaging sensor 44 via the objective lens 42 and the zoom lens 43. As a result, an image of the observation object is formed on the imaging sensor 44. The zoom lens 43 is a lens for enlarging the observation object, and is moved between the telephoto end and the wide end by operating the zoom operation unit 12h.

The imaging sensor 44 is a color sensor, and in this embodiment, a primary color sensor is used that has three types of pixels: a B pixel with a B (blue) color filter, a G pixel with a G (green) color filter, and an R pixel with an R (red) color filter. As such an imaging sensor 44, a CCD (Charge-Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) imaging sensor or the like can be used. The imaging sensor 44 is controlled by an imaging processor 45. Specifically, the imaging processor 45 reads the signal of the imaging sensor 44, and an image signal is output from the imaging sensor 44.

The imaging processor 45 also controls the light source processor 21 described above to perform normal imaging, which irradiates the observation subject with normal light to capture an image (signal readout), and special imaging, which irradiates the observation subject with special light to capture an image. In normal imaging, a normal image is captured as an endoscopic image, and in special imaging, a special image is captured as an endoscopic image. The imaging cycle between normal imaging and special imaging can be freely set, but in this embodiment, five normal images (five frames) and one special image (one frame) are repeatedly captured. In this embodiment, imaging is performed 60 times (60 frames) per second. As a result, in this embodiment, 50 normal images (50 frames) and 10 special images (10 frames) are acquired per second.

In addition, instead of the primary color image sensor 44 having RGB primary color filters, a complementary color image sensor having C (cyan), M (magenta), Y (yellow) and G (green) complementary color filters may be used. When a complementary color image sensor is used, image signals of the four colors CMYG are output. Therefore, by converting the four CMYG image signals into three RGB image signals by complementary color-primary color conversion, it is possible to obtain image signals of each of the RGB colors similar to those of the image sensor 38. Also, instead of the image sensor 38, a monochrome sensor without color filters may be used.

The CDS/AGC (Correlated Double Sampling/Automatic Gain Control) circuit 46 performs correlated double sampling (CDS) and automatic gain control (AGC) on the analog image signal obtained from the image sensor 44. The image signal that passes through the CDS/AGC circuit 46 is converted into a digital image signal by an A/D (Analog/Digital) converter 48. The digital image signal after A/D conversion is input to the processor device 16.

The processor device 16 is equipped with a central control unit 68 that constitutes the image processing processor of the present invention. The central control unit 68 is a hardware resource for executing program instructions stored in a memory 69, and drives and controls each part of the processor device 16 to execute the program instructions. Through the drive control of the central control unit 68 associated with the execution of the program instructions, the processor device 16 functions as an image acquisition unit 50, a DSP (Digital Signal Processor) 52, a noise reduction unit 54, an image processing unit 58, and a display control unit 60.

The image acquisition unit 50 acquires an endoscopic image input from the endoscope 12. In this embodiment, the endoscopic image is a color image composed of a blue signal (B image signal), a green signal (G image signal), and a red signal (R image signal) output from the B pixel, G pixel, and R pixel of the imaging sensor 44. In addition, in this embodiment, normal imaging is performed by irradiating the observation target with normal light and special imaging is performed by irradiating the observation target with special light. As a result, in this embodiment, a normal image by normal imaging and a special image by special imaging are acquired as an endoscopic image, which is a color image. Furthermore, in this embodiment, five normal imaging sessions and one special imaging session are repeatedly performed. As a result, five normal images (frames) and one special image (frame) are repeatedly acquired.

The endoscopic image acquired by the image acquisition unit 50 is transmitted to the DSP 52. The DSP 52 performs various signal processing such as defect correction processing, offset processing, gain correction processing, matrix processing, gamma conversion processing, demosaic processing, and YC conversion processing on the received endoscopic image.

In the defect correction process, the signal of the defective pixel of the imaging sensor 44 is corrected. In the offset process, dark current components are removed from the image signal that has been subjected to the defect correction process, and an accurate zero level is set. In the gain correction process, the signal level of the endoscopic image is adjusted by multiplying the image signal of each color after the offset process by a specific gain coefficient. Note that, when a monochrome sensor is used as the imaging sensor 44, it is preferable that the endoscopic image is captured by the monochrome sensor for each emission of light of a specific color, and a multiple-color monochrome image is output from the monochrome sensor.

After gain correction processing, the image signals of each color are subjected to matrix processing to improve color reproducibility. Then, the brightness and saturation of the endoscopic image are adjusted by gamma conversion processing. After matrix processing, the endoscopic image is subjected to demosaic processing (also called isotropic processing or synchronization processing), and signals of the missing colors of each pixel are generated by interpolation. Through demosaic processing, all pixels have signals of each of the RGB colors. The DSP 52 performs YC conversion processing on the endoscopic image after demosaic processing, and outputs the luminance signal Y and the color difference signals Cb and Cr to the noise reduction unit 54.

The noise reduction unit 54 performs noise reduction processing, for example using the moving average method or median filter method, on the endoscopic image that has been subjected to demosaic processing etc. by the DSP 56. The endoscopic image with reduced noise is input to the image processing unit 58. Then, after various image processing is performed in the image processing unit 58, it is input to the display control unit 60. The display control unit 60 controls the display of the display 18, and the endoscopic image that has been subjected to various image processing in the image processing unit 58 is converted into a video signal for display in the display control unit 60 and displayed on the display 18.

The image processing unit 58 includes a normal observation mode image processing unit 62 and a detailed observation mode image processing unit 64. The normal observation mode image processing unit 62 operates when the observation mode is the normal observation mode. On the other hand, the detailed observation mode image processing unit 64 operates when the observation mode is the detailed observation mode.

The normal observation mode image processing unit 62 performs color conversion processing, color enhancement processing, and structure enhancement processing on the endoscopic image input from the noise reduction unit 54. In the color conversion processing, color conversion processing is performed on the endoscopic image using 3x3 matrix processing, gradation conversion processing, and three-dimensional LUT (Look Up Table) processing. Color enhancement processing is performed on the endoscopic image that has already been color converted. Structure enhancement processing is processing that emphasizes the structure of the object of observation, and is performed on the endoscopic image after color enhancement processing. The endoscopic image that has undergone the above-mentioned image processing is then displayed on the display 18 via the display control unit 60.

The detailed observation mode image processing unit 64 also performs image processing similar to that performed by the normal observation mode processing unit, and the processed endoscopic image is displayed on the display 18. As shown in FIG. 3, the detailed observation mode image processing unit 64 is provided with a recognition processing unit 70, an image determination unit 72, and a warning processing unit 74. The recognition processing unit 70 analyzes the endoscopic image input from the noise reduction unit 54 and performs recognition processing. The recognition processing performed by the recognition processing unit 70 includes a detection processing for detecting an attention area from the recognition image (in this embodiment, the endoscopic image) and a discrimination processing for discriminating the type of lesion contained in the recognition image. The discrimination processing includes processing performed on the attention area and processing performed on the entire recognition image, but in this embodiment, an example of performing discrimination processing on the attention area will be described.

In the recognition process, for example, an endoscopic image is divided into a number of small regions, and image features are calculated from the divided endoscopic image. In the detection process, it is determined whether each small region is a lesion based on the calculated features, a group of regions identified as the same type is extracted as one lesion, and the region containing the extracted lesion is detected as a region of interest. In the discrimination process, the type and/or degree (stage) of the lesion is determined for the detected region of interest based on the features within the region of interest and the state (position, size, shape, etc.) of the region of interest. The judgment in the above-mentioned recognition process (detection process, discrimination process) is performed by machine learning algorithms such as a convolutional neural network and deep learning.

It is preferable that the feature amount calculated from the endoscopic image by the recognition processing unit 70 is related to biological information and is the shape or color of a specific part of the observation target, or an index value obtained from the shape, color, etc. Furthermore, it is preferable that the feature amount is at least any one of the following: blood vessel density, blood vessel shape, number of blood vessel branches, blood vessel thickness, blood vessel length, blood vessel tortuosity, blood vessel depth, duct shape, duct opening shape, duct length, duct tortuosity, and color information, or a combination of two or more of them.

The result of the above-mentioned recognition process, i.e., the analysis result obtained by the recognition processing unit 70 analyzing the endoscopic image, is displayed on the display 18 via the display control unit 60. Note that in this embodiment, an example in which the recognition processing unit 70 is provided in the endoscopic system 10 (more specifically, the processor device 16) has been described, but the present invention is not limited to this. The recognition processing unit may be provided in an external device other than the processor device 16 or the endoscopic system 10. In this case, the endoscopic image to be analyzed (the subject of the recognition process) is transmitted to the external device, the recognition process (analysis) is performed in the external device, and the result of the recognition process (analysis result) sent from the external device is received (acquired) by the endoscopic system 10 (more specifically, the processor device 16).

In FIG. 4, an endoscopic image that has been recognized by the recognition processing unit 70 is input to the image determination unit 72. The image determination unit 72 analyzes the input endoscopic image and performs a determination process to determine whether or not the input endoscopic image satisfies a predetermined standard required for the recognition process (analysis performed by the recognition processing unit 70). Note that the determination process may be performed on the endoscopic image before the recognition processing is performed by the recognition processing unit 70. Furthermore, the configuration is not limited to performing the recognition process and the determination process in series, and equivalent endoscopic images may be input to each of the recognition processing unit 70 and the image determination unit 72, and the recognition process and the determination process may be performed in parallel.

The warning processing unit 74 is activated when the image determination unit 72 determines that the image does not satisfy the predetermined criteria required for the recognition processing, and performs warning processing. The warning processing is to warn that the reliability of the recognition processing has decreased due to the endoscopic image not satisfying the predetermined criteria required for the recognition processing. The warning is issued, for example, by displaying a message on the display 18 to the effect that the reliability of the recognition processing has decreased, and/or by outputting a warning sound from a speaker (not shown).

For example, the amount of image blur is set as the predetermined criterion used in the judgment process. When the amount of image blur is set as the predetermined criterion, as shown in FIG. 5, the image judgment unit 72 analyzes the endoscopic image to be judged and calculates an index value related to the amount of image blur (the larger the value, the greater the amount of image blur). Then, the calculated index value is compared with a preset threshold value for image blur judgment, and if it is equal to or greater than the threshold value, i.e., if the amount of image blur is equal to or greater than a predetermined amount of blur, it is judged that the predetermined criterion is not met. On the other hand, if the calculated index value is less than the threshold value for image blur judgment, it is judged that the predetermined criterion is met.

The specified criteria are not limited to the amount of image blur. The specified criteria may be the degree of focus, image brightness, halation, adhesions, features, or the position of the area of interest. When the degree of focus is set as the specified criteria, as shown in FIG. 6, the image determination unit 72 analyzes the endoscopic image to be determined and calculates an index value related to the degree of focus (the higher the value, the higher the degree of focus (the better the focus)). Then, the calculated index value is compared with a preset threshold value for determining the degree of focus, and if it is equal to or less than the threshold value, i.e., if the degree of focus is equal to or less than the predetermined degree of focus, it is determined that the specified criteria is not met. On the other hand, if the calculated index value is greater than the threshold value for determining the degree of focus, it is determined that the specified criteria is met.

When image brightness is set as the specified criterion, as shown in FIG. 7, the image judgment unit 72 analyzes the endoscopic image to be judged and detects image brightness (e.g., the average brightness gradation value of each pixel of the entire image or the region of interest). If the detected image brightness is equal to or less than a preset threshold value for brightness judgment, that is, if the image brightness is equal to or less than the specified luminance, it is judged that the specified criterion is not met. On the other hand, if the image brightness is greater than the threshold value for brightness judgment, it is judged that the specified criterion is met.

Furthermore, when halation is set as the specified criterion, as shown in FIG. 8, the image determination unit 72 analyzes the endoscopic image to be determined and detects a halation area in the entire image or in the region of interest. A halation area is an area where the luminance gradation value of the image is at its maximum, that is, an area where the image is blown out. Then, when the proportion of the halation area is equal to or greater than a preset threshold value for determining halation, that is, when the proportion of the halation area is equal to or greater than a specified proportion, the image determination unit 72 determines that the specified criterion is not met. On the other hand, when the proportion of the halation area is less than the threshold value for determining halation, the image determination unit 72 determines that the specified criterion is met.

When an adhesion is set as the specified criterion, as shown in FIG. 9, the image determination unit 72 analyzes the endoscopic image to be determined and detects an adhesion area in the entire image or in the area of interest. The adhesion area is an area in which an image of adhesion such as bodily fluids adhering to an imaging optical system such as the objective lens 42 is captured. Then, the image determination unit 72 determines that the specified criterion is not met if the proportion of the adhesion area is equal to or greater than a preset threshold for determining adhesion, that is, if the proportion of the adhesion area is equal to or greater than a specified proportion. On the other hand, if the proportion of the adhesion area is less than the threshold for determining adhesion, it determines that the specified criterion is met.

Furthermore, when a feature amount is set as the predetermined criterion, as shown in Fig. 10, the image determination unit 72 determines that the predetermined criterion is not satisfied if the feature amount calculated in the recognition process is equal to or less than a preset threshold value for determining the feature amount, i.e., if the feature amount is equal to or less than the predetermined amount. On the other hand, if the feature amount exceeds the threshold value for determining the feature amount, the image determination unit 72 determines that the predetermined criterion is satisfied.

Furthermore, when setting the attention area position as the specified criterion, as shown in Fig. 11, the image determination unit 72 detects the position (center position) of the attention area detected in the recognition process, and calculates the distance between the detected position and the center of the endoscopic image. Then, if the calculated distance is equal to or greater than a preset threshold for determining the attention area position, that is, if the attention area is away from the center of the endoscopic image by a predetermined distance or more, it is determined that the specified criterion is not met. On the other hand, if the calculated distance is less than the threshold for determining the attention area position, it is determined that the specified criterion is met.

In the above embodiment, an example is described in which it is determined that a predetermined criterion is met when one preset criterion is met, but the present invention is not limited to this. It may also be determined that a predetermined criterion is met when all of a plurality of preset criteria are met, or when a predetermined percentage or more of a plurality of preset criteria are met.

As shown in Fig. 12, it may be determined that a predetermined criterion is met when the proportion of endoscopic images that satisfy the primary criterion exceeds the secondary criterion for multiple endoscopic images captured in succession (the predetermined criterion is not met when the proportion of endoscopic images that satisfy the primary criterion is equal to or less than the secondary criterion). Here, the primary criterion may be one or more of the above-mentioned image blur amount, focus degree, image brightness, halation, attachment, feature amount, and attention area position, and the primary criterion is determined to be met when all of these are satisfied or a predetermined proportion or more of them are satisfied. Then, it is determined to be met when the proportion of endoscopic images that satisfy the primary criterion exceeds the secondary criterion (e.g., 10%) among the continuously captured endoscopic images.

It is preferable that the multiple endoscopic images captured consecutively are images of a common observation target. Whether or not the endoscopic images are images of a common observation target can be determined by analyzing the endoscopic images. Also, the endoscopic images captured during one detailed observation mode (from the time of switching to the detailed observation mode until the detailed observation mode ends) may be considered to be images of a common observation target. Furthermore, for the multiple endoscopic images captured consecutively, the user may specify the start and end points of imaging of the common observation target, and the endoscopic images captured during this period may be set as images of a common observation target.

In the above embodiment, the predetermined criteria are the same for normal images and special images, but the predetermined criteria may be different depending on whether the endoscopic image to be judged is a normal image or a special image. In addition, the above embodiment is an example in which it is determined whether the predetermined criteria are met for both normal images and special images, but it may be configured to determine whether the predetermined criteria are met for only one of normal images and special images.

Similarly, in the above embodiment, an example was described in which the contents of the recognition process are common to normal images and special images, but the contents of the recognition process may be different between normal images and special images. Also, the recognition process may be performed only on one of the normal image and the special image.

Furthermore, whether or not a predetermined criterion is met may be determined by comparing the normal image with the special image. In this case, for example, the feature amounts of a normal image and a special image of a common observation target may be compared, and if the difference between the two is equal to or greater than a predetermined value, it may be determined that the predetermined criterion is not met.

In addition, the feature amounts of the normal image and the special image may be compared, and when one is equal to or greater than the reference amount but the other is less than the reference amount, it may be determined that the predetermined criterion is not met. For example, in the above embodiment, the resolution of the vascular structure and ductal structure is higher in the special image than in the normal image, so when feature amounts related to the vascular structure and ductal structure are calculated in the recognition process, it is natural that the feature amount calculated from the special image has a larger amount of information than the feature amount calculated from the normal image. Nevertheless, when the feature amount calculated from the normal image is equal to or greater than the reference amount and the feature amount calculated from the special image is less than the reference amount, for example, when the attention area can be detected in the recognition process from the information image but cannot be detected in the special image, it is highly likely that the endoscopic image to be recognized was not captured under appropriate conditions. For this reason, in the example of FIG. 13, when the attention area was detected in the information image but not in the special image, it is determined that the predetermined criterion is not met.

In the above embodiment, an example of issuing a warning when it is determined that the endoscopic image does not meet the predetermined criteria required for the recognition process (analysis) has been described, but the present invention is not limited to this. As shown in FIG. 14, when it is determined that the endoscopic image does not meet the predetermined criteria required for the recognition process (analysis), an advice process may be performed to advise an imaging method so that an endoscopic image that meets the predetermined criteria required for the recognition process is captured. The advice process is performed, for example, by the warning processing unit 74 (see FIG. 3), and in the advice process, advice on the imaging method is given by displaying a message on the display 18, as in the warning process described above. In the example of FIG. 14, an example of issuing an advice process in addition to a warning process has been described, but the configuration may be such that advice process is executed instead of the warning process.

It is preferable that the contents of advice in the advice process (the contents of the message for advice) are determined by referring to the cause of the judgment that the predetermined criteria are not met. In this case, for example, if it is judged that the predetermined criteria are not met with respect to the amount of image blur, a message is displayed urging the user to "move the camera (tip 12d of the insertion part 12a) slowly." Also, if it is judged that the predetermined criteria are not met with respect to the degree of focus (if the high-frequency components of the image are below the criteria), a message is displayed urging the user to "move the camera (tip 12d of the insertion part 12a) back and forth" to focus. Furthermore, if it is judged that the predetermined criteria are not met with respect to image brightness, a message is displayed urging the user to "increase the amount of illumination light" or "accurately apply illumination light to the object of observation." Also, if it is judged that the predetermined criteria are not met with respect to halation, a message is displayed urging the user to "change the shooting angle." Furthermore, if it is judged that the predetermined criteria are not met with respect to adhesions, a message is displayed urging the user to "clean the objective lens (tip 12d of the insertion part 12a)." Also, when it is determined that the feature amount or the attention area position does not satisfy the predetermined standard, or when it is determined that the attention area is not detected from the special image despite being detected from the general image, a message urging the user to "capture the observation target in front (the center of the image)" is displayed.

In addition, even if the image is re-imaged according to the above-mentioned advice, there are cases where it is determined that the specified criteria are not met. In such cases, it is not only determined that the specified criteria are not met due to inappropriate user operations such as the imaging method, but also includes cases where the specified criteria are not met due to a malfunction on the device side such as an error in recognition processing. And, even in the latter case, repeated warnings and/or advice are problematic as they can hinder diagnosis. For this reason, when warnings and/or advice are given for endoscopic images of a common observation target that do not meet the specified criteria, it is preferable to set an upper limit on the number of warnings and/or advice. Of course, when the endoscopic images are images of a common observation target and the cause of the determination that the specified criteria are not met is also common, an upper limit may be set on the number of warnings and/or advice.

In the above embodiment, an example was described in which switching between the general observation mode and the detailed observation mode was performed by a user operation (operation of the mode changeover switch 12f), but the general observation mode and the detailed observation mode may be configured to be switched automatically without user operation. In this case, for example, it is possible to perform a detection process in the general observation mode, and when a region of interest is detected, to switch to the detailed observation mode and perform a discrimination process. Also, when it is determined in the detailed observation mode that the endoscopic image meets a predetermined standard, that is, when it is deemed that an accurate analysis (recognition process) has been performed, it is possible to display the analysis result (the result of the recognition process) and return to the general observation mode.

In the above embodiment, the hardware structure of the processing units that execute various processes, such as the light source processor 21, the imaging processor 45, the image acquisition unit 50, the DPS 52, the noise reduction unit 54, the image processing unit 58, the normal observation mode image processing unit 62, the detailed observation mode image processing unit 64, the central control unit 68, the recognition processing unit 70, the image determination unit 72, and the warning processing unit 74, is various processors as shown below. The various processors include a CPU (Central Processing Unit), which is a general-purpose processor that executes software (programs) and functions as various processing units, a programmable logic device (PLD), which is a processor whose circuit configuration can be changed after manufacture, such as an FPGA (Field Programmable Gate Array), and a dedicated electric circuit, which is a processor having a circuit configuration designed specifically to execute various processes.

A processing unit may be composed of one of these various processors, or may be composed of a combination of two or more processors of the same or different types (for example, multiple FPGAs, or a combination of a CPU and an FPGA). In addition, multiple processing units may be composed of one processor. As an example of configuring multiple processing units with one processor, first, as represented by computers such as clients and servers, one processor is configured with a combination of one or more CPUs and software, and this processor functions as multiple processing units. Secondly, as represented by system on chip (SoC), there is a form using a processor that realizes the functions of the entire system including multiple processing units with one IC (Integrated Circuit) chip. In this way, the various processing units are configured using one or more of the above various processors as a hardware structure.

More specifically, the hardware structure of these various processors is an electric circuit (circuitry) in the form of a combination of circuit elements such as semiconductor elements. The hardware structure of the memory unit is a storage device such as a hard disk drive (HDD) or a solid state drive (SSD).

10 endoscopic system 12 endoscope 12a insertion section 12b operation section 12c bending section 12d tip section 12e angle knob 12f mode changeover switch 12g still image acquisition instruction section 12h zoom operation section 14 light source device 16 processor device 18 display 19 user interface 20 light source section 21 light source processor 23 optical path coupling section 25 light guide 30a illumination optical system 30b imaging optical system 32 illumination lens 42 objective lens 43 zoom lens 44 imaging sensor 45 imaging processor 46 CDS/AGC circuit 48 A/D converter 50 image acquisition section 52 DSP
54 Noise reduction unit 58 Image processing unit 60 Display control unit 62 Normal observation mode image processing unit 64 Detailed observation mode image processing unit 68 Central control unit 69 Memory 70 Recognition processing unit 72 Image determination unit 74 Warning processing unit

Claims (13)

A processor device for acquiring an analysis result of an endoscopic image captured by an endoscope,
The image processor
determining whether the endoscopic image satisfies a predetermined criterion required for the analysis;
If it is determined that the predetermined criterion is not satisfied, a warning is issued regarding the reliability of the analysis; and
a normal imaging process for irradiating an observation object with normal light and capturing an endoscopic image as the normal image; and a special imaging process for irradiating an observation object with at least one type of special light having an emission spectrum different from that of the normal light and capturing at least one type of special image as the endoscopic image, each of which is automatically and repeatedly performed in a predetermined imaging cycle;
the normal light is white light, and the special light has a higher ratio of purple light than the normal light,
the region of interest is a region including a lesion,
A processor device that determines that the predetermined criterion is not met when the attention area is detected from the normal image and the attention area is not detected from the special image. The amount of blur of the endoscopic image is set as the predetermined criterion,
2. The processor device according to claim 1, wherein when the amount of shaking is equal to or greater than a predetermined amount of shaking, it is determined that the predetermined criterion is not satisfied. As the predetermined criterion, a focus degree of the endoscopic image is set,
3. The processor device according to claim 1, wherein when the degree of focus is equal to or lower than a predetermined degree of focus, it is determined that the predetermined criterion is not satisfied. The luminance of the endoscopic image is set as the predetermined criterion,
The processor device according to claim 1 , wherein when the luminance is equal to or lower than a predetermined luminance, it is determined that the predetermined criterion is not satisfied. The processor device according to any one of claims 1 to 4, wherein the predetermined criterion is set to the percentage of the halation area with the maximum brightness gradation value, and when the percentage of the halation area is equal to or greater than the predetermined percentage, it is determined that the predetermined criterion is not met. The processor device according to any one of claims 1 to 5, wherein the predetermined criterion is set to the percentage of the area of the object that is imaged from the imaging optical system, and when the percentage of the object area is equal to or greater than the predetermined percentage, it is determined that the predetermined criterion is not met. In the case where a feature amount related to biometric information is calculated by the analysis,
The processor device according to claim 1 , wherein when the feature amount is equal to or smaller than a predetermined amount, it is determined that the predetermined criterion is not satisfied. When detecting a region of interest from the endoscopic image by the analysis,
The processor device according to claim 1 , wherein when the region of interest is away from the center of the endoscopic image by a predetermined distance or more, it is determined that the predetermined criterion is not satisfied. In the case of analyzing a plurality of endoscopic images of a common observation target,
the predetermined criteria include a primary criterion and a secondary criterion related to a ratio of the endoscopic images that satisfy the primary criterion,
The processor device according to claim 1 , wherein when a ratio of the endoscopic images that satisfy the primary criterion is equal to or less than the secondary criterion, it is determined that the predetermined criterion is not satisfied. In the case of performing normal imaging in which a target object is irradiated with normal light and a normal image is captured as the endoscopic image, and special imaging in which a target object is irradiated with special light having an emission spectrum different from that of the normal light and a special image is captured as the endoscopic image,
The processor device according to claim 1 , further comprising: a processor that compares the normal image with the special image to determine whether or not the normal image satisfies the predetermined criterion. The processor device according to claim 10, wherein the comparison includes a comparison of the analysis results of the normal image and the special image. In the case where a feature amount related to biometric information is calculated by the analysis,
The processor device according to claim 11, further comprising: a processor that determines that the specified standard is not met if the feature amount obtained by analysis of the normal image is equal to or greater than a specified standard amount and the feature amount obtained by analysis of the special image is less than the specified standard amount. A method for operating a processor device to obtain an analysis result of an endoscopic image captured by an endoscope, comprising:
The image processor
determining whether the endoscopic image satisfies a predetermined criterion required for the analysis;
if it is determined that the predetermined criterion is not satisfied, a warning is issued regarding the reliability of the analysis.
Along with,
a normal imaging process for irradiating an observation object with normal light and capturing an endoscopic image as the normal image; and a special imaging process for irradiating an observation object with at least one type of special light having an emission spectrum different from that of the normal light and capturing at least one type of special image as the endoscopic image, each of which is automatically and repeatedly performed in a predetermined imaging cycle;
the normal light is white light, and the special light has a higher ratio of purple light than the normal light,
the region of interest is a region including a lesion,
A method for operating a processor device, the method including determining that the predetermined criterion is not met when the attention area is detected from the normal image and the attention area is not detected from the special image.

Images