JP7628871B2 - Medical image processing device and method of operation thereof - Google Patents

Description

The present invention relates to a medical image processing device that performs color enhancement processing on an object to be observed, and to a method of operating the same.

In recent years, endoscope systems equipped with a light source device, an endoscope, and a processor device have been widely used in the medical field. In an endoscope system, an endoscope irradiates an observation target with illumination light, and an image of the observation target illuminated by the illumination light is captured by an imaging element in the endoscope, based on the RGB image signals obtained, and an image of the observation target is displayed on a monitor.

In images of objects observed by endoscopes, diagnosis is performed based on color and shape. In this regard, in Patent Document 1, slight color differences between normal and abnormal areas are emphasized by performing a process to expand the color difference between normal and abnormal areas. In Patent Document 2, even when an image of an object containing a mixture of light and dark areas is captured, the color balance of, for example, normal mucosa is maintained between the light and dark areas. Although not related to the field of endoscopy, in Patent Document 3, brightness information is obtained for each hue region from the image signal of an image captured by a digital camera or the like, and exposure control is performed based on the obtained brightness information, thereby optimizing the saturation of the captured image.

Patent No. 5932894 Patent No. 6580778 JP 2013-77879 A

A common method of image enhancement is to treat brightness and color as independent properties and enhance them. Specifically, it has been common to project a color RGB image into a color space such as Lab, HSV, or LHS, and control brightness, hue, and saturation as independent events. In such cases, when emphasizing shading information such as blood concentration or pigment concentration as the target of image enhancement, it is common to adjust brightness or contrast. However, when there is only a slight difference in shading information, adjusting brightness or contrast alone is insufficient to improve visibility.

On the other hand, when emphasizing shading information, it is possible to use a pseudo-color display that uses changes in information unrelated to color, such as a heat map. In this case, shading information can be emphasized by assigning colors that make it easier to distinguish slight differences in shading information, but since the original color is lost, it becomes difficult to distinguish information other than shading from the pseudo-color display.

The present invention aims to provide a medical image processing device and its operating method that can emphasize slight differences in shading information without significantly impairing the original color of the object being observed.

The medical image processing device of the present invention includes a processor, and the processor acquires a first color medical image, acquires first brightness information relating to brightness, which is one of three color attributes, from the image signal value for each pixel constituting the first color medical image, acquires first color information relating to the color information remaining after removing brightness from the three color attributes, and, by referring to the first color information and the first brightness information, if the first brightness information exceeds a predetermined upper limit value, changes the first color information to a first color direction, and if the first brightness information is less than a predetermined lower limit value, changes the first color information to a complementary or inverse color to the first color direction. The first color information is obtained by changing the color in a second color direction that is a pair of colors, the first color information and the second color information are combined according to a first combination coefficient that is set based on the first color information to obtain combined color information, the first brightness information is converted according to a brightness conversion coefficient that is set based on the first color information to obtain second brightness information, the first brightness information and the second brightness information are combined according to a second combination coefficient that is set based on the first color information to obtain combined brightness information, and the combined brightness information and combined color information are converted into a second color medical image in which pixel signal values are changed.

It is preferable that the color indicated by the first color direction is yellow, and the color indicated by the second color direction is blue. It is preferable that the first and second synthesis coefficients are determined according to the color information within the first color gamut in which the first color information is within a predetermined range, and are fixed values outside the first color gamut. It is preferable that the first and second synthesis coefficients are greater than 0 within the first color gamut, are changed according to the first distance or the first angle determined according to the color information, and are set to 0 outside the first color gamut.

It is preferable that the processor acquires the third color information by changing the second distance or the second angle determined according to the color information within a second color gamut within which the second color information falls within a predetermined range. It is preferable that the first color information is any one of R/G, which is a ratio of a red signal R of the first color medical image to a green signal G of the first color medical image, and B/G, which is a ratio of a blue signal to a green signal G of the first color medical image, color difference signals Cr and Cb obtained from the first color medical image, hue and saturation obtained from the first color medical image, and a ^* b ^* of CIE1976La ^* b ^* obtained from the first color medical image.

The first color gamut is preferably a color gamut corresponding to the color of concentrated blood. The first color gamut is preferably a color gamut corresponding to the color of mucous membranes onto which a dye has been applied.

The method of operating the medical image processing device of the present invention includes the steps of: a processor acquiring a first color medical image; acquiring, for each pixel constituting the first color medical image, first brightness information relating to brightness, which is one of three color attributes, from the image signal value; acquiring first color information relating to the color information remaining after removing brightness from the three color attributes; and referring to the first color information and the first brightness information, if the first brightness information exceeds a predetermined upper limit value, changing the first color information in a first color direction, and if the first brightness information is less than a predetermined lower limit value, changing the first color information in a second color direction that is complementary or opposite to the first color direction. the first color information to obtain second color information; the step of obtaining composite color information by combining the first color information and the second color information according to a first combination coefficient set based on the first color information; the step of obtaining second brightness information by converting the first brightness information according to a brightness conversion coefficient set based on the first color information; the step of obtaining composite brightness information by combining the first brightness information and the second brightness information according to a second combination coefficient set based on the first color information; and the step of converting the composite brightness information and the composite color information into a second medical image in color with the image signal value changed.

The present invention makes it possible to emphasize slight differences in shading information without significantly impairing the original color of the object being observed.

1 is an external view of an endoscope system according to a first embodiment. 2 is a block diagram showing the functions of the endoscope system according to the first embodiment. FIG. 1 is a graph showing emission spectra of purple light V, blue light B, green light G, and red light R. 11 is a block diagram showing functions of a gradation information emphasis processing unit when the first color information has a B/G ratio and a G/R ratio. FIG. 13 is a block diagram showing functions of a first color change processing unit when the first color information is a B/G ratio and a G/R ratio. FIG. 13A and 13B are explanatory diagrams showing before and after processing in a first color change processing unit when the first color information is a B/G ratio and a G/R ratio. 10 is an explanatory diagram showing a first color gamut when the first color information is a B/G ratio and a G/R ratio. FIG. 11 is an explanatory diagram showing the distribution of a first synthesis coefficient. FIG. 1 is an explanatory diagram showing a color gamut corresponding to the color of high-density blood. FIG. 2 is an explanatory diagram showing a color gamut corresponding to the color of a mucous membrane onto which a blue pigment has been sprayed. FIG. 4 is a block diagram showing the functions of a color information synthesis unit. 4 is a block diagram showing the functions of a second color change processing unit; FIG. FIG. 2 is an explanatory diagram showing a second color gamut. 13 is a graph showing the relationship between a second distance R2in and a second distance R2out. 13 is a graph showing the relationship between a second angle θ2in and a second angle θ2out. 3 is an explanatory diagram showing functions of a brightness information acquisition unit, a brightness information synthesis unit, and a color image conversion unit. FIG. 13 is a flowchart showing a sequence of operations in a special light observation mode. 11 is a block diagram showing functions of a shading information emphasis processing unit when the first color information is color difference signals Cr and Cb. FIG. 11A and 11B are explanatory diagrams showing before and after processing in a first color change processing unit when the first color information is color difference signals Cr and Cb; 1 is an explanatory diagram showing a first color gamut when the first color information is color difference signals Cr and Cb; 13 is a block diagram showing functions of a shading information emphasis processing unit when the first color information is hue H and saturation S. FIG. 13 is an explanatory diagram showing before and after processing in a first color change processing unit when the first color information has hue H and saturation S; FIG. 10 is an explanatory diagram showing a first color gamut when the first color information has hue H and saturation S; FIG. FIG. 11 is a block diagram showing the functions of an endoscope system according to a second embodiment. 1 is a graph showing an emission spectrum of normal light. 13 is a graph showing an emission spectrum of special light. FIG. 11 is a block diagram showing the functions of an endoscope system according to a third embodiment. FIG. 2 is a plan view showing a rotary filter. 13 is a graph showing a distribution of a second synthesis coefficient.

[First embodiment]
1 , an endoscopic system 10 of the first embodiment includes an endoscope 12, a light source device 13, a processor device 14, a display 15, and a user interface 16. The endoscope 12 is optically connected to the light source device 13 and electrically connected to the processor device 14. The light source device 13 supplies illumination light to the endoscope 12.

The endoscope 12 is used to illuminate the observation target with illumination light and capture the observation target to obtain an endoscopic image. The endoscope 12 has an insertion section 12a that is inserted into the body of the observation target, an operation section 12b provided at the base end of the insertion section 12a, and a bending section 12c and a tip section 12d provided at the tip side of the insertion section 12a. The bending section 12c is bent by operating the operation section 12b. The tip section 12d irradiates the observation target with illumination light and receives reflected light from the observation target to capture an image of the observation target. The tip section 12d is directed in a desired direction by the bending operation of the bending section 12c. The operation section 12b is provided with a mode changeover switch 12f used for mode switching operation, a still image acquisition instruction switch 12g used for instructing acquisition of a still image of the observation target, and a zoom operation section 12h used for operating the zoom lens 21b.

The processor device 14 is electrically connected to the display 15 and the user interface 16. The processor device 14 receives endoscopic images from the endoscope 12. The display 15 outputs and displays images or information of the observation target processed by the processor device 14. The user interface 16 has a keyboard, mouse, touchpad, microphone, etc., and has the function of accepting input operations such as function settings. The extended processor device 17 is electrically connected to the processor device 14.

The endoscope system 10 has a normal observation mode and a special light observation mode, which can be switched by the mode switch 12f. The normal observation mode is a mode in which a normal image obtained by illuminating the observation target with normal light such as white light is displayed on the display 15. The special light observation mode is a mode in which a shade-enhanced image in which the shades of the bright and dark areas are emphasized is displayed on the display 15, which is obtained by illuminating the observation target with special light including narrow band blue light. In this embodiment, the shade-enhanced image is generated based on a special image signal described below obtained by illuminating the observation target with special light. However, the shade-enhanced image may be generated based on an image signal obtained by illuminating the observation target with illumination light having various wavelength ranges other than special light, such as normal light, instead of special light.

As shown in FIG. 2, the light source device 13 includes a light source section 20 having a V-LED (Violet Light Emitting Diode) 20a, a B-LED (Blue Light Emitting Diode) 20b, a G-LED (Green Light Emitting Diode) 20c, and an R-LED (Red Light Emitting Diode) 20d, a light source control section 21 that controls the driving of the four color LEDs 20a to 20d, and an optical path combining section 23 that combines the optical paths of the four color lights emitted from the four color LEDs 20a to 20d. The light combined by the optical path combining section 23 is irradiated into the subject via a light guide 24 and an illumination lens 33 inserted into the insertion section 12a. Note that an LD (Laser Diode) may be used instead of an LED.

The light guide 24 is built into the endoscope 12 and the universal cord (cord connecting the endoscope 12 with the light source device 13 and the processor device 14), and propagates the light combined at the optical path combining section 23 to the tip 12d of the endoscope 12. Note that a multimode fiber can be used as the light guide 41. As an example, a thin-diameter fiber cable with a core diameter of 105 μm, a cladding diameter of 125 μm, and a diameter of φ0.3 to 0.5 mm including the protective layer that serves as the outer shell can be used.

The tip 12d of the endoscope 12 is provided with an illumination optical system 30 and an imaging optical system 31. The illumination optical system 30 has an illumination lens 33, and light from the light guide 24 is irradiated onto the observation object via the illumination lens 33. The imaging optical system 31 has an objective lens 35 and an imaging sensor 36. Reflected light from the observation object is incident on the imaging sensor 36 via the objective lens 35. As a result, a reflected image of the observation object is formed on the imaging sensor 36.

The imaging sensor 36 is a color imaging sensor that captures a reflected image of the subject and outputs an image signal. The imaging sensor 36 is preferably a CCD (Charge Coupled Device) imaging sensor or a CMOS (Complementary Metal-Oxide Semiconductor) imaging sensor. The imaging sensor 36 used in the present invention is a color imaging sensor for obtaining RGB image signals of three colors, R (red), G (green), and B (blue), that is, a so-called RGB imaging sensor that has an R pixel with an R filter, a G pixel with a G filter, and a B pixel with a B filter. The R pixel outputs an R image signal, the G pixel outputs a G image signal, and the B pixel outputs a B image signal.

In the normal observation mode, the image sensor 36 captures an image of an object illuminated by normal light, and outputs a normal image signal. In the special light observation mode, the image sensor 36 captures an image of an object illuminated by special light, and outputs a special image signal.

In addition, the imaging sensor 36 may be a so-called complementary color imaging sensor equipped with complementary color filters of C (cyan), M (magenta), Y (yellow) and G (green) instead of an RGB color imaging sensor. When a complementary color imaging sensor is used, image signals of four colors CMYG are output, and therefore it is necessary to convert the four color image signals of CMYG into three color image signals of RGB by complementary color-primary color conversion. The imaging sensor 36 may also be a monochrome imaging sensor without color filters. In this case, the light source control unit 21 must turn on the blue light B, green light G and red light R in a time-division manner, and add synchronization processing to the imaging signal processing.

The image signal output from the imaging sensor 36 is sent to the CDS/AGC circuit 40. The CDS/AGC circuit 40 performs correlated double sampling (CDS) and automatic gain control (AGC) on the analog image signal. The image signal that passes through the CDS/AGC circuit 40 is converted into a digital image signal by an A/D (Analog/Digital) converter 42. The A/D converted digital image signal is input to the processor device 14.

The processor unit 14 includes an image signal input unit 45, a DSP (Digital Signal Processor) 46, a noise removal unit 47, a signal switching unit 48, a normal image processing unit 49, a shading information emphasis processing unit 50, and a video signal generation unit 51.

A digital image signal from the endoscope 12 is input to the image signal input unit 45. The DSP 46 performs various signal processing such as defect correction processing, offset processing, gain processing, matrix processing, gamma conversion processing, and demosaic processing on the received image signal. In defect correction processing, the signal of a defective pixel of the imaging sensor 36 is corrected. In offset processing, dark current components are removed from the image signal that has been subjected to defect correction processing, and an accurate zero level is set. In gain processing, the signal level is adjusted by multiplying the image signal after offset processing by a specific gain.

Gain processing differs between normal observation mode and special light observation mode. In gain processing in normal observation mode, the R image signal, G image signal, and B image signal of the normal image signal are multiplied by the R gain coefficient, G gain coefficient, and B gain coefficient for normal observation, respectively. In gain processing in special light observation mode, the R image signal, G image signal, and B image signal of the special image signal are multiplied by the R gain coefficient, G gain coefficient, and B gain coefficient for special observation, respectively.

The image signal after gain processing is subjected to matrix processing to improve color reproducibility. The matrix processing also differs between normal observation mode and special light observation mode. For matrix processing in normal observation mode, matrix processing for normal observation is applied to the normal image signal. For matrix processing in special light observation mode, matrix processing for special observation is applied to the special image signal.

Then, brightness and saturation are adjusted by gamma conversion processing. After matrix processing, the image signal is subjected to demosaic processing (also called isotropy processing or synchronization processing), and missing color signals for each pixel are generated by interpolation. This demosaic processing ensures that all pixels have signals for each of the RGB colors.

The noise removal unit 47 removes noise from the image signal by performing noise removal processing (e.g., moving average method, median filter method, etc.) on the image signal that has been subjected to gamma correction and other processes by the DSP 46. The image signal from which noise has been removed is transmitted to the signal switching unit 48.

When the mode switching SW13a is set to the normal observation mode, the signal switching unit 48 transmits a normal image signal to the normal image processing unit 49, and when the mode switching SW13a is set to the special light observation mode, the signal switching unit 48 transmits a special image signal to the shading information emphasis processing unit 50.

The normal image processing unit 49 performs image processing for normal observation on the normal image signal. Image processing for normal observation includes structure emphasis processing for normal observation. The normal image signal that has been subjected to image processing for normal observation is input from the normal image processing unit 49 to the video signal generation unit 51 as a normal image. The shading information emphasis processing unit 50 generates a shading-emphasized image based on the special image signal. Details of the shading information emphasis processing unit 50 will be described later. The shading-emphasized image is input to the video signal generation unit 51.

The video signal generating unit 51 converts the normal image or the shade-emphasized image input from the normal image processing unit 49 or the shade-emphasized image processing unit 50 into a video signal for displaying the image as an image that can be displayed on the display 15. Based on this video signal, the display 15 displays the normal image or the shade-emphasized image.

As shown in FIG. 3, the light source unit 20, which is comprised of the V-LED 20a, B-LED 20b, G-LED 20c, and R-LED 20d provided in the light source device 13, emits light with the emission spectrum shown below. The V-LED 20a emits purple light V with a central wavelength of 405±10 nm and a wavelength range of 380 to 420 nm. The B-LED 20b emits blue light B with a central wavelength of 460±10 nm and a wavelength range of 420 to 500 nm. The G-LED 20c emits green light G with a wavelength range of 480 to 600 nm. The R-LED 20d emits red light R with a central wavelength of 620 to 630 nm and a wavelength range of 600 to 650 nm.

In the normal observation mode, the light source control unit 21 turns on the V-LED 20a, B-LED 20b, G-LED 20c, and R-LED 20d. Thus, a mixture of the four colors of light, violet light V, blue light B, green light G, and red light R, is irradiated as normal light onto the observation subject. In addition, in the normal observation mode, the light source control unit 21 controls each of the LEDs 20a to 20d so that the light intensity ratio between the violet light V, blue light B, green light G, and red light R is Vc:Bc:Gc:Rc. On the other hand, in the special light observation mode, the light source control unit 21 controls each of the LEDs 20a to 20d so that the light intensity ratio between the violet light V, blue light B, green light G, and red light R is Vs:Bs:Gs:Rs.

In this specification, the light intensity ratio includes the case where the ratio of at least one semiconductor light source is 0 (zero). Therefore, it includes the case where one or more of the semiconductor light sources are not lit. For example, a light intensity ratio is also considered to exist even when only one of the semiconductor light sources is lit and the other three are not lit, such as when the light intensity ratio between the purple light V, blue light B, green light G, and red light R is 1:0:0:0.

As shown in FIG. 4, the shading information emphasis processing unit 50 includes an image acquisition unit 60, a color and brightness information acquisition unit 61, a first color change processing unit 62, a synthesis coefficient setting unit 63, a color information synthesis unit 65, a second color change processing unit 66, a brightness information acquisition unit 67, a brightness information synthesis unit 68, and a color image conversion unit 69.

In the processor device 14, programs related to various processes of the shading information emphasis processing unit are incorporated in the program memory. The programs are executed by a control unit (not shown) configured by a processor to realize the functions of an image acquisition unit 60, a color and brightness information acquisition unit 61, a first color change processing unit 62, a synthesis coefficient setting unit 63, a color information synthesis unit 65, a second color change processing unit 66, a brightness information acquisition unit 67, a brightness information synthesis unit 68, and a color image conversion unit 69.

The image acquisition unit 60 acquires the R image signal, G image signal, and B image signal of the special image signals as the first color medical image. Of the special image signals acquired by the image acquisition unit 60, the R image signal is regarded as the first R image signal, the G image signal is regarded as the first G image signal, and the B image signal is regarded as the first B image signal. The image acquisition unit 60 transmits the special image signals to the color and brightness information acquisition unit 61.

The image acquired by the image acquisition unit 60 may be a first color medical image acquired by various medical devices, such as a color ultrasound image acquired by an ultrasound diagnostic device, in addition to the first R image signal, first G image signal, and first B image signal of the special image signals, which are color endoscopic images acquired by the endoscope 12. Therefore, the medical image processing device of the present invention includes an ultrasound processor device that processes ultrasound images, in addition to the processor device 14 for the endoscope shown in this embodiment.

The color and brightness information acquisition unit 61 acquires, for each pixel constituting the special image signal, first brightness information related to brightness, which is one of the three color attributes, from the image signal value, and acquires first color information related to the color information remaining after removing brightness from the three color attributes. The color and brightness information acquisition unit 61 has a signal ratio calculation unit 61a for acquiring the first color information. In the color and brightness information acquisition unit 61, when the signal ratio calculation unit 61a acquires the first color information, it is preferable that the first brightness information is an image signal suitable for expressing the brightness of each pixel in the special image signal. In this embodiment, the first brightness information is the first G image signal of the special image signal, and is hereinafter referred to as G(L1) (see FIG. 5) to distinguish it from other brightness information (second brightness information).

The signal ratio calculation unit 61a calculates the B/G ratio (-log(B/G) without "-log" is referred to as "B/G ratio") by performing differential processing (logG-logB = logG/B = -log(B/G)) based on the first G image signal and the first B image signal that have been log-converted. The signal ratio calculation unit 61a also calculates the G/R ratio by performing differential processing (logR-logG = logR/G = -log(G/R)) based on the first R image signal and the first G image signal that have been log-converted. The G/R ratio, like the B/G ratio, is represented by -log(G/R) without "-log". The B/G ratio and G/R ratio calculated by the signal ratio calculation unit 61a are the first color information. Hereinafter, the B/G ratio and G/R ratio calculated by the signal ratio calculation unit 61a will be referred to as the B/G ratio (X1) and the G/R ratio (Y1) (see FIG. 5) to distinguish them from the other color information (second and third color information).

The B/G ratio and the G/R ratio are calculated for each pixel from the pixel values of pixels at the same position in the B image signal, the G image signal, and the R image signal. The B/G ratio and the G/R ratio are calculated for each pixel. Since the B/G ratio is correlated with the blood vessel depth (the distance from the mucosal surface to the position where a specific blood vessel is located), the B/G ratio also varies when the blood vessel depth varies. Since the G/R ratio is correlated with the blood volume (hemoglobin index), the G/R ratio also varies when the blood volume varies. Instead of the B/G ratio and the G/R ratio, the a ^* b ^* of CIE1976La ^* b ^* obtained from the first color medical image may be used. In this case, the a ^* b ^* of La ^* b ^* corresponds to the first color information, and L corresponds to the first brightness information.

The first color change processing unit 62 refers to the first color information and the first brightness information, and if the first brightness information exceeds a predetermined upper limit value, changes the first color information in a first color direction, and if the first brightness information is less than a predetermined lower limit value, changes the first color information in a second direction that is complementary or opposite to the first color direction, thereby acquiring second color information.

5 and 6, for pixel Pa whose pixel value of the G image signal G(L1) exceeds the upper limit, the first color change processing unit 62 changes the value Sa representing the B/G ratio and G/R ratio corresponding to pixel Pa in the first color direction D1. The value representing the B/G ratio and G/R ratio after the change in the first color direction D1 is defined as Sa ^* . Note that the change in the first color direction D1 is preferably performed by changing the radius vector and angle in a space obtained by polar coordinate conversion of the B/G ratio and G/R ratio.

The first color change processing unit 62 also refers to the pixel values of the G image signal G (L1), and for pixels Pb whose pixel values indicating brightness are less than the lower limit, changes the value Sb representing the B/G ratio and G/R ratio corresponding to the pixel Pa in the second color direction D2. The values representing the B/G ratio and G/R ratio after the change in the second color direction D2 are set as Sb ^* . The B/G ratio and G/R ratio obtained by the change in the first direction D1 in the first color change processing unit 62 are set as the B/G ratio (X2) and the G/R ratio (Y2). This makes it possible to realize a color change for shading emphasis to emphasize the color change of shading information such as a color gamut corresponding to the color of highly concentrated blood or a color gamut corresponding to the color of a mucous membrane to which a dye has been sprayed.

In order to emphasize the shading area where both the bright and dark colors exist, it is preferable that the color indicated by the first color direction D1 is a complementary or opposite color to the color indicated by the second color direction D2. For example, it is preferable that the color indicated by the first color direction is yellow, and the color indicated by the second color direction is blue. As yellow is perceived as bright, it is preferable to change the brightness of the color itself toward yellow in order to increase the brightness in a bright pixel area such as pixel Pa. In contrast, blue is perceived as dark, so it is preferable to change the brightness toward blue in order to suppress the brightness in a dark pixel area such as pixel Pb. In this way, the change in shading between the bright and dark areas is expressed by taking into account a combination of colors such as hue or saturation, making it easier to recognize the change in shading.

Based on the first color information, the synthesis coefficient setting unit 63 sets a first synthesis coefficient used to synthesize the first color information with the second color information described below, or a second synthesis coefficient used to synthesize the first brightness information with the second brightness information described below. It is preferable that the first synthesis coefficient and the second synthesis coefficient have values determined according to the color information within the first color gamut, in which the first color information is within a predetermined range, and are fixed values outside the first color gamut.

Specifically, as shown in FIG. 7, in the signal ratio space formed by the B/G ratio and the G/R ratio, the first color gamut is defined by the first center XC1, YC1, the first distance R1 from the first center XC1, YC1, or the first angle θ1 with the first center line CL1 through which the first center XC1, YC1 passes.

It is preferable that the first synthesis coefficient f1 and the second synthesis coefficient f2 are greater than 0 within the first color gamut, are varied according to the first distance R1 or the first angle θ1, and are set to 0 outside the first color gamut. For example, as shown in FIG. 8, it is preferable that the first synthesis coefficient f1 is greatest at the first centers XC1, YC1 and is smaller the farther away from the first centers XC1, YC1. On the other hand, as shown in FIG. 29, it is preferable that the second synthesis coefficient f2 is greatest at the first centers XC1, YC1 and is larger the farther away from the first centers XC1, YC1.

When the first color gamut corresponds to the color of high-density blood, such as a bleeding area, it is preferable to define the first color gamut in the first quadrant of the signal ratio space consisting of the B/G ratio and the G/R ratio. In this case, as shown in FIG. 9, it is preferable that the first color gamut 72 is an overlapping area of the distance area 72a defined by the first distance R1 and the angle area 72b defined by the first angle θ1 in the first quadrant of the signal ratio space. In addition, when the first color gamut corresponds to the color of mucous membranes on which a blue pigment such as indigo carmine is sprayed, it is preferable to define the first color gamut in the third quadrant of the signal ratio space. In this case, as shown in FIG. 10, it is preferable that the first color gamut 73 is an overlapping area of the distance area 73a defined by the first distance R1 and the angle area 73b defined by the first angle θ1 in the third quadrant of the signal ratio space.

The color information synthesis unit 65 obtains the synthetic color information by synthesizing the first color information and the second color information according to the first synthesis coefficient f1. Specifically, as shown in FIG. 11, the synthetic color information is obtained by adding the values of the B/G ratio (X1) and the G/R ratio (Y1) for each pixel to the f1×B/G ratio (X2) and the f1×G/R ratio (Y2) obtained by multiplying the values of the B/G ratio (X2) and the G/R ratio (Y2) for each pixel by the first synthesis coefficient f1. That is, the parts of the B/G ratio (X2) and the G/R ratio (Y2) multiplied by the intra-area coefficient larger than 0 are added to the B/G ratio (X1) and the G/R ratio (Y1). The B/G ratio and the G/R ratio obtained by the color information synthesis unit 65 are called the B/G ratio (MX) and the G/R ratio (MY).

The second color change processing unit 66 obtains the third color information by changing the second distance or second angle determined according to the color information within the second color gamut in which the second color information falls within a predetermined range. As shown in FIG. 12, the second color change processing unit 66 obtains the B/G ratio (X3) and the G/R ratio (Y3) by changing the second distance R2 or the second angle θ2 in the signal ratio space formed by B/G (MX) and G/R (MY).

The second color gamut determined by the second distance R2 or the second angle θ2 is a region for emphasizing the color difference of a predetermined color emphasis target. The color emphasis target preferably includes, for example, regions where a color difference from normal areas occurs due to a pathology such as atrophic gastritis (atrophic mucosa, deep blood vessels, redness, etc.). In the second color change processing unit 66, the color change correlated with the difference in shading is further emphasized by nonlinearly expanding or contracting the change in hue or saturation, making it possible to make the shading information easier to understand.

As shown in FIG. 13, in the signal ratio space, the second color gamut is defined, like the first color gamut, by the second distance R2 from the second center XC2, YC2 or the second angle θ2 from the second center line CL2 passing through the second center XC2, XC2. In order to change the second distance R2 or the second angle θ2, the second color change processing unit 66 needs to convert the B/G ratio (MX) and the G/R ratio (MY) into polar coordinates.

The second distance change process that changes the second distance R2 is preferably performed as follows. As shown in FIG. 14, the second distance R2out is output in response to the input of the second distance R2in of the coordinates included in the second color gamut. The smaller the second distance R2, the lower the saturation, and the larger the second distance R2, the higher the saturation. The second distance change process uses an S-shaped conversion curve TR that represents the relationship between input and output.

In the second distance change process, in the low saturation range LR in the second color gamut where the second distance R2 is small, the output value R2out is made smaller than the input value R2in, while in the high saturation range HR, the output value R2out is made larger than the input value R2in. This makes it possible to lower the saturation of the object of observation included in the low saturation range, while making it higher the saturation of the object of observation included in the high saturation range. This saturation emphasis makes it possible to increase the saturation difference between multiple observation object ranges. Note that outside the second color gamut, the output value R2out is the same as the input value R2in.

The second angle change process that changes the second angle θ2 is preferably performed as follows. As shown in FIG. 15, a second angle θ2out is output in response to an input of a second angle θ2in of coordinates included in the second gamut. The further the second angle θ2 is from 0, the greater the difference in hue between the second center XC2 and XC2. The second angle change process uses an S-shaped conversion curve TA that represents the input/output relationship.

In the second angle change process, in the second color gamut, in the hue range RA indicated by the second angle θ2 on the negative side, the output value θ2out is made smaller than the input value θ2in, while in the hue range RB indicated by the second angle θ2 on the positive side, the output value θ2out is made larger than the input value θ2in. This makes it possible to increase the difference in hue between the object of observation included in the hue range RA on the negative side and the object of observation included in the hue range RB on the positive side. Note that outside the second color gamut, the output value θ2out is the same as the input value θ2in.

As shown in FIG. 16, the brightness information acquisition unit 67 acquires the second brightness information by converting the first brightness information according to a brightness conversion coefficient that is set based on the first color information. The brightness conversion coefficient Tc is preferably the same as the first synthesis coefficient f1. Specifically, the brightness conversion coefficient Tc is multiplied by the pixel value of each pixel of the G image signal G(L1) to calculate the G image signal G(L2), which is the second brightness information (G(L2)=Tc×G(L1)).

The brightness information synthesis unit 68 synthesizes the first brightness information and the second brightness information according to a second synthesis coefficient that is set based on the first color information to obtain synthetic brightness information. Specifically, the G image signal G(LM) is calculated as synthetic brightness information by adding the pixel value of each pixel of the G image signal (L1), which is the first brightness information, to a value obtained by multiplying the pixel value of each pixel of the G image signal G(L2) by the second synthesis coefficient f2 (G(L1) + f2 × G(L2).

The color image conversion unit 69 converts the third color information and the composite brightness information into a color second medical image. Specifically, the B/G ratio (X3) is converted into a second B image signal by performing an operation based on the G image signal G (LM) and the B/G ratio (X3). The G/R ratio (Y3) is converted into a second R image signal by performing an operation based on the G image signal G (LM) and the G/R ratio (Y3). The G image signal G (LM) is output as a second G image signal without any special conversion. The second B image signal, the second G image signal, and the third image signal become the second color medical image. The color image conversion unit 69 outputs the second color medical image as a shade-enhanced image.

Next, the sequence of operations in the special light observation mode will be described with reference to the flowchart in Figure 17. The mode change switch 12f is operated to switch to the special light observation mode. This illuminates the observation object with special light. The observation object is imaged by the imaging sensor 36, and a special image signal is obtained. The image acquisition unit 60 acquires the first R image signal, the first G image signal, and the first B image signal of the special image signal as a first color medical image.

The color and brightness information acquisition unit 61 acquires the B/G ratio (X1) and G/R ratio (Y1) as the first color information from the first R image signal, the first G image signal, and the first B image signal of the special image signal. The color and brightness information acquisition unit 61 also acquires the first G image signal G (L1) as the first brightness information.

The first color change processing unit 62 refers to the B/G ratio (X1), G/R ratio (Y1) and first G image signal G (L1), and if the first G image signal G (L1) exceeds the upper limit, it changes the values representing the B/G ratio (X1) and G/R ratio (Y1) in the first color direction (yellow direction), and if the first G image signal G (L1) is below the lower limit, it changes the values representing the B/G ratio (X1) and G/R ratio (Y1) in the second color direction (blue direction). This obtains the B/G ratio (X2) and G/R ratio (Y2) as second color information.

The color information synthesis unit 65 synthesizes the B/G ratio (X1), G/R ratio (Y1) and the B/G ratio (X2), G/R ratio (Y2) according to a preset first synthesis coefficient f1 to obtain the B/G ratio (MX) and G/R ratio (MY) as synthesized color information. The color image conversion unit 69 converts the B/G ratio (MX) and G/R ratio (MY) into a second color medical image. The second color medical image is displayed on the display 15 as a gray-enhanced image. The above series of processes is repeated while the special light observation mode continues.

In the above embodiment, the B/G ratio (X1), the G/R ratio (Y1), and the first G image signal G (L1) calculated by the signal ratio calculation unit 61a are used to obtain a shade-enhanced image, but other first color information and first brightness information may be used to obtain a shade-enhanced image.

For example, from a first color medical image, color difference signals Cr and Cb may be obtained as the first color information, and a luminance signal Y may be obtained as the first brightness information. As shown in FIG. 18, the color difference signals Cr and Cb and the luminance signal Y are obtained by the luminance color difference signal converter 61b. Other than obtaining the color difference signals Cr, Cb and the luminance signal Y by the luminance color difference signal converter 61b, this is the same as the above embodiment using the B/G ratio and G/R ratio. Therefore, the color difference signals Cb and Cr correspond to the B/G ratio and G/R ratio, and the luminance signal Y corresponds to the first G image signal.

In addition, when using the color difference signals Cr, Cb and the luminance signal Y, as shown in Fig. 19, the first color change processing unit 62 refers to the pixel value of the luminance signal Y, and for a pixel Pa whose pixel value indicating brightness exceeds an upper limit value, changes the value Sa representing the color difference signals Cr, Cb corresponding to the pixel Pa in the first color direction D1. The value representing the color difference signals Cr, Cb after being changed in the first color direction D1 is Sa ^* . Note that the change in the first color direction is preferably performed by changing the radius vector and angle in a space obtained by polar coordinate conversion of the color difference signals Cr, Cb.

The first color change processing unit 62 also refers to the pixel values of the luminance signal Y, and for pixels Pb whose pixel values indicating brightness are less than the lower limit, changes the value Sb representing the color difference signals Cr and Cb corresponding to the pixels Pb in the second color direction D2. The value representing the color difference signals Cr and Cb after being changed in the second color direction D2 is denoted as Sb ^* .

When the color gamut corresponding to the high-density blood color is defined as the first color gamut, it is preferable to define the first color gamut 72 in the second quadrant of the Cr, Cb space. As shown in FIG. 20, it is preferable that the first color gamut 72 is defined as an overlapping region of a distance region 72a defined by the first distance R1 and an angle region 72b defined by the first angle θ1 in the second quadrant of the Cr, Cb space.

Also, from the first color medical image, hue H and saturation S may be obtained as the first color information, and value V may be obtained as the first brightness information. As shown in FIG. 21, the hue H, saturation S, and value V are obtained by the HSV conversion unit 61c. Other than obtaining the hue H, saturation S, and value V by the HSV conversion unit 61c, this is the same as the above embodiment using the B/G ratio and G/R ratio. Therefore, the hue H and saturation S correspond to the B/G ratio and G/R ratio, and the value V corresponds to the first G image signal.

In addition, when using hue H, saturation S, and lightness V, the first color change processing unit 62 refers to the pixel value of lightness V, and for pixel Pa whose pixel value indicating brightness exceeds the upper limit, changes the value Sa representing the hue H and saturation S corresponding to pixel Pa in the first color direction D1, as shown in Fig. 22. The value representing the hue H and saturation S after the change in the first color direction D1 is set to Sa ^* .

Furthermore, the first color change processing unit 62 refers to the pixel value of the luminosity V, and for pixels Pb whose pixel value indicating the luminosity is less than the lower limit, changes the value Sb representing the hue H and saturation S corresponding to the pixel Pb in the second color direction D2. The value representing the hue H and saturation S after the change in the second color direction D2 is defined as Sb ^* .

When the color gamut corresponding to the color of concentrated blood is defined as the first color gamut, it is preferable to define the first color gamut 72 in the second quadrant of the HS space. As shown in FIG. 23, it is preferable that the first color gamut 72 is an area in the HS space that has the hue range and saturation range of the color gamut corresponding to the color of concentrated blood.

[Second embodiment]
In the second embodiment, a laser light source and a phosphor are used to illuminate the observation target instead of the four color LEDs 20a to 20d shown in the first embodiment.

As shown in FIG. 24, in the endoscope system 100 of the second embodiment, instead of the four color LEDs 20a to 20d, the light source device 13 is provided with a blue laser light source (indicated as "445LD" in FIG. 24) 104 that emits blue laser light with a central wavelength of 445±10 nm, and a blue-violet laser light source (indicated as "405LD" in FIG. 24) 106 that emits blue-violet laser light with a central wavelength of 405±10 nm. The light emitted from the semiconductor light emitting elements of each of these light sources 104 and 106 is individually controlled by a light source control unit 108, and the light intensity ratio between the light emitted from the blue laser light source 104 and the light emitted from the blue-violet laser light source 106 can be freely changed.

In the normal observation mode, the light source control unit 108 drives the blue laser light source 104. In contrast, in the special light observation mode, it drives both the blue laser light source 104 and the blue-violet laser light source 106. The laser light emitted from each of the light sources 104 and 106 enters the light guide 24 via optical components such as a focusing lens, optical fiber, or multiplexer (none of which are shown).

It is preferable that the half-width of the blue laser light or the blue-violet laser light is about ±10 nm. The blue laser light source 104 and the blue-violet laser light source 106 can be broad-area InGaN-based laser diodes, and can also be InGaNAs-based laser diodes or GaNAs-based laser diodes. The light sources may also be configured to use light emitters such as light-emitting diodes.

In addition to the illumination lens 33, the illumination optical system 30 is provided with a phosphor 110 into which blue or blue-violet laser light from the light guide 24 is incident. When the blue laser light is irradiated onto the phosphor 110, fluorescence is emitted from the phosphor 110. In addition, a portion of the blue laser light passes through the phosphor 110 as is. The blue-violet laser light passes through the phosphor 110 without exciting it. The light emitted from the phosphor 110 is irradiated onto the observation object via the illumination lens 33.

Here, in the normal observation mode, mainly blue laser light is incident on the phosphor 110, so that normal light, which is a combination of blue laser light and fluorescence excited and emitted from the phosphor 110 by the blue laser light, as shown in FIG. 25, is irradiated onto the observation object. On the other hand, in the special light observation mode, both blue-violet laser light and blue laser light are incident on the phosphor 110, so that special light, which is a combination of excitation light, which is blue-violet laser light, blue laser light, and fluorescence excited and emitted from the phosphor 110 by the blue laser light, as shown in FIG. 26, is irradiated onto the observation object.

It is preferable to use phosphor 110 that is configured to include multiple types of phosphors (e.g., YAG-based phosphors or phosphors such as BAM (BaMgAl ₁₀ O ₁₇ )) that absorb a portion of the blue laser light and emit light by excitation in the range of green to yellow. If a semiconductor light-emitting element is used as the excitation light source for phosphor 110 as in this configuration example, high-intensity white light can be obtained with high luminous efficiency, the intensity of the white light can be easily adjusted, and changes in the color temperature and chromaticity of the white light can be kept small.

[Third embodiment]
In the third embodiment, instead of the four color LEDs 20a to 20d shown in the first embodiment, a broadband light source such as a xenon lamp and a rotary filter are used to illuminate the observation target. Also, instead of the color image sensor 36, an image of the observation target is captured by a monochrome image sensor. Also, the processor device 14 does not include a signal switching unit 48, and the image signal output from the noise removal unit 47 is input to a normal image processing unit 49 and a grayscale information emphasis processing unit 50 directly connected to the normal image processing unit 49.

In the third embodiment, in the normal observation mode, the normal image processing unit 49 operates, and the shading information emphasis processing unit 50 transmits the normal image output from the normal image processing unit 49 as is to the video signal generating unit 51. On the other hand, in the special light observation mode, the shading information emphasis processing unit 59 operates, and the normal image processing unit 49 transmits the special image signal output from the noise removal unit 47 as is to the shading information emphasis processing unit 50. Other than the above, it is the same as the first embodiment.

As shown in FIG. 27, in the endoscope system 200 of the third embodiment, a broadband light source 202, a rotary filter 204, and a filter switching unit 205 are provided in the light source device 13 instead of the four color LEDs 20a to 20d. Also, the imaging optical system 30b is provided with a monochrome imaging sensor 206 without a color filter instead of the color imaging sensor 36.

The broadband light source 202 is a xenon lamp, a white LED, or the like, and emits white light with a wavelength range from blue to red. The rotating filter 204 has a normal observation mode filter 208 provided on the inside and a special light observation mode filter 209 provided on the outside (see FIG. 28). The filter switching unit 205 moves the rotating filter 204 in the radial direction, and when the normal observation mode is set by the mode switching SW13a, the filter 208 for normal observation mode of the rotating filter 204 is inserted into the optical path of the white light, and when the special light observation mode is set, the filter 209 for special light observation mode of the rotating filter 204 is inserted into the optical path of the white light.

As shown in FIG. 28, the normal observation mode filter 208 is provided with a B filter 208a that transmits blue light from the white light, a G filter 208b that transmits green light from the white light, and an R filter 208c that transmits red light from the white light, arranged in the circumferential direction. Therefore, in the normal observation mode, the rotating filter 204 rotates so that blue light, green light, and red light are alternately irradiated onto the observation object.

The filter 209 for special light observation mode is provided with a Bn filter 209a that transmits blue narrowband light of a specific wavelength from the white light, a G filter 209c that transmits green light from the white light, and an R filter 209d that transmits red light from the white light, arranged in the circumferential direction. Therefore, in the special light observation mode, the rotating filter 204 rotates so that the observation subject is alternately irradiated with blue narrowband light, green light, and red light. The image signal based on the blue narrowband light is used as the B image signal of the special image signal. The image signal based on the green light is used as the G image signal of the special image signal, and the image signal based on the red light is used as the R image signal of the special image signal.

In the above embodiment, the first color change processing unit 62 or the second color change processing unit 66 may use a two-dimensional LUT (Look Up Table) that outputs a corresponding output value (two types of color information such as B/G ratio and G/R ratio) when an input value (two types of color information such as B/G ratio and G/R ratio) is input.

In the above embodiment, the hardware structure of the processing units such as the normal image processing unit 49, the shading information emphasis processing unit 50, the video signal generating unit 51, the image acquiring unit 60, the color and brightness information acquiring unit 61, the first color change processing unit 62, the synthesis coefficient setting unit 63, the color information synthesizing unit 65, the second color change processing unit 66, the brightness information acquiring unit 67, the brightness information synthesizing unit 68, the color image conversion unit 69, the signal ratio calculation unit 61a, the luminance color difference signal conversion unit 61b, the HSV conversion unit 61c, and the light source control unit 108 is various processors as shown below. The various processors include a CPU (Central Processing Unit), which is a general-purpose processor that executes software (programs) and functions as various processing units, a programmable logic device (PLD), which is a processor whose circuit configuration can be changed after manufacturing such as an FPGA (Field Programmable Gate Array), and a dedicated electric circuit, which is a processor having a circuit configuration designed specifically for executing various processes.

A single processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, multiple FPGAs, or a combination of a CPU and an FPGA). Multiple processing units may also be configured with one processor. As an example of configuring multiple processing units with one processor, first, as represented by computers such as clients and servers, there is a form in which one processor is configured with a combination of one or more CPUs and software, and this processor functions as multiple processing units. Second, as represented by system on chip (SoC), there is a form in which a processor is used that realizes the functions of the entire system including multiple processing units with a single IC (Integrated Circuit) chip. In this way, the various processing units are configured using one or more of the above various processors as a hardware structure.

More specifically, the hardware structure of these various processors is an electric circuit (circuitry) that combines circuit elements such as semiconductor elements. The hardware structure of the memory unit is a storage device such as a hard disk drive (HDD) or a solid stage drive (SSD).

10 Endoscope system 12 Endoscope 12a Insertion section 12b Operation section 12c Bending section 12d Tip section 12f Mode changeover switch 12g Still image acquisition instruction switch 12h Zoom operation section 13 Light source device 14 Processor device 15 Display 16 User interface 20 Light source section 20a V-LED
20b B-LED
20c G-LED
20d R-LED
21 Light source control unit 23 Optical path coupling unit 24 Light guide 30 Illumination optical system 31 Imaging optical system 33 Illumination lens 35 Objective lens 36 Imaging sensor 40 CDS/AGC circuit 42 A/D converter 45 Image signal input unit 46 DSP
47 Noise removal unit 48 Signal switching unit 49 Normal image processing unit 50 Grayscale information emphasis processing unit 51 Video signal generation unit 60 Image acquisition unit 61 Color and brightness information acquisition unit 61a Signal ratio calculation unit 61b Luminance color difference signal conversion unit 61c HSV conversion unit 62 First color change processing unit 63 Synthesis coefficient setting unit 65 Color information synthesis unit 66 Second color change processing unit 67 Brightness information acquisition unit 68 Brightness information synthesis unit 69 Color image conversion unit 72, 73 First color gamut 72a, 73a Distance area 72b, 73b Angle area 100 Endoscope system 104 Blue laser light source 106 Blue-violet laser light source 108 Light source control unit 110 Phosphor 200 Endoscope system 202 Broadband light source 204 Rotary filter 205 Filter switching unit 206 Imaging sensor 208 Normal observation mode filter 208a B filter 208b G filter 208c R filter 209 Filter for special light observation mode 209a Bn filter 209b G filter 209c R filter Sa, Sa ^* value Sb, Sb ^* value D1 First color direction D2 Second color direction XC1, YC1 First center R1 First distance θ1 First angle CL1 First center line XC2, YC2 Second center R2 Second distance θ2 Second angle CL2 Second center line TR, TA S-shaped conversion curve LR Low saturation range HR High saturation range RA Negative hue range RB Positive hue range

Claims (3)

A processor is provided.
The processor,
acquiring a first medical image in color;
For each pixel constituting the first color medical image, first brightness information relating to brightness, which is one of three color attributes, is obtained from the image signal value, and first color information relating to color information remaining after removing brightness from the three color attributes is obtained;
With reference to the first color information and the first brightness information, the first color information, of which the first brightness information exceeds a predetermined upper limit value, is changed to a first color direction, and the first color information, of which the first brightness information is less than a predetermined lower limit value, is changed to a second color direction that is a complementary color or an opposite color to the first color direction, thereby acquiring second color information;
obtaining composite color information by combining the first color information with the second color information multiplied by a first combination coefficient set based on the first color information ;
converting the image signal values into a second medical image having a modified color based on the first brightness information and the composite color information ;
the color indicated in the first color direction is yellow, and the color indicated in the second color direction is blue;
the first blending coefficient has a value determined according to the color information within a first color gamut in which the first color information is within a predetermined range, and has a fixed value outside the first color gamut;
the first synthesis coefficient is greater than 0 within the first color gamut, and is changed according to a first distance or a first angle determined according to the color information, and is set to 0 outside the first color gamut;
The first color gamut is a color gamut corresponding to the color of high-density blood . 2. The medical image processing apparatus according to claim 1, wherein the first color information is any one of R/G, which is a ratio of a red signal R of the first color medical image to a green signal G of the first color medical image, and B/G, which is a ratio of a blue signal B of the first color medical image to the green signal G, color difference signals Cr and Cb obtained from the first color medical image, hue and saturation obtained from the first color medical image, and a ^* b ^* of CIE1976 ^La* b ^* obtained from the first color medical image. The processor:
acquiring a first medical image in color;
obtaining, for each pixel constituting the first color medical image, first brightness information relating to brightness, which is one of three color attributes, from an image signal value, and obtaining first color information relating to color information remaining after removing brightness from the three color attributes;
a step of acquiring second color information by referring to the first color information and the first brightness information , changing, in the first color information, the first brightness information of which exceeds a predetermined upper limit value, in a first color direction, and changing, in the first color information, the first brightness information of which is less than a predetermined lower limit value, in a second color direction that is a complementary color or an opposite color to the first color direction;
obtaining composite color information by combining the first color information with the second color information multiplied by a first combination coefficient set based on the first color information ;
converting the image signal values into a modified color second medical image based on the first brightness information and the composite color information ;
the color indicated in the first color direction is yellow, and the color indicated in the second color direction is blue;
the first blending coefficient has a value determined according to the color information within a first color gamut in which the first color information is within a predetermined range, and has a fixed value outside the first color gamut;
the first synthesis coefficient is greater than 0 within the first color gamut, and is changed according to a first distance or a first angle determined according to the color information, and is set to 0 outside the first color gamut;
A method of operating a medical image processing apparatus , wherein the first color gamut is a color gamut corresponding to a thick blood color .

Images