JP6570490B2 - Endoscope system and method for operating endoscope system - Google Patents

Description

  The present invention relates to an endoscope system and a method for operating the endoscope system.

  In the medical field, diagnosis is generally performed using an endoscope system including a light source device, an endoscope, and a processor device. The light source device generates illumination light. An endoscope images an observation target using an image sensor. Then, the processor device generates an image to be observed and displays it on the monitor.

  Depending on the model, the image sensor mounted on the endoscope may be a monochrome sensor or a color sensor. The monochrome sensor is not provided with a color filter in the pixel, and receives incident light regardless of the wavelength and images an observation target. For this reason, when using a monochrome sensor, in order to obtain a color image to be observed, for example, the illumination light is sequentially switched to red, green, and blue (hereinafter referred to as RGB) to obtain an image of each color. On the other hand, in the color sensor, each pixel has one of RGB color filters, for example. For this reason, when a color sensor is used, for example, if white light is used as illumination light, RGB images are obtained simultaneously.

  Some image sensors can adjust the length of a period for accumulating charges by photoelectrically converting light from an observation target (hereinafter referred to as an accumulation period). When an image sensor whose length can be adjusted is a color sensor, for example, the storage period can be adjusted for each color of RGB. For example, in the endoscope system of Patent Document 1, the image quality is improved by making the accumulation period of red pixels that receive red light longer than the accumulation period of pixels that receive light of other colors. (Patent Document 1).

JP 2006-086955 A

  In an endoscope system, information carried about an observation object differs for each color of an image obtained by imaging. For example, a blue image (hereinafter referred to as B image) obtained by imaging an observation target with blue pixels, a green image (hereinafter referred to as G image) obtained by imaging the observation target with green pixels, and an observation target with red pixels In an endoscope system that obtains a red image (hereinafter referred to as an R image) obtained and generates a color image for display using these images, the fine structure of the mucosal surface such as a pit pattern and the submucosa An image of a thin blood vessel at a relatively shallow position appears mainly in the B image. In the R image, almost no pit pattern, thin blood vessels, and the like are shown, and general unevenness of the observation object appears mainly. The G image carries intermediate information between the B image and the R image. That is, an image obtained by imaging an observation target with short-wavelength light carries more information such as a fine structure at a shallow position.

  In diagnosis, a pit pattern and a thin blood vessel are particularly important judgment materials. Therefore, an image showing a pit pattern such as a B image and a thin blood vessel is preferably an image with little distortion. Conversely, an image in which a pit pattern such as an R image and a thin blood vessel are not reflected in the first place can allow a certain degree of distortion. Instead, a bright image is required to clearly observe the observation target.

  By the way, in recent years, an endoscope system which employs a CMOS (Complementary Metal Oxide Semiconductor) sensor as an image sensor is increasing for performance and other reasons. In general, a CMOS sensor performs readout by a rolling shutter system that sequentially reads out signals of pixels arranged vertically and horizontally for each row. In the rolling shutter method readout, there is a time difference in the readout timing for each pixel row, and thus distortion due to the readout time difference (hereinafter referred to as rolling shutter distortion) is likely to occur in the obtained image. For this reason, when reading is performed by a rolling shutter method using a CMOS sensor, it is particularly required to reduce rolling shutter distortion of an image in which a pit pattern such as a B image or a thin blood vessel is captured, and an R image A device to ensure brightness is required.

  An object of the present invention is to provide an endoscope system and an operation method of the endoscope system that can reduce the influence of rolling shutter distortion on an image particularly important in diagnosis, and as a result, can obtain a clear image.

In the endoscope system of the present invention, the first color light and the first color light can be independently controlled to turn on and off the second color light having different wavelength bands or spectral spectra, and the first color light or the second color light can be controlled. A light source unit that generates illumination light including any one of the following: a first color pixel that receives the first color light, and a second color pixel that receives the second color light, and at least the first color pixel or the second color pixel. An image sensor that picks up an observation target using color pixels, charges are accumulated in at least the first color pixel and the second color pixel during the accumulation period, and the first color pixel and the second color are read during the readout period following the accumulation period. An imaging control unit that reads out a signal from a color pixel by a rolling shutter method, lights up the first color light and the second color light during the accumulation period, maintains the first color light during the readout period, and emits the second color light during the readout period. Light source control to turn off Comprising a part, the second color light is red light, and the first color light is the light in the short wavelength band than the red light.

  The image sensor includes a first pixel group that is an aggregate of pixel rows including the first color pixels, and a second pixel group that is an aggregate of pixel rows that do not include the first color pixels. It is preferable that the control unit reads a signal for each pixel group of the first pixel group and the second pixel group.

  The image sensor includes a first pixel group that is an aggregate of pixel rows that include the first color pixels, and a second pixel group that is an aggregate of pixel rows that do not include the first color pixels. It is preferable to generate an image for each color included in the one pixel group or the second pixel group.

  The light source control unit preferably makes the light amount of the first color light in the readout period equal to the light amount of the first color light in the accumulation period.

  The light source control unit preferably makes the light amount of the second color light in the reading period smaller than at least the light amount of the second color light in the accumulation period.

  The light source controller preferably changes the wavelength band or spectral spectrum of the illumination light for each accumulation period.

  It is preferable to include an image generation unit that generates one image using a plurality of signals acquired in a plurality of readout periods or a plurality of images generated using signals acquired in a plurality of readout periods.

  It is preferable to include a staticity calculation unit that calculates the staticity of the observation target using an image generated using the signal obtained from the first color pixel or the signal obtained from the first color pixel.

The operation method of the endoscope system according to the present invention is capable of independently controlling the turning on and off of the second color light in which the first color light and the first color light have different wavelength bands or spectral spectra, and the first color light or A light source unit that generates illumination light including any one of the second color lights, a first color pixel that receives the first color light, and a second color pixel that receives the second color light, and at least the first color pixel Or an image sensor that captures an image of an observation object using a second color pixel , the second color light is red light, and the first color light is light in a shorter wavelength band than the red light. In the operation method, the imaging control unit accumulates charges in at least the first color pixel and the second color pixel during the accumulation period, and the imaging control unit performs the first color pixel and the first color during the readout period following the accumulation period. Rolling shutter method from 2 color pixels The light source control unit turns on the first color light and the second color light during the accumulation period, and the light source control unit maintains the first color light lighting during the readout period and the second light source during the readout period. Turning off the colored light.

  According to the endoscope system and the operation method of the endoscope system of the present invention, the first color light and the second color light are turned on in the accumulation period, the first color light is kept on in the reading period, and the second color light is turned on in the reading period. Turn off the colored light. Therefore, according to the endoscope system and the operation method of the endoscope system of the present invention, when reading is performed by the rolling shutter method using a CMOS sensor as an image sensor, the influence of rolling shutter distortion on an image particularly important in diagnosis is affected. Reduced and clear images can be obtained.

It is an external view of an endoscope system. It is a block diagram of an endoscope system. It is a block diagram which shows the structure of the light source part of 1st Embodiment. It is a graph which shows the wavelength band and spectral spectrum of the illumination light of 1st Embodiment. It is a graph which shows the spectral transmission characteristic of a color filter. It is explanatory drawing which shows the pixel arrangement | sequence of an image sensor. It is a flowchart which shows the flow of operation | movement of an endoscope system. It is explanatory drawing which shows the effect | action of an image process part. It is a timing chart which shows the relation between operation of an image sensor, and illumination light. It is a block diagram which shows the structure of the light source part of 2nd Embodiment. It is a graph which shows the wavelength band and spectral spectrum of the illumination light of 2nd Embodiment. It is a timing chart which shows the relation between operation of an image sensor, and illumination light. It is explanatory drawing which shows the effect | action of the image process part of 2nd Embodiment. It is a block diagram which shows the structure of the light source part in the modification of 2nd Embodiment. It is a timing chart in the case of lighting illumination light in a time division manner. It is a timing chart in the case of lighting illumination light in a time division manner. It is a block diagram which shows the structure of the light source part of 3rd Embodiment. It is a graph which shows the light absorption coefficient of oxygenated hemoglobin and reduced hemoglobin. It is a timing chart which shows the relation between operation of an image sensor, and illumination light. It is a block diagram which shows the structure of the image process part of 3rd Embodiment. It is a graph which shows the correlation of a signal ratio and oxygen saturation. It is a block diagram which shows the structure of the light source part of 4th Embodiment. It is a graph which shows the wavelength band and spectral spectrum of the illumination light of 4th Embodiment. It is a timing chart which shows the relation between operation of an image sensor, and illumination light. It is a block diagram which shows the structure of the image process part of 4th Embodiment. It is a block diagram which shows the structure of the light source part of 5th Embodiment. It is a graph which shows the wavelength band and spectral spectrum of the illumination light of 5th Embodiment. It is a timing chart which shows the relation between operation of an image sensor, and illumination light. It is a block diagram which shows the structure of the image generation part of 5th Embodiment. It is a timing chart which shows the relation between operation of an image sensor, and illumination light. It is explanatory drawing which shows the effect | action of the image generation part in a modification. It is the schematic of a capsule endoscope.

[First Embodiment]
As shown in FIG. 1, an endoscope system 10 includes an endoscope 12 that images an observation target, a light source device 14 that generates illumination light, and an image obtained by imaging the observation target (hereinafter referred to as a captured image). ), A processor device 16 that generates an image for observation (hereinafter referred to as an observation image), a monitor 18 that displays the observation image, and a console 19 that is one of user interfaces. The endoscope 12 is optically connected to the light source device 14 and electrically connected to the processor device 16. The endoscope 12 includes an insertion portion 12a to be inserted into the subject, an operation portion 12b at the proximal end portion of the insertion portion 12a, a bending portion 12c provided on the distal end side of the insertion portion 12a, and a distal end portion 12d. ,have. When the angle knob 12e in the insertion portion 12a is operated, the bending portion 12c is bent. As a result of the bending of the bending portion 12c, the distal end portion 12d faces in a desired direction. Note that the distal end portion 12d is provided with a jet outlet (not shown) that jets air, water, or the like toward the observation target.

  In addition to the angle knob 12e, the operation unit 12b is provided with a mode switch 13a and a zoom operation unit 13b. The mode switch 13a is used for an operation of switching the observation mode when the endoscope system 10 has a plurality of observation modes. In the present embodiment, the endoscope system 10 operates in a normal observation mode that is one of the observation modes. In the normal observation mode, the observation target is imaged using white light as illumination light. Then, using the obtained captured image, an observation image having a natural hue (hereinafter referred to as a normal observation image) is generated and displayed on the monitor 18 serving as a display unit.

  As shown in FIG. 2, the light source device 14 includes a light source unit 20 and a light source control unit 22 that controls the light source unit 20. The light source unit 20 can independently control the turning on and off of the first color light and the second color light having a wavelength band or spectral spectrum different from that of the first color light, and at least either the first color light or the second color light. Illumination light including

  More specifically, as shown in FIG. 3, in the present embodiment, the light source unit 20 includes three light sources: a B broadband light source 24, a G broadband light source 25, and an R broadband light source 26. The B broadband light source 24, the G broadband light source 25, and the R broadband light source 26 are semiconductor light sources such as LEDs (Light Emitting Diodes).

  The B broadband light source 24 is a light source that emits blue broadband light (hereinafter referred to as B broadband light). Blue refers mainly to the color of light having a wavelength of about 400 nm to about 500 nm. The broadband means a wavelength band that is so wide that the endoscope system 10 cannot be regarded as a single wavelength (single wavelength). For example, a wavelength band from several tens of nm to 100 nm or more is a wide band. The B broadband light 27 emitted from the B broadband light source 24 has, for example, a wavelength band and a spectrum shown in FIG.

  The G broadband light source 25 is a light source that emits green broadband light (hereinafter referred to as G broadband light). Green means a color of light mainly having a wavelength of about 500 nm to about 600 nm. The R broadband light source 26 is a light source that emits red broadband light (hereinafter referred to as R broadband light). Red means a color of light mainly having a wavelength of about 600 nm to about 700 nm. The G broadband light 28 emitted from the G broadband light source 25 and the R broadband light 29 emitted from the R broadband light source 26 have, for example, the wavelength band and the spectrum shown in FIG.

  The light source control unit 22 controls lighting, extinction, and light amount of the B broadband light source 24, the G broadband light source 25, and the R broadband light source 26 according to the drive timing of the image sensor 48. The lighting means that the image sensor 48 emits light with an amount of light that allows the observation target to be imaged (that is, an image of the observation target can be visually recognized in the observation image). Turning off the light includes stopping the light emission completely and reducing the light amount to such an extent that the image sensor 48 cannot image the observation target.

  In this embodiment, the R broadband light 29 that is red light is the first color light, and the B broadband light 27 that is blue light having a shorter wavelength band than the wavelength band of the red light is the second color light. . As described above, the R broadband light 29 and the B broadband light 27 have different wavelength bands and spectral spectra. In the present embodiment, the B broadband light 27 is repeatedly turned on and off in accordance with the drive timing of the image sensor 48. On the other hand, the light source unit 20 maintains lighting of the R broadband light 29 when the endoscope 12 images the observation target. Therefore, the light source unit 20 switches the illumination light between the illumination light including both the B broadband light 27 and the R broadband light 29 and the illumination light including only the R broadband light 29 in accordance with the drive timing of the image sensor 48. Generates illumination light including at least either the R broadband light 29 or the B broadband light 27. In this embodiment, the G broadband light 28 can be regarded as the second color light instead of the B broadband light 27. This is because the G broadband light 28 has the same relationship as the B broadband light 27 with respect to the R broadband light 29.

  The illumination light generated by the light source unit 20 enters the light guide 41. The light guide 41 is built in the endoscope 12 and the universal cord, and propagates the illumination light to the distal end portion 12d of the endoscope 12. The universal cord is a cord that connects the endoscope 12 to the light source device 14 and the processor device 16. A multimode fiber can be used as the light guide 41. As an example, a thin fiber cable having a core diameter of 105 μm, a cladding diameter of 125 μm, and a diameter including a protective layer serving as an outer shell of φ0.3 to 0.5 mm can be used.

  An illumination optical system 30a and a photographing optical system 30b are provided at the distal end portion 12d of the endoscope 12. The illumination optical system 30 a has an illumination lens 45, and irradiates an observation target with illumination light through the illumination lens 45. The photographing optical system 30b includes an objective lens 46, a zoom lens 47, and an image sensor 48. The image sensor 48 reflects reflected light or the like of illumination light returning from the observation target via the objective lens 46 and the zoom lens 47 (in addition to the reflected light, scattered light, fluorescence emitted from the observation target, or medicine administered to the observation target) The observation object is photographed using the image including the fluorescence caused by The zoom lens 47 moves by operating the zoom operation unit 13b, and enlarges or reduces the observation target to be photographed using the image sensor 48.

  The image sensor 48 is a CMOS sensor and a so-called primary color sensor. That is, each pixel of the image sensor 48 includes any one of a blue color filter that transmits blue light, a green color filter that transmits green light, and a red color filter that transmits red light. A pixel having a blue color filter is a B pixel (“B”), a pixel having a green color filter is a G pixel (“G”), and a pixel having a red color filter is an R pixel (“R”). . These color filters have the spectral transmission characteristics shown in FIG. 5, for example. For this reason, for example, the image sensor 48 receives the reflected light of the B broadband light 27 by the observation target by the B pixel, receives the reflected light of the G broadband light 28 by the observation target by the G pixel, and R by the observation target. The reflected light of the broadband light 29 is received by the R pixel. In the present embodiment, since the R broadband light 29 is the first color light, the R pixel is the first color pixel that receives the first color light. In the present embodiment, since the B broadband light 27 is the second color light, the B pixel is the second color pixel that receives the second color light. Therefore, the image sensor 48 images the observation object using at least the first color pixel or the second color pixel.

  As shown in FIG. 6, the image sensor 48 has an imaging surface in which pixels of RGB colors are arranged. For example, as shown in FIG. 6, B pixels and G pixels are alternately arranged in odd-numbered pixel rows (a set of pixels arranged in the horizontal direction in FIG. 6), and even-numbered pixel rows G pixels and R pixels are arranged alternately. Further, when viewed in the column direction (vertical direction in FIG. 6), the B pixel and the G pixel are alternately arranged in the pixel column including the B pixel, and the G pixel and the R pixel are included in the pixel column including the R pixel. They are arranged alternately.

  Since the image sensor 48 is a CMOS sensor, it is possible to arbitrarily read and reset signals for each pixel. In this embodiment, the image sensor 48 performs operations such as reading and resetting signals on a pixel row. Do it all together. In addition, the image sensor 48 can perform operations such as signal readout and reset by selecting an arbitrary pixel row. In the present embodiment, the image sensor 48 reads and outputs signals from all the pixel rows in ascending order of row numbers. Reset sequentially. That is, in the present embodiment, the signal reading method of the image sensor 48 is a so-called progressive method.

  Further, the reading method of the image sensor 48 is a so-called rolling shutter method. That is, when each pixel of the image sensor 48 reads out the signal of each pixel and discards (resets) the charge accumulated in each pixel, the pixel can be accumulated by photoelectric conversion. As described above, since the image sensor 48 sequentially reads and resets the signal for each pixel row, the start and end timings of the period during which charge can be accumulated by photoelectric conversion in one imaging operation are shifted for each pixel row. is there.

  The processor device 16 includes a control unit 52, an image acquisition unit 54, an image processing unit 61, and a display control unit 66.

  The control unit 52 is a CPU (Central Processing Unit) or the like that comprehensively controls the endoscope system 10, and includes at least an imaging control unit 53 that controls the operation of the image sensor 48. For example, the control unit 52 performs synchronization control of the illumination light emission timing and the imaging frame. Specifically, as a result of the synchronization control by the control unit 52, the light source control unit 22 lights the first color light and the second color light during the accumulation period, maintains the first color light during the reading period, and reads the reading period. In step 2, the second color light is turned off. “Maintaining lighting” means that the light amount of the first color light in the reading period is substantially equal to the light amount of the first color light in the accumulation period. “Making the light amount substantially equal” means controlling the light amount so that the brightness of captured images to be acquired is equal, including a change in light amount by automatic exposure control. Further, in the present embodiment, the light source control unit 22 turns off the second color light during the reading period by making the light amount of the second color light during the reading period at least smaller than the light amount of the second color light during the accumulation period.

  When the endoscope system 10 has a plurality of observation modes, the control unit 52 receives a mode switching signal from the mode switching switch 13a and receives the light source control unit 22, the imaging control unit 53, and the image processing unit 61. The observation mode is switched by inputting a control signal to, for example.

  The imaging control unit 53 controls the image sensor 48 in an accumulation period and a readout period, and images are alternately repeated at regular intervals (for example, every 1/60 seconds) between the accumulation period and the readout period. The sensor 48 is controlled. In the present embodiment, the total of one accumulation period and the readout period following this one accumulation period is an imaging frame of one unit.

  The accumulation period is a period in which all the pixels of the image sensor 48 receive the reflected light of the illumination light and the like without accumulating charges without reading signals from any pixel row. The readout period is a period in which a signal is read from each pixel of the image sensor 48 following the accumulation period. In the present embodiment, the imaging control unit 53 discards the charge accumulated in the pixel immediately after reading out the signal and resets the pixel. Therefore, the reading period is a period for reading and resetting signals.

  In other words, the imaging control unit 53 accumulates electric charge in each pixel of the image sensor 48 during the accumulation period, and reads out a signal from each pixel of the image sensor 48 by the rolling shutter method during the readout period following the accumulation period. Each pixel of the image sensor 48 mentioned here includes at least a first color pixel (R pixel) or a second color pixel (B pixel).

  The image acquisition unit 54 acquires a captured image from the image sensor 48 when the imaging control unit 53 receives a signal read from the image sensor 48. More specifically, the image acquisition unit 54 uses a B image formed by a signal read from the B pixel, a G image formed by a signal read from the G pixel, and a signal read from the R image for each imaging frame. Three types of captured images of the R image to be formed are acquired.

  The image acquisition unit 54 includes a DSP (Digital Signal Processor) 56, a noise reduction unit 58, and a conversion unit 59, and performs various processes on the acquired captured image using these.

  The DSP 56 performs various processes such as a defect correction process, an offset process, a gain correction process, a linear matrix process, a gamma conversion process, a demosaic process, and a YC conversion process on the acquired captured image as necessary.

  The defect correction process is a process for correcting the pixel value of the pixel corresponding to the defective pixel of the image sensor 48. The offset process is a process for reducing the dark current component from the captured image subjected to the defect correction process and setting an accurate zero level. The gain correction process is a process for adjusting the signal level of each captured image by multiplying the captured image subjected to the offset process by a gain. The linear matrix process is a process for improving the color reproducibility of the captured image subjected to the offset process, and the gamma conversion process is a process for adjusting the brightness and saturation of the captured image after the linear matrix process. The demosaic process (also referred to as an isotropic process or a synchronization process) is a process of interpolating the pixel values of the missing pixels, and is applied to the captured image after the gamma conversion process. The missing pixel is a pixel having no pixel value because pixels of other colors are arranged in the image sensor 48 due to the arrangement of the color filters. For example, since the B image is a captured image obtained by capturing an observation target at the B pixel, the pixel at the position corresponding to the G pixel or the R pixel of the image sensor 48 has no pixel value. In the demosaic process, the B image is interpolated to generate pixel values of the pixels at the positions of the G pixel and the R pixel of the image sensor 48. The YC conversion process is a process for converting the demosaiced image into a luminance channel Y, a color difference channel Cb, and a color difference channel Cr.

  The noise reduction unit 58 performs noise reduction processing on the luminance channel Y, the color difference channel Cb, and the color difference channel Cr using, for example, a moving average method or a median filter method. The conversion unit 59 reconverts the luminance channel Y, the color difference channel Cb, and the color difference channel Cr after the noise reduction processing into a captured image of each color of BGR again.

  The image processing unit 61 performs, for example, a color conversion process, a color enhancement process, a structure enhancement process, and the like on the captured image subjected to the above-described various processes to generate one color observation image. In the normal observation mode, the observation image generated by the image processing unit 61 is a normal observation image. In the color conversion process, a 3 × 3 matrix process, a gradation conversion process, a three-dimensional LUT (look-up table) process, and the like are performed on an image of each BGR color. The color enhancement process is a process for enhancing the color of an image, and the structure enhancement process is a process for enhancing a tissue or structure to be observed such as a blood vessel or a pit pattern. The display control unit 66 sequentially acquires the observation images generated by the image processing unit 61, converts them into a suitable format, and sequentially outputs and displays them on the monitor 18. Thereby, a doctor etc. can observe an observation object using an observation image.

  Next, a flow of a series of operations of the endoscope system 10 will be described along the flowchart shown in FIG. First, when observation is started, the imaging control unit 53 sets the operation of the image sensor 48 to the operation of the accumulation period (S11). At the same time, the light source control unit 22 generates white illumination light by the light source unit 20 (S12). Specifically, the light source control unit 22 turns on all of the B broadband light source 24, the G broadband light source 25, and the R broadband light source 26. Accordingly, the light source unit 20 includes the B broadband light 27, the G broadband light 28, and the R broadband light 29, and generates white illumination light as a whole. For this reason, in the accumulation period (S11), the image sensor 48 photoelectrically converts reflected light of white illumination light or the like in each pixel and accumulates electric charges.

  When a certain time has elapsed after the start of the accumulation period (S11), the imaging control unit 53 switches the operation of the image sensor 48 to the operation of the reading period (S13). At the same time, the light source control unit 22 switches the illumination light (S14). Specifically, the light source control unit 22 turns off the B broadband light 27 and the G broadband light 28 and maintains the R broadband light 29 on during the readout period. For this reason, the light source unit 20 generates the R broadband light 29 as illumination light even during the readout period.

  When the imaging control unit 53 reads a signal for each pixel row from the image sensor 48 in the readout period (S13), the image acquisition unit 54 acquires captured images of RGB colors (S15). Then, as shown in FIG. 8, the image processing unit 61 generates a normal observation image using the three color captured images of the R image, the G image, and the B image subjected to various processes by the image acquisition unit 54, Further, the display control unit 66 displays the normal observation image on the monitor 18 (S16). The endoscope system 10 repeats these operations until the normal observation mode is ended and switched to another observation mode or the observation itself is ended (S17).

  As described above, in the endoscope system 10, when the image sensor 48 that performs readout by the rolling shutter method repeatedly repeats the operation of the accumulation period and the readout period, during the accumulation period, the B broadband light 27 and the G broadband Light 28 and R broadband light 29 are turned on. In the readout period, the B broadband light 27 and the G broadband light 28 are turned off, while the R broadband light 29 is kept on. For this reason, the endoscope system 10 can improve the simultaneity of the B image and the G image, and can reduce the rolling shutter distortion of the B image and the G image. In addition, together with the B image and the G image with reduced rolling shutter distortion, the endoscope system 10 can obtain a brighter R image than when the R broadband light 29 is also turned off during the readout period. Therefore, the observation image generated and displayed by the endoscope system 10 is an image in which an image particularly important in diagnosis is captured, compared to an observation image in a conventional endoscope system using an image sensor that performs reading by a rolling shutter method. Rolling shutter distortion is small and clear.

  More specifically, when the image sensor 48 alternately repeats the accumulation period and the readout period and performs readout by the rolling shutter method in the readout period, the B broadband light 27, the G broadband light 28, and When the R broadband light 29 is turned on and the B broadband light 27 and the G broadband light 28 are extinguished during the readout period, the image sensor 48 in the one imaging frame F1 has the B broadband light 27 and the G broadband light as shown in FIG. The period for receiving the reflected light of the light 28 coincides with the accumulation period. For example, the signal of each pixel that has accumulated charges during the accumulation period from time T1 to time T2 (hereinafter referred to as accumulation period T1-T2; the same applies to other accumulation periods and readout periods) is the accumulation period T1-T2. In the next reading period T2-T3 following the reading. However, since the reflected light of the B broadband light 27 and the G broadband light 28 is not generated in the readout period T2-T3, the amount of charges accumulated in the B pixel and the G pixel in the readout period T2-T3 is the accumulation period T1-T3. The amount accumulated at T2 is not increased or decreased. Therefore, even when signals are sequentially read out for each pixel row by the rolling shutter method in the readout period T2-T3, the rolling shutter distortion hardly occurs in the B image and the G image. Due to its fineness, images of fine blood vessels or the like that appear in the B and G images may be difficult to observe even with slight rolling shutter distortion. However, as described above, by suppressing the rolling shutter distortion of the B image and the G image, if an observation image generated using the B image and the G image is used, an image of a fine blood vessel or the like that is particularly important for diagnosis can be obtained. Clearly observable.

  On the other hand, since the R broadband light 29 is lit not only during the accumulation period but also during the readout period, the period during which the R pixel receives the reflected light or the like of the R broadband light 29 is longer than the accumulation period in the image sensor 48. long. For example, during a period in which the R pixel that reads a signal in the reading period T2-T3 receives the reflected light of the R broadband light 29, the signal is read and reset in the previous reading period T0-T1, and then the reading period T2- The period until the signal is actually read at T3 (hereinafter referred to as a substantial accumulation period) L2. The substantial accumulation period L2 of the R pixel is longer than the length L1 of each accumulation period (L1 <L2). If the length of the accumulation period and the length of the reading period are all equal, the substantial accumulation period L2 is twice the length L1 of each accumulation period. Therefore, the brightness of the R image is easily ensured as compared with the B image and the G image. If this bright R image is used to generate an observation image, the brightness of the observation image is improved. Of course, since the R broadband light 29 is kept on until the readout period, rolling shutter distortion occurs in the R image. However, since an image of a fine blood vessel or the like is not originally captured in the R image compared to the B image or the like, there is no problem in diagnosis even if there is some rolling shutter distortion.

[Second Embodiment]
In the first embodiment, the endoscope system 10 operates in the normal observation mode, and generates and displays a normal observation image having a natural hue. However, the narrow band observation that generates and displays a so-called narrow band observation image. The present invention is also suitable in the mode. The narrow-band light observation mode is an image of an observation target using blue and green narrow-band light, and using the obtained captured image, an observation image (narrow-band observation image) that emphasizes blood vessels is generated and displayed. Observation mode.

  When the narrow band observation mode is provided in the endoscope system 10, as shown in FIG. 10, at least the B narrow band light source 224, the G narrow band light source 225, and the R wide band light source 26 are provided in the light source unit 20. The B narrowband light source 224 and the G narrowband light source 225 may be provided in place of the B broadband light source 24 and the G broadband light source 25 of the first embodiment. However, when the narrowband observation mode is switched to the normal observation mode, In addition to the B broadband light source 24 and the G broadband light source 25 of the first embodiment, a B narrow band light source 224 and a G narrow band light source 225 are provided. Further, the B narrow band light source 224 and the G narrow band light source 225 may be configured by the optical filter or the like and the B broadband light source 24 and the G broadband light source 25 of the first embodiment. The R broadband light source 26 is the same as in the first embodiment.

  The B narrow band light source 224 is a light source that emits blue narrow band light (hereinafter referred to as B narrow band light). The narrow band refers to a wavelength band that is narrow enough to be regarded as a single wavelength in the endoscope system 10. For example, a wavelength band of ± several tens of nm with respect to the center wavelength is a narrow band. The B narrowband light 227 emitted from the B narrowband light source 224 has, for example, a wavelength band and a spectrum shown in FIG. That is, the B narrowband light 227 has a center wavelength of about 450 nm, and has a wavelength band of about ± several tens of nm centering on this center wavelength. The reflected light of the B narrow band light 227 is received by the B pixel (see FIG. 5).

  The G narrow band light source 225 is a light source that emits green narrow band light (hereinafter referred to as G narrow band light). The G narrow band light 228 emitted from the G narrow band light source 225 has, for example, a wavelength band and a spectrum shown in FIG. That is, the G narrow-band light 228 has a center wavelength of about 550 nm, and has a wavelength band of about ± several tens of nm around the center wavelength. The G pixel receives the reflected light of the G narrow band light 228 (see FIG. 5).

  In the narrow band observation mode, as shown in FIG. 12, the light source control unit 22 uses the B narrow band in accordance with the accumulation period of the image sensor 48 instead of the B wide band light 27 and the G wide band light 28 in the first embodiment. The light 227 and the G narrowband light 228 are turned on, and the B narrowband light 227 and the G narrowband light 228 are turned off during the readout period. The R broadband light 29 is the same as in the normal observation mode of the first embodiment. That is, the light source control unit 22 lights the R broadband light 29 during the accumulation period of the image sensor 48 and maintains the lighting of the R broadband light 29 even during the reading period.

  Therefore, in the narrowband observation mode, the observation object is imaged using the B image obtained by imaging the observation object using the reflected light of the B narrowband light 227, the reflected light of the G narrowband light 228, and the like. The obtained G image has almost no rolling shutter distortion. Also, the R image obtained by imaging the observation object using the reflected light of the R broadband light 29 is brighter than before.

  In the narrow band observation mode, as shown in FIG. 13, the image processing unit 61 acquires the B image and the G image from the image acquisition unit 54. Then, a narrowband observation image that is an observation image in the narrowband observation mode is generated and displayed using the B image and the G image. Specifically, for example, the image processing unit 61 generates a narrowband observation image by assigning the B image to the B channel and the G channel and assigning the G image to the R channel.

  As described above, in the narrow band observation mode, the R image is not used to generate the narrow band observation image. Instead, as shown in FIG. 13, in the endoscope system 10 that provides the narrow-band observation mode, it is preferable to provide the processor unit 16 with a quiescence degree calculation unit 251.

  The stillness calculation unit 251 calculates the stillness using one or a plurality of R images. The degree of stillness is a numerical value representing the magnitude of the movement of the observation target, the direction of the movement of the observation target, or the magnitude and direction of the movement of the observation target. The movement of the observation target mentioned here includes the movement of the observation target itself and the movement of the observation target relative to the movement of the endoscope 12. The degree of staticity is calculated by frequency analysis, for example, when calculated using one R image. If the R image has few high-frequency components, it can be evaluated that the movement of the observation object is large, so that the degree of stillness can be calculated based on the amount of the high-frequency components. In addition, when calculating the degree of staticity using a plurality of R images, for example, the degree of staticity can be calculated from the direction and size of a motion vector obtained from the position between corresponding points of the observation target in each R image. . In any case, the staticity calculation unit 251 uses an R image obtained by imaging the observation target using the reflected light of the R broadband light 29 that has been lit until the readout period. Since this R image is brighter than the conventional R image, as a result of using this R image, the degree of stillness can be calculated more accurately than the conventional one.

  The stillness calculation unit 251 inputs the calculated stillness to the display control unit 66, for example, and displays the stillness on the monitor 18 together with the narrowband observation image. A doctor or the like can easily and objectively grasp the magnitude of the movement of the observation object by looking at the degree of stillness.

  In the second embodiment, the B narrow-band light source 224 is used. However, when the endoscope system 10 is provided with a narrow-band observation mode, as shown in FIG. Alternatively, a V narrow band light source 252 can be provided in addition to the B narrow band light source 224. The V narrow band light source 252 is a light source that emits purple narrow band light (hereinafter referred to as V narrow band light). Purple means a color of light mainly having a wavelength of about 350 nm to about 400 nm.

  When the V narrowband light source 252 is provided, the light source control unit 22 uses the V narrowband light together with the B narrowband light 227 of the second embodiment or instead of the B narrowband light 227 in the accumulation period of the image sensor 48. Is turned on and turned off during the readout period. Comparing the B narrowband light 227 and the V narrowband light, the V narrowband light having a shorter wavelength is easier to catch blood vessels and the like closer to the surface of the mucosa to be observed. For this reason, if the V narrowband light is used instead of the B narrowband light 227, the blood vessels to be emphasized in the narrowband observation image are different from the case where the B narrowband light 227 is used. Further, when the V narrowband light is used together with the B narrowband light 227, blood vessels or the like that could not be enhanced when only the B narrowband light 227 is used in the narrowband observation image may be emphasized.

  In the second embodiment, the B narrowband light 227 and the G narrowband light 228 are turned on simultaneously during the accumulation period of the image sensor 48. However, the light amounts of the B narrowband light 227 and the G narrowband light 228 are sufficient. If so, as shown in FIG. 15, the B narrowband light 227 and the G narrowband light 228 can be sequentially turned on within the accumulation period. That is, the B narrowband light 227 and the G narrowband light 228 can be time-divisionally lit. The same applies to the B broadband light 27 and the G broadband light 28 in the first embodiment. Further, as shown in FIG. 16, the B narrowband light 227, the G narrowband light 228, and the R wideband light 29 can be time-divisionally lit. The same applies to the B broadband light 27, the G broadband light 28, and the R broadband light 29 in the first embodiment.

  In the second embodiment, the R broadband light 29 is the first color light, and both or one of the B narrow band light 227 and the G narrow band light 228 is the second color light. When the B narrowband light 227 is the second color light, the B pixel is the second color pixel, and when the G narrowband light 228 is the second color light, the G pixel is the second color pixel. In the case where the B narrowband light 227 and the G narrowband light 228 are the second color light, the image sensor 48 has no pixels that receive both the B narrowband light 227 and the G narrowband light 228. A set of G pixels (for example, a pair of B pixels and G pixels adjacent in the row direction) substantially constitutes a second color pixel. In any case, in the second embodiment, the first color light is red light, and the second color light is light in a shorter wavelength band than the wavelength band of red light.

[Third Embodiment]
In addition to the normal observation mode of the first embodiment and the narrow-band observation mode of the second embodiment, the present invention is also suitable for an oxygen saturation observation mode for calculating the oxygen saturation of an observation target.

  When the oxygen saturation observation mode is provided in the endoscope system 10, as shown in FIG. 17, the light source unit 20 includes an oxygen saturation in addition to the B broadband light source 24, the G broadband light source 25, and the R broadband light source 26. A narrow-band light source 301 that emits narrow-band light for measuring the above is provided. The narrow-band light source 301 emits light for measuring oxygen saturation (hereinafter referred to as measurement light). The measurement light is narrowband light having a wavelength having a large difference in extinction coefficient between oxyhemoglobin and reduced hemoglobin as a central wavelength. As shown in FIG. 18, for example, at a wavelength of about 470 nm, the difference between the extinction coefficient 303 of oxyhemoglobin and the extinction coefficient 304 of reduced hemoglobin increases. Therefore, in the present embodiment, the narrowband light source 301 is narrowband light having a center wavelength of about 470 nm. As can be seen from FIG. 18, in addition to about 470 nm, there are wavelengths where the difference between the absorption coefficient 303 of oxyhemoglobin and the absorption coefficient 304 of reduced hemoglobin is large in the purple, blue, or green wavelength band. Therefore, the narrow-band light source 301 can be a light source that emits narrow-band light having any one of these wavelengths as a central wavelength.

  In the oxygen saturation observation mode, when the imaging frame F1 is the first imaging frame and the next imaging frame F2 is the second imaging frame, as illustrated in FIG. 19, the light source control unit 22 performs the first imaging frame F1. In the accumulation period, the R broadband light 29 and the measurement light are turned on. In the readout period of the first imaging frame F1, the R broadband light 29 is kept on and other light such as measurement light is turned off. On the other hand, during the accumulation period of the second imaging frame F2, the B broadband light 27, the G broadband light 28, and the R broadband light 29 are turned on. In the readout period of the second imaging frame F2, the R broadband light 29 is kept on and the other lights are turned off.

  Therefore, the B image acquired in the first imaging frame F1 is an image obtained by imaging the observation target using the reflected light of the measurement light and has almost no rolling shutter distortion. The R image acquired in the first imaging frame F1 is brighter than the conventional one, as in the first embodiment. Further, the B image, the G image, and the R image acquired in the second imaging frame F2 are the same as the B image, the G image, and the R image in the first embodiment, and the B image and the G image are rolling shutter distortion. There is almost no image, and the R image is brighter than before. Hereinafter, for distinction, the B image acquired in the first imaging frame F1 is referred to as a B1 image, and the R image acquired in the first imaging frame F1 is referred to as an R1 image. The B image, the G image, and the R image acquired in the second imaging frame F2 are referred to as a B2 image, a G2 image, and an R2 image, respectively.

  As shown in FIG. 20, when the oxygen saturation observation mode is provided in the endoscope system 10, the image processing unit 61 includes a signal ratio calculation unit 311, an oxygen saturation calculation unit 312, a correlation storage unit 313, and An image generation unit 314 is provided. In the oxygen saturation observation mode, the image processing unit 61 acquires the B1 image, the B2 image, the G2 image, and the R2 image from the image acquisition unit 54.

  The signal ratio calculation unit 311 calculates, for example, the ratio of the B1 image to the G2 image (hereinafter referred to as signal ratio B1 / G2) and the ratio of the G2 image to the R2 image (hereinafter referred to as signal ratio R2 / G2) for each pixel. To calculate. The signal ratio B1 / G2 changes mainly depending on the value of the oxygen saturation level of the observation target and the blood volume, and the signal ratio R2 / G2 changes mainly according to the blood volume of the observation target.

  The oxygen saturation calculation unit 312 compares the signal ratio B1 / G2 and the signal ratio R2 / G2 calculated by the signal ratio calculation unit 311 with the correlation stored in the correlation storage unit 313, thereby determining the observation target. The oxygen saturation is calculated for each pixel. For example, as shown in FIG. 21, the correlation storage unit 313 stores the correlation between the signal ratio B1 / G2 and the signal ratio R2 / G2 and the oxygen saturation in the Log scale. The correlation between the signal ratio B1 / G2 and the signal ratio R2 / G2 and the oxygen saturation can be obtained in advance by simulation or the like.

  The image generation unit 314 generates an observation image (hereinafter referred to as an oxygen saturation image) that expresses the oxygen saturation of the observation target with a color. More specifically, the image generation unit 314 generates a color observation image using the B2 image, the G2 image, and the R2 image, as in the first embodiment. Thereafter, each pixel of the generated observation image is colored according to the value of the oxygen saturation calculated by the oxygen saturation calculation unit 312 to generate an oxygen saturation image. The image generation unit 314 inputs the generated oxygen saturation image to the display control unit 66 and displays it on the monitor 18.

  In the oxygen saturation observation mode, if there is a rolling shutter distortion in the captured image, the oxygen saturation calculation accuracy decreases. As described above, the B1 image, the G2 image, and the R2 image are used for the oxygen saturation calculation. Of these, the rolling shutter distortion needs to be more strictly removed to improve the oxygen saturation calculation accuracy. The captured images are a B1 image and a G2 image. It is because the signal ratio B1 / G2 has information on oxygen saturation essentially. The signal ratio B1 / G2 is a reference for eliminating the blood volume dependency of the signal ratio B1 / G2.

  In the third embodiment, the measurement light for obtaining the B1 image and the G broadband light 28 for obtaining the G2 image are turned on during the accumulation period of the image sensor 48 and are turned off during the readout period. The G2 image has almost no rolling shutter distortion. Therefore, if the endoscope system 10 is provided with the oxygen saturation observation mode as described above, the oxygen saturation can be calculated and displayed more accurately than in the past.

  Note that when the oxygen saturation observation mode is provided in the endoscope system 10 as in the third embodiment, it is preferable that the processor device 16 is provided with a staticity calculation unit 321 (see FIG. 20). The staticity calculation unit 321 calculates the staticity using the R image in the same manner as the staticity calculation unit 251 of the second embodiment. However, the stillness calculation unit 321 of the present embodiment acquires the R1 image and the R2 image from the image acquisition unit 54, and calculates the stillness using the R1 image, the R2 image, or the R1 image and the R2 image. Further, the stillness calculation unit 321 inputs the calculated stillness to the image processing unit 61.

  In the image processor 61, for example, the signal ratio calculator 311 uses the degree of stillness. Specifically, the signal ratio calculation unit 311 aligns the B1 image and the G2 image using the degree of stillness when calculating the signal ratio B1 / G2. Even if there is no rolling shutter distortion in these captured images, the acquired captured frames are different. Therefore, if there is a movement in the observation target or the like, the calculation accuracy of the signal ratio B1 / G2 decreases. Therefore, the signal ratio calculation unit 311 calculates the signal ratio B1 / G2 more accurately by aligning the B1 image and the G2 image using the degree of stillness. As a result, the oxygen saturation calculation unit 312 can calculate the oxygen saturation more accurately.

  In the third embodiment, the R broadband light 29 is the first color light, and both or one of the measurement light and the G broadband light 28 is the second color light. When the measurement light is the second color light, the pixel capable of receiving the measurement light is the second color pixel. In the third embodiment, since the measurement light is blue light, the B pixel is the second color pixel. When the G broadband light 28 is the second color light, the G pixel is the second color pixel. When both the measurement light and the G broadband light 28 are the second color light, depending on the wavelength of the measurement light used, the image sensor 48 may not have pixels that receive both the measurement light and the G broadband light 28. However, a set of pixels that can receive measurement light and G pixels substantially constitutes the second color pixel. In the third embodiment, a set of B pixels and G pixels (for example, a pair of B pixels and G pixels adjacent in the row direction) substantially constitutes a second color pixel. In any case, in the third embodiment, the first color light is red light, and the second color light is light in a shorter wavelength band than the wavelength band of red light.

[Fourth Embodiment]
In addition to the normal observation mode of the first embodiment, the narrow-band observation mode of the second embodiment, and the oxygen saturation observation mode of the third embodiment, the present invention selects a tissue or structure at a specific depth. It is also suitable for a specific depth emphasis observation mode for emphasizing automatically.

  When the endoscope system 10 is provided with a specific depth enhancement observation mode, as shown in FIG. 22, the light source unit 20 includes at least a first narrowband light source 401, a second narrowband light source 402, and an R broadband light source. 26 is provided. The first narrowband light source 401 and the second narrowband light source 402 emit narrowband light having different center wavelengths. In the present embodiment, as shown in FIG. 23, the first narrowband light source 401 emits V narrowband light (hereinafter referred to as first narrowband light in this embodiment) 403 and the second narrowband light source. 402 emits B narrowband light (hereinafter referred to as second narrowband light in this embodiment) 404. In addition, it is preferable that the first narrowband light source 401 and the second narrowband light source 402 can change the wavelength of the emitted narrowband light. In this case, the first narrowband light source 401 can be configured by, for example, a plurality of LEDs that emit narrowband light having different wavelengths, a combination of a broadband light source and a plurality of optical filters, or the like. The same applies to the second narrowband light source 402.

  In the specific depth enhancement observation mode, when the imaging frame F1 is the first imaging frame and the next imaging frame F2 is the second imaging frame, as shown in FIG. In the accumulation period of F1, the R broadband light 29 and the first narrowband light 403 are turned on. In the readout period of the first imaging frame F1, the R broadband light 29 is kept on and the first narrowband light 403 is turned off. On the other hand, during the accumulation period of the second imaging frame F2, the R broadband light 29 and the second narrowband light 404 are turned on. In the readout period of the second imaging frame F2, the R broadband light 29 is kept on and the second narrowband light 404 is turned off.

  Therefore, the B image acquired in the first imaging frame F1 is an image obtained by imaging the observation target using the reflected light of the first narrowband light 403 and has almost no rolling shutter distortion. The B image acquired in the second imaging frame F2 is an image obtained by imaging the observation target using the reflected light of the second narrowband light 404 and has almost no rolling shutter distortion. The R images acquired in the first imaging frame F1 and the second imaging frame F2 are brighter than the conventional one, as in the first embodiment. Hereinafter, for distinction, the B image acquired in the first imaging frame F1 is referred to as a B1 image, and the B image acquired in the second imaging frame is referred to as a B2 image. The R image is an image obtained by imaging the observation target using the reflected light of the R broadband light 29 in the first imaging frame F1 and the second imaging frame F2, and the length of the substantial accumulation period is also common. Without calling it an R image.

  As shown in FIG. 25, when the specific depth enhancement observation mode is provided in the endoscope system 10, the image processing unit 61 is provided with a change amount calculation unit 411 and an image generation unit 412. In the specific depth enhancement observation mode, the image processing unit 61 acquires the B1 image and the B2 image from the image acquisition unit 54.

  The change amount calculation unit 411 calculates the change amount of the B1 image and the B2 image for each pixel, and generates a change amount image (not shown) using the change amount as the pixel value of each pixel. The amount of change is, for example, the ratio or difference between the pixels of the B1 image and the B2 image.

  The image generation unit 412 generates an observation image in which one of the B1 image and the B2 image is assigned to the luminance channel and the change amount image generated by the change amount calculation unit 411 is assigned to the color difference channel. This observation image is an image in which a tissue or structure at a specific depth determined by the combination of the wavelengths of the first narrowband light 403 and the second narrowband light 404 is enhanced (hereinafter referred to as a specific depth enhanced image). . The image generation unit 412 inputs the generated specific depth emphasized image to the display control unit 66 and displays it on the monitor 18. Since the color difference channel includes two channels, that is, a Cr channel and a Cb channel, the image generation unit 412 assigns, for example, change amount images with different weights to the Cr channel and the Cb channel.

  In the specific depth emphasis observation mode, if the captured image has rolling shutter distortion, the calculation accuracy of the change amount calculated by the change amount calculation unit 411 is lowered. When the calculation accuracy of the change amount is low, for example, the depth accuracy (depth selection accuracy) of the tissue or structure to be emphasized in the specific depth emphasized image is lowered. However, since there is almost no rolling shutter distortion in the B1 image and the B2 image, an accurate amount of change can be calculated in the specific depth enhancement observation mode of the present embodiment. As a result, in the specific depth-enhanced image, it is possible to accurately emphasize a tissue or structure having an arbitrary depth determined by the wavelengths of the first narrowband light 403 and the second narrowband light 404.

  In addition, when providing the specific depth emphasis observation mode in the endoscope system 10 like the said 4th Embodiment, it is preferable to provide the stationary degree calculation part 421 in the processor apparatus 16 (refer FIG. 25). Similar to the static degree calculator 251 of the second embodiment and the static degree calculator 321 of the third embodiment, the static degree calculator 421 calculates the static degree using the R image. In addition, the staticity calculation unit 421 of the present embodiment inputs the calculated staticity to the image processing unit 61 as in the staticity calculation unit 321 of the third embodiment.

  In the image processing unit 61, the change amount calculation unit 411 uses the stillness. Specifically, the change amount calculation unit 411 aligns the B1 image and the B2 image using the degree of staticity when calculating the change amount. This is because, even if there is no rolling shutter distortion in these captured images, the acquired captured frames are different, so that if the observation target or the like moves, the calculation accuracy of the amount of change is reduced. The change amount calculation unit 411 can calculate the change amount more accurately by aligning the B1 image and the B2 image using the degree of stillness. As a result, in the specific depth emphasized image, the selection accuracy of the depth of the tissue or structure to be enhanced is improved.

  In the fourth embodiment, the R broadband light 29 is the first color light, and both or one of the first narrow band light 403 and the second narrow band light 404 is the second color light. In the fourth embodiment, since the first narrowband light 403 and the second narrowband light 404 are received by the B pixel, the B pixel is the second color pixel. However, depending on how each wavelength of the first narrowband light 403 and the second narrowband light 404 is selected, the pixel that receives the reflected light of the first narrowband light 403 and the reflected light of the second narrowband light 404 However, when the first narrowband light 403 is the second color light, the pixels that receive the reflected light of the first narrowband light 403 are the second color pixels. When the reflected light or the like of the second narrowband light 404 is the second color light, the pixel that receives the second narrowband light 404 is the second color pixel. When both the first narrowband light 403 and the second narrowband light 404 are the second color light, the pixels that receive the reflected light of the first narrowband light 403 and the second narrowband light 404 are reflected. A set of pixels that receive light or the like substantially constitutes the second color pixel. In any case, the first color light is red light, and the second color light is light in a shorter wavelength band than the wavelength band of red light.

[Fifth Embodiment]
In addition to the observation modes of the first to fourth embodiments, the present invention is also suitable for a deep blood vessel emphasis observation mode for emphasizing a thick blood vessel (hereinafter referred to as a deep blood vessel) in a particularly deep position under the mucous membrane. It is.

  When the deep blood vessel enhancement observation mode is provided in the endoscope system 10, as shown in FIG. 26, the light source unit 20 includes at least a first R narrowband light source 501, a second R narrowband light source 502, an R broadband light source 26, and , Cy broadband light source 503 is provided. As shown in FIG. 27, the first R narrowband light source 501 emits red narrowband light (hereinafter referred to as first R narrowband light) 506 having a center wavelength of about 600 nm, for example. The second R narrowband light source 502 emits red narrowband light (hereinafter referred to as second R narrowband light) 507 having a center wavelength of about 630 nm, for example. The Cy broadband light source 503 emits cyan broadband light (hereinafter referred to as Cy broadband light) 509 having a wavelength band ranging from about 400 nm to about 550 nm.

  In the deep blood vessel enhancement observation mode, when the imaging frame F1 is the first imaging frame and the next imaging frame F2 is the second imaging frame, as illustrated in FIG. 28, the light source control unit 22 performs the first imaging frame F1. In the accumulation period, the first R narrowband light 506 and the Cy broadband light 509 are turned on. In the readout period of the first imaging frame F1, the first R narrowband light 506 is turned off, the R broadband light 29 is turned on instead, and the Cy broadband light 509 is kept on. On the other hand, in the accumulation period of the second imaging frame F2, the second R narrowband light 507 and the Cy broadband light 509 are turned on. In the readout period of the second imaging frame F2, the second R narrowband light 507 is turned off, the R broadband light 29 is turned on instead, and the Cy broadband light 509 is kept on.

  Therefore, the R image acquired in the first imaging frame F1 is the image component obtained by imaging the observation target using the reflected light of the first R narrowband light 506, the reflection target of the R broadband light 29, and the like. This is an image obtained by superimposing the captured image component, and at least the image component captured using the reflected light of the first R narrowband light 506 has almost no rolling shutter distortion. Similarly, the R image acquired in the second imaging frame F2 is an observation object using an image component obtained by imaging the observation object using the reflected light of the second R narrowband light 507 and the reflected light of the R broadband light 29, and the like. And an image component captured using at least the reflected light of the second R narrowband light 507 has almost no rolling shutter distortion. Hereinafter, for distinction, the R image acquired in the first imaging frame F1 is referred to as an R1 image, and the R image acquired in the second imaging frame F2 is referred to as an R2 image. Note that the B image acquired in the first imaging frame F1 and the second imaging frame F2 is an image obtained by imaging an observation target using reflected light of a blue component included in the Cy broadband light 509 or the like. Similarly, the G image acquired in the first imaging frame F1 and the second imaging frame F2 is an image obtained by capturing an observation target using reflected light of a green component included in the Cy broadband light 509 or the like.

  As shown in FIG. 29, when the deep blood vessel enhancement observation mode is provided in the endoscope system 10, the image processing unit 61 is provided with a change amount calculation unit 511 and an image generation unit 512. In the deep blood vessel enhancement observation mode, the image processing unit 61 acquires the R1 image and the R2 image and at least the G image from the image acquisition unit 54.

  The change amount calculation unit 511 calculates the change amount of the R1 image and the R2 image for each pixel. The amount of change in the present embodiment is, for example, the ratio or difference between the pixels of the R1 image and the R2 image. In addition, when the change amount is calculated, the change amount calculation unit 511 calculates a correction coefficient based on the calculated change amount, and multiplies the R1 image or the R2 image by the correction coefficient to generate a correction image (not shown). Then, the change amount calculation unit 511 inputs the generated corrected image to the image generation unit 512.

  The image generation unit 512 uses the R1 image or the R2 image that is not used for generating the correction image, the correction image generated by the change amount calculation unit 511, and the G image acquired from the image acquisition unit 54. An observation image is generated by assigning to the B channel, the G channel, and the R channel. This observation image is an image in which a deep blood vessel is emphasized (hereinafter referred to as a deep blood vessel emphasis image). The image generation unit 512 displays the generated deep blood vessel emphasized image on the monitor 18 by inputting it to the display control unit 66.

  In the deep blood vessel enhancement observation mode, there is a rolling shutter distortion in the image component captured by the first R narrowband light 506 in the R1 image and the image component captured by the second R narrowband light 507 in the R2 image. Then, the calculation accuracy of the change amount calculated by the change amount calculation unit 511 decreases. When the calculation amount of the change amount is low, for example, in the deep blood vessel emphasis image, the deep blood vessel emphasis is weakened, and the visibility of the deep blood vessel is lowered. However, in the R1 image, there is almost no rolling shutter distortion in the image component captured by at least the first R narrowband light 506, and in the R2 image, the image is obtained by capturing the observation target using at least the second R narrowband light 507. There is almost no rolling shutter distortion in the components. Therefore, an accurate amount of change can be calculated in the deep blood vessel enhancement observation mode of the present embodiment. As a result, deep blood vessels can be clearly emphasized in the deep blood vessel enhancement image.

  Note that when the deep blood vessel emphasis observation mode is provided in the endoscope system 10 as in the fifth embodiment, it is preferable that the processor device 16 is provided with a stasis degree calculation unit 521 (see FIG. 29). The stationary degree calculation unit 521 calculates the same degree of stationary as the respective stationary degree calculation units of the second embodiment to the fourth embodiment. However, in the present embodiment, the stillness calculation unit 521 calculates the stillness using a G image or a B image. Since the G image and the B image in the present embodiment have rolling shutter distortion, images of fine blood vessels and the like are less likely to be captured than the G image or the B image of the first to fourth embodiments. For this reason, in this embodiment, an accurate stillness can be calculated using a G image or a B image.

  The stillness calculation unit 521 inputs the calculated stillness to the image processing unit 61 in the same manner as the stillness calculation unit 421 of the fourth embodiment. In the image processing unit 61, when the change amount calculation unit 511 calculates the change amount, the R1 image and the R2 image are aligned using the degree of stillness. Thereby, the change amount calculation unit 511 can calculate the change amount more accurately. As a result, in the deep blood vessel emphasis image, the deep blood vessels can be more clearly emphasized, so that the visibility of the deep blood vessels is improved.

  In the fifth embodiment, the Cy broadband light 509 is the first color light, and the G pixel or the B pixel that receives the Cy broadband light 509 is the first color pixel. In the fifth embodiment, the first R narrowband light 506 and the second R narrowband light 507 are the second color light, and the R pixel that receives the reflected light and the like is the second color pixel. Therefore, unlike the first to fourth embodiments, in the fifth embodiment, the second color light is red light, and the first color light is light in a shorter wavelength band than the red light.

  In each of the observation modes of the third to fifth embodiments, the light source control unit 22 changes the wavelength band or spectral spectrum of the illumination light for each accumulation period of the image sensor 48, and the plurality of these An operation using a plurality of captured images acquired in the imaging frame (hereinafter referred to as multi-frame calculation) and an observation image (hereinafter referred to as a multi-frame observation image) using the plurality of captured images acquired in the plurality of imaging frames. Generated). As can be seen from the third embodiment to the fifth embodiment, when multiframe calculation is performed or when a multiframe observation image is generated, a staticity calculation unit is provided, and the staticity calculated by the staticity calculation unit is provided. It is preferable that the captured image used for multi-frame calculation or generation of the multi-frame observation image is aligned using.

  In the third embodiment to the fifth embodiment, one multi-frame observation image is stationary using a plurality of captured images generated using signals acquired in a plurality of readout periods. One multi-frame observation image can be generated using a plurality of signals respectively acquired in the readout period. For example, when the image processing unit 61 has the function of the image acquisition unit 54 or when the image acquisition unit 54 has the function of the image processing unit 61, 1 is used by using a plurality of signals respectively acquired in a plurality of readout periods. One multi-frame observation image will be generated.

  Further, as described in the third to fifth embodiments, when a static degree calculation unit is provided in a system that performs multi-frame calculation or a system that generates a multi-frame observation image, the static degree calculation unit includes the first color. It is preferable to calculate the degree of stillness using a captured image generated using a signal obtained from a pixel or a signal obtained from a first color pixel. This is because the degree of stillness can be accurately calculated by using a signal obtained from the first color pixel or a captured image generated using the signal obtained from the first color pixel.

  Note that the observation modes of the first to fifth embodiments can be provided in any combination in the endoscope system 10.

  In the first to fifth embodiments, the imaging control unit 23 reads signals from the pixels of the image sensor 48 using a so-called progressive method, but may read signals from the pixels of the image sensor 48 using a so-called interlace method. good. In this case, the imaging control unit 53 reads and resets the signal every other pixel row. In this case, each pixel of the image sensor 48 includes a first pixel group that is an assembly of pixel rows including R pixels (first color pixels) and an assembly of pixel rows that does not include R pixels (first color pixels). And a second pixel group. That is, when reading is performed by the interlace method, signal reading and resetting are performed for each pixel group of the first pixel group and the second pixel group. Then, a captured image is acquired for each color included in the first pixel group or the second pixel group.

  For example, in the normal observation mode of the first embodiment, as shown in FIG. 30, reading for each pixel row is divided into odd-numbered row reading and even-numbered row reading and sequentially read. Reading and resetting signals of each pixel group is a rolling shutter system. In each pixel group, signals are sequentially read and reset in ascending order of pixel row numbers. As described above, when reading is performed by the interlace method, an R image formed using the R pixel signal of the first pixel group and a G image formed using the G image signal of the first pixel group (hereinafter, referred to as “R image”). , A Gr image for distinction), a B image formed using the B pixel signal of the second pixel group, and a G image formed using the G pixel signal of the second pixel group (hereinafter, for distinction). 4 types of captured images are acquired. For this reason, as shown in FIG. 31, the image processing unit 61 first generates a G image similar to that in the first embodiment using the Gr image and the Gb image, and then the G image, the B image, A normal observation image is generated using the R image. In each of the observation modes of the second to fifth embodiments, signals can be read out in an interlaced manner as in the normal observation mode of the first embodiment.

  In the first to fifth embodiments, the image sensor 48 is a so-called primary color sensor, but a complementary color sensor using complementary color filters such as cyan, magenta, yellow, and green is also used. can do. When a complementary color sensor is used as the image sensor 48, the components of the first color light and the second color light included in the illumination light may be controlled as in the first to fifth embodiments. In addition, since the complementary color image acquired from the complementary color sensor can be converted into an image of each RGB color as in the first to fifth embodiments, image processing and the like are performed in the first to fifth embodiments. It can be done in the same way as the form.

  In the first to fifth embodiments, the pixel array of the image sensor 48 is a so-called square array, but the pixel array of the image sensor 48 may be an array other than a square array such as a so-called honeycomb array.

  In the first to fifth embodiments, the light source of the light source unit 20 is an LED, but the endoscope system 10 uses another semiconductor light source such as an LD (Laser Diode) instead of the LED. May be. A combination of a semiconductor light source and a phosphor that emits light of another color using light emitted from the semiconductor light source as excitation light may be used. A lamp light source such as a xenon lamp may also be used for the light source unit 20. The light source of the light source unit 20 may be configured by combining a semiconductor light source, a semiconductor light source and a phosphor, and an optical filter that adjusts a wavelength band or a spectral spectrum together with a lamp light source. For example, by using a white LED in combination with an optical filter, light of each color used in the first to fifth embodiments can be generated.

  In the first to fifth embodiments, the present invention is implemented in an endoscope system that performs observation by inserting an endoscope 12 provided with an image sensor 48 into a subject. The present invention is also suitable for an endoscope system. As shown in FIG. 32, for example, a capsule endoscope system includes at least a capsule endoscope 700 and a processor device (not shown).

  The capsule endoscope 700 includes a light source unit 702, a control unit 703, an image sensor 704, an image processing unit 706, and a transmission / reception antenna 708. The light source unit 702 corresponds to the light source unit 20. The control unit 703 functions in the same manner as the light source control unit 22 and the control unit 52. In addition, the control unit 703 can communicate with the processor device of the capsule endoscope system using the transmission / reception antenna 708 wirelessly. The processor device of the capsule endoscope system is substantially the same as the processor device 16 of each of the above embodiments, but the image processing unit 706 corresponding to the image acquisition unit 54 and the image processing unit 61 is provided in the capsule endoscope 700. The observation image is transmitted to the processor device via the transmission / reception antenna 708. The image sensor 704 is the same as the image sensor 48.

DESCRIPTION OF SYMBOLS 10 Endoscope system 12 Endoscope 12a Insertion part 12b Operation part 12c Bending part 12d Tip part 12e Angle knob 13a Mode change switch 13b Zoom operation part 14 Light source apparatus 16 Processor apparatus 18 Monitor 19 Console 20, 702 Light source part 22 Light source control Unit 23 imaging control unit 24 B broadband light source 25 G broadband light source 26 R broadband light source 27 B broadband light 28 G broadband light 29 R broadband light 30 a illumination optical system 30 b imaging optical system 41 light guide 45 illumination lens 46 objective lens 47 zoom lens 48 , 704 Image sensor 52, 703 Control unit 53 Imaging control unit 54 Image acquisition unit 56 DSP
58 Noise reduction unit 59 Conversion unit 61, 706 Image processing unit 66 Display control unit 224 Narrowband light source 225 G Narrowband light source 227 B Narrowband light 228 G Narrowband light 251, 321, 421, 521 Stillness calculation unit 252 V Narrow Band light source 301 Narrow band light source 311 Signal ratio calculation unit 312 Oxygen saturation calculation unit 313 Correlation storage unit 314, 412, 512 Image generation unit 401 First narrow band light source 402 Second narrow band light source 403 First narrow band light 404 First 2 narrowband light 411,511 change amount calculation unit 501 1R narrowband light source 502 2R narrowband light source 503 Cy broadband light source 506 1R narrowband light 507 2R narrowband light 509 Cy broadband light 700 capsule endoscope 708 transmission / reception Antenna F1, F2 Imaging frame L1 Length of accumulation period L2 Substantial accumulation period

Claims (9)

The first color light and the first color light can be independently controlled to turn on and off the second color light having different wavelength bands or spectral spectra, and the illumination includes either the first color light or the second color light A light source unit for generating light;
A first color pixel that receives the first color light and a second color pixel that receives the second color light, and images an observation target using at least the first color pixel or the second color pixel. An image sensor;
Charges are accumulated in at least the first color pixel and the second color pixel during the accumulation period, and signals are output from the first color pixel and the second color pixel in a rolling shutter system during the readout period following the accumulation period. An imaging control unit for reading;
A light source controller that turns on the first color light and the second color light in the accumulation period, maintains lighting of the first color light in the readout period, and turns off the second color light in the readout period;
Equipped with a,
The endoscope system, wherein the second color light is red light, and the first color light is light in a shorter wavelength band than the red light . The image sensor includes a first pixel group that is an aggregate of pixel rows including the first color pixels, and a second pixel group that is an aggregate of pixel rows that do not include the first color pixels. And,
The endoscope system according to claim 1, wherein the imaging control unit reads a signal for each pixel group of the first pixel group and the second pixel group. The image sensor includes a first pixel group that is an aggregate of pixel rows including the first color pixels, and a second pixel group that is an aggregate of pixel rows that do not include the first color pixels. And,
The endoscope system according to claim 1 or 2, wherein an image is generated for each color included in the first pixel group or the second pixel group.   The endoscope system according to claim 1, wherein the light source control unit makes the light amount of the first color light in the readout period equal to the light amount of the first color light in the accumulation period.   The endoscope system according to any one of claims 1 to 4, wherein the light source control unit makes the light amount of the second color light in the readout period smaller than at least the light amount of the second color light in the accumulation period. .   The endoscope system according to any one of claims 1 to 5, wherein the light source control unit changes a wavelength band or a spectral spectrum of the illumination light for each accumulation period.   The image generation part which produces | generates one image using the some signal each acquired in the said some read-out period, or the some image produced | generated using the signal acquired in the said some read-out period is provided. The endoscope system according to any one of 6.   A stasis degree calculation unit that calculates the stasis degree of the observation target using an image generated using a signal obtained from the first color pixel or an image obtained from the signal obtained from the first color pixel. The endoscope system according to any one of 7. The first color light and the first color light can be independently controlled to turn on and off the second color light having different wavelength bands or spectral spectra, and the illumination includes either the first color light or the second color light A light source unit that generates light; a first color pixel that receives the first color light; and a second color pixel that receives the second color light, and at least the first color pixel or the second color pixel. An image sensor for imaging an observation object using the second imaging device , wherein the second color light is red light, and the first color light is light in a shorter wavelength band than the red light. In the method
An imaging controller that accumulates charges in at least the first color pixel and the second color pixel during an accumulation period;
The imaging control unit reading a signal from the first color pixel and the second color pixel by a rolling shutter method in a reading period following the accumulation period;
A light source control unit lighting the first color light and the second color light in the accumulation period;
The light source control unit maintaining the lighting of the first color light in the reading period and turning off the second color light in the reading period;
A method of operating an endoscope system comprising:

Images