JP7733754B2 - Image processing method, image processing device, image processing program, and endoscope system - Google Patents

Description

The present invention relates to an image processing method, an image processing device, and an image processing program for processing images captured using an image guide. It also relates to an endoscope system equipped with such an image processing device.

Endoscope systems are widely used, in which one end of an image guide is inserted into the body, and the other end is imaged to obtain images that include objects inside the body as subjects. For example, Patent Document 1 discloses a vascular endoscope system that reduces the risk of damage to the intima of small blood vessels by making the image guide thinner and more flexible.

Japanese Patent Application Publication No. 8-191439

In an endoscopic system using an image guide, the resolution is determined by the core density of the image guide. For example, structures approximately 100 times the inter-core distance can be seen from the image formed on the end face of the image guide, but structures approximately 10 times the inter-core distance are difficult to see from the image formed on the end face of the image guide, and structures approximately the same size as the inter-core distance cannot be seen from the image formed on the end face of the image guide.

The vascular endoscope system described in Patent Document 1 improves the quality of the resulting image by correcting differences in brightness and color tone for each pixel, which are caused by differences in the transmission characteristics of each core in the image guide. However, such corrections cannot exceed the resolution determined by the core density of the image guide.

One aspect of the present invention was made in consideration of the above-mentioned problems, and aims to provide image processing technology that improves the resolution or visibility of the resulting images.

An image processing method according to one aspect of the present invention includes an acquisition process for acquiring multiple target images by imaging one end of an image guide having multiple cores, the other end of which faces the target, and a super-resolution process for generating a super-resolution image by simply averaging or weighted averaging pixel values of pixels in the multiple target images that correspond to the same point on the target.

An image processing device according to one aspect of the present invention is an image processing device equipped with at least one processor, the processor having multiple cores, and performing an acquisition process to acquire multiple target images obtained by imaging one end of an image guide facing the other end of the target, and a super-resolution process to generate a super-resolution image by averaging or weighted averaging pixel values of pixels corresponding to the same point on the target in the multiple target images.

An image processing program according to one aspect of the present invention is an image processing program for operating at least one processor, causing the processor to perform an acquisition process in which the processor acquires multiple target images obtained by imaging one end of an image guide having multiple cores and the other end facing the target, and a super-resolution process in which the processor generates a super-resolution image by averaging or weighted averaging pixel values of pixels in the multiple target images that correspond to the same point on the target.

According to one aspect of the present invention, the resolution or visibility of the resulting image can be improved.

1 is a schematic diagram showing the configuration of an endoscope system according to an embodiment of the present invention. FIG. 2 is a block diagram showing the configuration of an image processing device included in the endoscope system shown in FIG. 1 . FIG. 1 is a flowchart showing the flow of an image processing method according to an embodiment of the present invention. 4 is a diagram showing an example of an execution of the image processing method shown in Fig. 3. Specifically, it is a diagram showing an object to be observed in the execution example. 4 is a diagram showing an example of an execution of the image processing method shown in Fig. 3. Specifically, it is a diagram showing a mesh image obtained by processing an image of a white standard plate in the execution example. 4 is a diagram showing an example of execution of the image processing method shown in Fig. 3. Specifically, it is a diagram showing a first object image obtained by object imaging processing in the example of execution. 4 is a diagram showing an example of execution of the image processing method shown in Fig. 3. Specifically, it is a diagram showing a second target image obtained by target image capturing processing in the example of execution. 4 is a diagram showing an example of execution of the image processing method shown in Fig. 3. Specifically, it is a diagram showing a third object image obtained by object imaging processing in the example of execution. 4 is a diagram showing an example of execution of the image processing method shown in Fig. 3. Specifically, it is a diagram showing a fourth object image obtained by object imaging processing in the example of execution. 4 is a diagram showing an example of execution of the image processing method shown in Fig. 3. Specifically, it is a diagram showing a super-resolution image obtained by super-resolution processing (simple averaging) in the example of execution. 4 is a diagram showing an example of execution of the image processing method shown in Fig. 3. Specifically, it is a diagram showing an output image obtained by mask processing in the example of execution. FIG. 4 is a flowchart showing the flow of an image processing method according to a modified example of the image processing method shown in FIG. 3 . 13 is a diagram showing an example of execution of the image processing method shown in Fig. 12. Specifically, it is a diagram showing a first target image obtained by transparency processing in the example of execution. 13 is a diagram showing an example of execution of the image processing method shown in Fig. 12. Specifically, it is a diagram showing a second target image obtained by transparency processing in the example of execution. 13 is a diagram showing an example of execution of the image processing method shown in Fig. 12. Specifically, it is a diagram showing a third target image obtained by transparency processing in the example of execution. 13 is a diagram showing an example of execution of the image processing method shown in Fig. 12. Specifically, it is a diagram showing a fourth target image obtained by transparency processing in the example of execution. 13 is a diagram showing an example of execution of the image processing method shown in Fig. 12. Specifically, it is a diagram showing a super-resolution image obtained by super-resolution processing (weighted average) in the example of execution. 13 is a diagram showing an example of execution of the image processing method shown in Fig. 12. Specifically, it is a diagram showing an output image obtained by mask processing in the example of execution. 13A and 13B are diagrams illustrating an example of execution of the image processing method shown in FIG. 12. Specifically, they are enlarged views showing a portion of a super-resolution image obtained by core rendering processing in this example. (a) corresponds to a case where opacity resetting is not performed, (b) corresponds to a case where opacity resetting is performed so that the opacity of each pixel whose distance from the center of the core region is equal to or less than a predetermined threshold decreases linearly as the distance increases, and (c) corresponds to a case where the opacity of each pixel whose distance from the center of the core region is equal to or less than a predetermined threshold decreases according to a Gaussian distribution as the distance increases.

(Configuration of endoscope system)
An endoscope system 1 according to an embodiment of the present invention will be described with reference to Fig. 1. In Fig. 1, (a) is a schematic diagram showing the configuration of the endoscope system 1, (b) is a plan view and a cross-sectional view showing the structure of a light guide 13p in a light guide plug portion 13a, (c) is a plan view and a cross-sectional view showing the structure of an image guide 13q in an image guide plug portion 13c, and (d) is a plan view and a cross-sectional view showing the structures of the light guide 13p and image guide 13q in an insertion portion 13g.

The endoscope system 1 is a system for displaying an image of an object O to be observed on a display D. As shown in FIG. 1(a), the endoscope system 1 includes an image processing device 10, a light source device 11, a camera device 12, and an endoscopic probe 13.

As shown in FIG. 1(a), the light source device 11 includes a light source 11a and a condenser lens 11b. The light source 11a is arranged on the optical axis of the condenser lens 11b. The light source 11a may be, for example, an LED (Light Emitting Diode), a xenon lamp, or a halogen lamp.

As shown in FIG. 1(a), the camera device 12 includes an image sensor 12a, an objective lens 12b, and a signal processing circuit 12c. The image sensor 12a is disposed on the optical axis of the objective lens 12b. The signal processing circuit 12c is electrically connected to the image sensor 12a. The image sensor 12a may be, for example, a CMOS (Complementary Metal Oxide Semiconductor) sensor or a CCD (Charge Coupled Device) sensor.

As shown in FIG. 1(a), the endoscopic probe 13 includes: (1) a light guide plug portion 13a mechanically connected to the light source device 11; (2) a light guide lead portion 13b having one end connected to the light guide plug portion 13a; (3) an image guide plug portion 13c mechanically connected to the camera device 12; (4) an image guide lead portion 13d having one end connected to the image guide plug portion 13c; (5) a branch portion 13e having one end connected to the other end of the light guide lead portion 13b and the image guide lead portion 13d; (6) an intermediate portion 13f having one end connected to the other end of the branch portion 13e; and (7) an insertion portion 13g having one end connected to the other end of the intermediate portion 13f. The endoscopic probe 13 may further include a water passage lumen, a working lumen, an insertion portion bending mechanism, and a bending operation portion (all not shown).

As shown in Figures 1(b) and 1(d), one or more light guides 13p that function as waveguides for guiding the illumination light generated by the light source device 11 are provided inside the light guide plug portion 13a, light guide lead portion 13b, branch portion 13e, intermediate portion 13f, and insertion portion 13g. Each light guide 13p is embedded in the cladding. The light guides 13p contained in the light guide plug portion 13a and light guide lead portion 13b are bundled with an image guide 13q (described below) at the branch portion 13e, and are contained together with this image guide 13q in the intermediate portion 13f and insertion portion 13g.

As shown in (c) and (d) of Figure 1, image guide 13q, a waveguide that guides scattered light generated by object O, is provided inside image guide plug portion 13c, image guide lead portion 13d, branch portion 13e, intermediate portion 13f, and insertion portion 13g. Image guide 13q is composed of a cladding and multiple cores (e.g., thousands to tens of thousands of cores) arranged regularly or randomly inside the cladding. Image guide 13q contained in image guide plug portion 13c and image guide lead portion 13d is bundled with the aforementioned light guide 13p at branch portion 13e, and is contained together with this light guide 13p in intermediate portion 13f and insertion portion 13g.

Examples of materials that can be used as the image guide 13q include an image guide fiber, an image fiber, a fiber bundle, and a fiber conduit. The material of the image guide 13q may be silica, multi-component glass, or plastic. Furthermore, there are no particular limitations on the number of cores and core arrangement of the image guide 13q. As an example, an image guide fiber with 3,000 cores arranged on the lattice points of a hexagonal close-packed lattice can be used as the image guide 13q. However, the resolution of the image guide 13q is lower than the resolution of the image sensor 12a; that is, the number of cores of the image guide 13q is less than the number of cells of the image sensor 12a.

As shown in FIG. 1(d), an objective lens 13r is provided at the tip of the insertion section 13g. One end of the objective lens 13r forms the end face of the insertion section 13g, and the other end of the objective lens section 13s faces one end of the image guide 13q. The objective lens 13r can be, for example, a GRIN lens or a lens unit consisting of multiple lenses.

The endoscope system 1 operates, for example, as follows. That is, with the insertion section 13g inserted into the patient's body, illumination light is emitted from the light source 11a. The illumination light emitted from the light source 11a is focused by the focusing lens 11b on the end face of the light guide 13p on the light guide plug section 13a side. The illumination light that enters the core of the light guide 13p at the end face of the light guide 13p on the light guide plug section 13a side is guided through the core of the light guide 13p and exits from the core of the light guide 13p at the end face of the light guide 13p on the insertion section 13g side. The illumination light that exits from the core of the light guide 13p at the end face of the light guide 13p on the insertion section 13g side is irradiated onto an object O inside the patient's body.

Scattered light generated at each point on the object O by scattering the illumination light inside the patient's body is focused by the objective lens 13r at each point on the end face of the image guide 13q on the insertion section 13g side. As a result, an image of the object O is formed on the end face of the image guide 13q on the insertion section 13g side. The scattered light incident on the core of the image guide 13q at the end face of the image guide 13q on the insertion section 13g side by the objective lens 13r is guided through the core of the image guide 13q and exits from the core of the image guide 13q at the end face of the image guide 13q on the image guide plug section 13c side. As a result, an image of the object O is formed on the end face of the image guide 13q on the image guide plug section 13c side, just like on the end face of the image guide 13q on the insertion section 13g side.

Light emitted from each core of image guide 13q at the end face of image guide plug portion 13c of image guide 13q (light derived from scattered light generated at each point on object O) is focused by objective lens 12b at each point on the light-receiving surface of image sensor 12a. This forms an image of object O on the light-receiving surface of image sensor 12a. Image sensor 12a generates an electrical signal representing the image of object O and inputs the generated electrical signal to signal processing circuit 12c. Signal processing circuit 12c generates an object image representing the image of object O from the electrical signal input from image sensor 12a.

The camera device 12 inputs the generated target image to the image processing device 10. The image processing device 10 generates an output image to be displayed on the display D from the target image input from the camera device 12 in accordance with an image processing method S100 described below. Note that the functions performed by the image processing device 10 may also be performed by the camera device 12. In other words, instead of a configuration in which the image processing method S100 described below is performed by the image processing device 10, a configuration in which the camera device 12 performs the method may be adopted. In this case, the image processing device 10 can be omitted from the endoscope system 1.

In this specification, the term "image" refers to a two-dimensional array of pixel values. A pixel value in a monochrome image consists of, for example, a single numerical value corresponding to brightness. In a D-level monochrome image, brightness takes on an integer value between 0 and D-1 inclusive. A pixel value in a color image consists of, for example, three numerical values corresponding to RGB. Here, R represents the red component, G represents the green component, and B represents the blue component. In a D-level color image, the red component, green component, and blue component each take on an integer value between 0 and D-1 inclusive. A pixel value in a transparent color image consists of, for example, four numerical values corresponding to RGBA. Here, A represents opacity. In the case of a D-level transparent color image, the red component, green component, blue component, and opacity each take on an integer value between 0 and D-1 inclusive.

Pixels in an image are identified by their coordinates (x, y). If the image size is W pixels wide by H pixels high, then x is an integer between 0 and W-1, inclusive, and y is an integer between 0 and H-1, inclusive. A pixel with coordinates (x, y) will hereafter be referred to as pixel (x, y). For example, pixel (0, 0) is the pixel in the upper left corner of the image, pixel (1, 0) is the pixel immediately to the right of pixel (0, 0), and pixel (0, 1) is the pixel immediately below pixel (0, 0). The pixel value (brightness value) of pixel (x, y) in a monochrome image is referred to as A(x, y). Furthermore, for the pixel value of pixel (x, y) in a color image, its red component is referred to as R(x, y), its green component is referred to as G(x, y), and its blue component is referred to as B(x, y). Similarly, for the pixel value of pixel (x, y) in an opacity-added color image, its red component is written as R(x, y), its green component as G(x, y), its blue component as B(x, y), and its opacity as A(x, y).

There are no particular limitations on the file format used to save images. For example, file formats such as PNG, JPEG, and GIF can be used to save monochrome or color images. Furthermore, file formats such as transparent PNG can be used to save transparent color images.

(Configuration of image processing device)
The configuration of the image processing device 10 will be described with reference to Fig. 2. Fig. 2 is a block diagram showing the configuration of the image processing device 10.

As shown in FIG. 2, the image processing device 10 includes a processor 101, a memory 102, a storage 103, an input/output interface 104, and a bus 105. The processor 101, the memory 102, the storage 103, and the input/output interface 104 are connected to each other via the bus 105.

Memory 102 is configured to expand and store an image processing program P100 for implementing an image processing method S100 (described below) in a state that can be referenced by processor 101. Memory 102 is also used to store various images acquired from camera device 12. Note that memory 102 can be, for example, a semiconductor RAM (Random Access Memory).

The processor 101 is configured to execute the image processing method S100 (described below) in accordance with the image processing program P100 loaded in the memory 102. The processor 101 may be, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or a combination of these.

Storage 103 is configured to store (non-volatilely store) the image processing program P100. When implementing the image processing method S100 described below, processor 101 expands and references the image processing program P100 stored in storage 103 in memory 102. Note that storage 103 may be, for example, a flash memory, HDD (Hard Disk Drive), SSD (Solid State Drive), or a combination of these.

The input/output interface 104 is configured to connect the camera device 12 and the display D to the image processing device 10. The input/output interface 104 may be, for example, a USB (Universal Serial Bus) interface.

Note that while the configuration described here is one in which the image processing method S100, described below, is executed by a single processor 101 provided in a single computer (a computer that functions as the image processing device 10), the configuration is not limited to this. In other words, it is also possible to adopt a configuration in which the image processing method S100, described below, is executed jointly by multiple processors provided centrally in a single computer, or distributed across multiple computers.

The image processing program P100 may be recorded on a computer-readable, non-transitory, tangible recording medium. This recording medium may be memory 102, storage 103, or other recording medium. For example, tape, disk, card, semiconductor memory, programmable logic circuit, etc. may be used as other recording media.

(Flow of image processing method) The flow of image processing method S100 according to one embodiment of the present invention will be described with reference to Figure 3. Figure 3 is a flow diagram showing the flow of image processing method S100.

As shown in FIG. 3, the image processing method S100 includes a white standard plate imaging process S101, a white balance adjustment process S102, an object imaging process S103, an object image acquisition process S104, a super-resolution process S105, a masking process S106, and an image display process S107. Here, the white standard plate imaging process S101, the white balance adjustment process S102, and the object imaging process S103 are processes performed primarily by the camera device 12. On the other hand, the object image acquisition process S104, the super-resolution process S105, the masking process S106, and the image display process S107 are processes performed primarily by the image processing device 10.

The white standard plate imaging process S101 is a process in which the user faces the objective lens 13r of the endoscopic probe 13 toward the white standard plate, and the camera device 12 images the end face of the image guide plug portion 13c side of the image guide 13q. The white balance adjustment process S102 is a process in which the camera device 12 adjusts the white balance by referring to the image captured in the white standard plate imaging process S101. The image captured in the white standard plate imaging process S101 and referenced in the white balance adjustment process S102 is a color image in which areas corresponding to the cores of the image guide 13q are bright and other areas are dark. This image will be referred to below as the mesh image I0. Specific examples of the mesh image I0 obtained in the white standard plate imaging process S101 will be described later with reference to different drawings.

In the object imaging process S103, the user positions the objective lens 13r of the endoscopic probe 13 toward the object O, and the camera device 12 images the end face of the image guide 13q on the image guide plug portion 13c side. In the object image acquisition process S104, the image processing device 10 acquires the image captured in the object imaging process S103 from the camera device 12 and stores it in the memory 102. The object imaging process S103 and the object image acquisition process S104 are repeated n times (n is a predetermined natural number). The image captured in the object imaging process S103 and acquired in the object image acquisition process S104 is a color image representing the image of the object O formed on the end face of the image guide 13q on the image guide plug portion 13c side. These images are hereinafter referred to as object images I1, I2, ..., In. Specific examples of the target images I1, I2, ..., In obtained in the target imaging process S103 will be described later with reference to different drawings.

The super-resolution process S105 is a process for generating a super-resolution image I by averaging the pixel values of pixels corresponding to the same point on the object O in the target images I1, I2, ..., In. In this embodiment, the super-resolution process S105 is composed of an alignment process S105a and an averaging process S105b.

The alignment process S105a is a process in which the image processing device 10 (1) calculates the positional deviation vector vi of the target image Ii relative to the target image In for each target image Ii (i is a natural number between 1 and n-1 inclusive) other than the most recently acquired target image In, and (2) shifts the pixel value of each pixel of the target image Ii by the positional deviation vector vi. The positional deviation vector vi can be calculated using, for example, a known method that utilizes the features of the target images Ii and In. The image obtained by the alignment process S105a for the target image Ii is a color image representing the image of the object O formed on the end face of the image guide plug portion 13c side of the image guide 13q. Hereinafter, this image will be referred to as the target image Ii'.

When pixel (xi, yi) of target image Ii and pixel (xn, yn) of target image In correspond to the same point on the subject, the displacement vector vi = (vix, viy) is given by, for example, the following equation:

vix=xi−xn,
viy=yi-yn.

Furthermore, if the pixel value of pixel (x, y) of target image Ii is given by {Ri(x, y), Gi(x, y), Bi(x, y)}, the pixel value of pixel (x, y) of target image Ii' {Ri'(x, y), Gi'(x, y), Bi'(x, y)} is given by, for example, the following formula:

If 0≦x+vix≦W and 0≦y+viy≦H:
Ri'(x,y)=Ri(x+vix,y+viy),
Gi' (x, y) = Gi (x + vix, y + viy),
Bi'(x,y)=Bi(x+vix,y+viy),
If x+vix<0, W<x+vix, y+viy<0, or H<y+viy:
Ri'(x,y)=0,
Gi'(x,y)=0,
Bi'(x,y)=0.

Note that the last acquired target image In is not subjected to the alignment process S105a, but for ease of explanation, the target image In will also be referred to as target image In' below. In other words, the pixel value {Rn'(x,y), Gn'(x,y), Bn'(x,y)} at pixel (x,y) of target image In' is the same as the pixel value {Rn(x,y), Gn(x,y), Bn(x,y)} at pixel (x,y) of target image In. Specific examples of target images I1', I2', ..., In' obtained by the alignment process S105a will be described later with reference to different drawings.

Note that, while the configuration described here is one in which the last acquired target image In is used as the reference image and the pixel values of each target image Ii other than the reference image are shifted, this is not limiting. In other words, a configuration may be adopted in which a target image Ip (p is a predetermined natural number between 1 and n-1) other than the last acquired target image In is used as the reference image and the pixel values of each target image Iq (q is a natural number between 1 and n, different from p) other than the reference image are shifted.

The averaging process S105b is a process in which the image processing device 10 simply averages, for each pixel (x, y), the pixel values {Rj'(x, y), Gj'(x, y), Bj'(x, y)} of each target image Ij' (j is a natural number between 1 and n, inclusive) for each component. The image obtained by the averaging process S105b is a color image representing the image of the target O formed on the end face of the image guide plug portion 13c side of the image guide 13q. This image will be referred to below as the super-resolution image I. The pixel values {R(x, y), G(x, y), B(x, y)} of pixel (x, y) of the super-resolution image I are given, for example, by the following equation: Specific examples of the super-resolution image I generated by the super-resolution process S105 will be described later with reference to different drawings.

R(x,y)={R1'(x,y)+R2'(x,y)+...+Rn'(x,y)}/n,
G(x,y)={G1'(x,y)+G2'(x,y)+...+Gn'(x,y)}/n,
B(x,y)={B1'(x,y)+B2'(x,y)+...+Bn'(x,y)}/n.

Masking processing S106 is processing in which the image processing device 10 generates an output image I' by masking areas outside the field of view circle in the super-resolution image I. Here, masking areas outside the field of view circle refers to, for example, setting the pixel values {R(x,y), G(x,y), B(x,y)} of pixels (x,y) located outside the field of view circle in the super-resolution image I to R(x,y) = 0, G(x,y) = 0, B(x,y) = 0. Specific examples of the output image I' generated in masking processing S106 will be described later with reference to different drawings.

Image display processing S107 is processing in which the image processing device 10 displays the output image I' on the display D. This completes the series of processes in the image processing method S100.

Note that while the image processing method S100 for displaying a still image of the object O on the display D has been described above, this method may also be used to display a moving image of the object O on the display D. In this case, the camera device 12 repeatedly executes the object imaging process S103 at a cycle corresponding to the frame rate, and the image processing device 10 executes the super-resolution process S105, the mask process S106, and the image display process S107 on the most recent n target images of the target images captured by the camera device 12 up to that point. This allows a moving image of the object O to be displayed on the display D.

As pre-processing for the super-resolution process S105, each of the target images I1, I2, ..., In may be subjected to deformation correction such as affine transformation. If differences in the position and shape of the core images contained in the target images I1, I2, ..., In occur due to differences in the tilt angle during image capture, these differences can be eliminated in advance using deformation correction, resulting in a super-resolution image I with higher visibility. As pre-processing for the super-resolution process S105, each of the target images I1, I2, ..., In may be subjected to color and brightness unevenness correction as described in JP 2021-177604 (Patent No. 7189173). If color and brightness unevenness occurs among the core images contained in the target images I1, I2, ..., In, these can be eliminated in advance using color and brightness unevenness correction, resulting in a super-resolution image I with higher visibility.

(Example of image processing method)
An example of the execution of the image processing method S100 will be described with reference to Figures 4 to 11. In this example, for simplicity, the number n of target images I1, I2, ..., In is set to 4.

Figure 4 is a plan view of object O used as a sample in this example. Object O used as a sample in this example is a white standard plate printed with the letters A to Z and the numbers 0 to 9.

Figure 5 is a diagram illustrating a mesh image I0 obtained in the white standard plate imaging process S101. As described above, the mesh image I0 is an image in which the areas corresponding to the cores of the image guide 13q are bright and the other areas are dark.

Figure 6 is a diagram illustrating target image I1 obtained in object imaging process S103, and target image I1' obtained in alignment process S105a. The positions of the letters and alphabets included as subjects in target image I1 are shifted to the bottom right compared to the positions of the letters and alphabets included as subjects in target image I4, which is used as the reference image. Therefore, the positional shift vector v1 points to the bottom right, as indicated by the white arrow in Figure 6. The positions of the letters and alphabets included as subjects in target image I1', which is obtained by shifting the pixel values of each pixel in target image I1 by the positional shift vector v1, match or approximately match the positions of the letters and alphabets included as subjects in target image I4, which is used as the reference image.

Figure 7 is a diagram illustrating target image I2 obtained in object imaging process S103 and target image I2' obtained in alignment process S105a. The positions of the letters and alphabets included as subjects in target image I2 are shifted to the left compared to the positions of the letters and alphabets included as subjects in target image I4, which is used as the reference image. Therefore, the positional shift vector v2 points to the left, as indicated by the white arrow in Figure 7. The positions of the letters and alphabets included as subjects in target image I2', which is obtained by shifting the pixel values of each pixel in target image I2 by the positional shift vector v2, match or approximately match the positions of the letters and alphabets included as subjects in target image I4, which is used as the reference image.

Figure 8 is a diagram illustrating target image I3 obtained in object imaging process S103 and target image I3' obtained in alignment process S105a. The positions of the letters and alphabets included as subjects in target image I3 are shifted to the upper right and to the right of the positions of the letters and alphabets included as subjects in target image I4, which is used as the reference image. Therefore, the positional deviation vector v3 points to the upper right, as indicated by the white arrow in Figure 8. The positions of the letters and alphabets included as subjects in target image I3', which is obtained by shifting the pixel values of each pixel in target image I3 by the positional deviation vector v3, match or approximately match the positions of the letters and alphabets included as subjects in target image I4, which is used as the reference image.

Figure 9 is a diagram showing an example of target image I4 obtained in the target image capture process S103. Since target image I4 is used as a reference image in the alignment process S105a, the alignment process S105a is not performed on target image I4.

Figure 10 is a diagram illustrating a super-resolution image I obtained by super-resolution processing S105. In target images I1 to I4, the alphabet and letters contained as subjects are difficult to read. In contrast, in super-resolution image I, the alphabet and letters contained as subjects are easy to read. In other words, it was confirmed that by performing super-resolution processing S105, a super-resolution image I with higher visibility than target images I1 to I4 was obtained.

Figure 11 is a diagram illustrating an output image I' obtained by masking process S106. In super-resolution image I, jaggies (jagged edges caused by image guide 13q being composed of multiple cores) can be seen at the boundaries of the end faces of image guide 13q, which is included as a subject. On the other hand, in output image I', jaggies cannot be seen at the boundaries of the end faces of image guide 13q, which is included as a subject. In other words, it was confirmed that by performing masking process S106, an output image I' with fewer jaggies than super-resolution image I can be obtained.

(Modification of image processing method)
A modified example of the image processing method S100 will be described with reference to Fig. 12. Fig. 12 is a flow chart showing the flow of an image processing method S100' according to this modified example.

The image processing method S100' according to this modified example (1) adds a white standard plate imaging process S110, a mesh image acquisition process S111, and a transparency process S112 to the image processing method S100 described above, and (2) replaces the averaging process S105b, which takes a simple average, included in the super-resolution process S105 described above with an averaging process S105c, which takes a weighted average.

The white standard plate imaging process S110 and the mesh image acquisition process S111 are processes executed between the white balance adjustment process S102 and the object imaging process S103 described above.

The white standard plate imaging process S110 is a process in which the user faces the objective lens 13r of the endoscopic probe 13 toward the white standard plate, and the camera device 12 images the end face of the image guide plug portion 13c of the image guide 13q. The mesh image acquisition process S111 is a process in which the image processing device 10 acquires the image captured in the white standard plate imaging process S110 from the camera device 12 and stores it in the memory 102. The image captured in the white standard plate imaging process S101 and acquired in the mesh image acquisition process S111 is a color image in which areas corresponding to the cores of the image guide 13q are bright and other areas are dark. The image processing device 10 converts this color image into grayscale for use. The monochrome image obtained by converting this color image into grayscale is hereinafter referred to as mesh image I0'. The difference between mesh image I0 and the mesh image I1 described above is that mesh image I0 is a color image captured before white balance adjustment, while mesh image I0' is a monochrome image obtained by grayscaling a color image captured after white balance adjustment.

The transparency processing S112 and core drawing processing S113 are pre-processing steps for the super-resolution processing S105, which are executed between the target image acquisition processing S104 and the super-resolution processing S105 described above.

The transparency process S112 is a process in which the image processing device 10, for each target image Ij (j is a natural number between 1 and n), references the mesh image I0' captured in the white standard plate capture process S110 and renders transparent or semi-transparent areas of the target image Ij other than the area corresponding to the core of the image guide 13q (i.e., the areas corresponding to the mesh and background in the mesh image I0'). The image generated by the transparency process S112 is an opaque color image with high opacity in the area corresponding to the core of the image guide 13q and low opacity in other areas. This image will be referred to below as the "transparent target image Ij." The pixel value of pixel (x, y) of the transparent target image Ij is, for example, the pixel value {Rj(x, y), Gj(x, y), Bj(x, y)} of pixel (x, y) of the target image Ij plus the pixel value A(x, y) of pixel (x, y) of the mesh image I0' as an opaque component. That is, the pixel value of pixel (x, y) of the transparent target image Ij is {Rj(x, y), Gj(x, y), Bj(x, y), A(x, y)}.

In the alignment process S105a of the super-resolution process S105, for each transparent target image Ii (i is a natural number between 1 and n-1) other than the transparent target image In, the image processing device 10 (1) calculates a positional deviation vector vi of the transparent target image Ii relative to the transparent target image In, and (2) shifts the pixel value of each pixel of the transparent target image Ii by the amount of the positional deviation vector vi. The image obtained in the alignment process S105a for the transparent target image Ii is a color image with opacity that represents the image of the object O formed on the end face of the image guide plug portion 13c side of the image guide 13q. This image will be referred to as the transparent target image Ii' hereinafter. Note that the alignment process S105a is not performed on the transparent target image In, but for convenience of explanation, the transparent target image In will also be referred to as the transparent target image In' hereinafter. Specific examples of the transparent target images I1', I2', ..., In' obtained in the alignment process S105a will be described later with reference to different drawings.

In the averaging process S105c of the super-resolution process S105, the image processing device 10 generates a super-resolution image I by weighting the pixel values {Rj'(x,y), Gj'(x,y), Bj'(x,y), Aj'(x,y)} of each transparent object image Ij' for each pixel (x,y) for each component. The opacity Aj'(x,y) is used as the weight in the weighted average. The image obtained in the averaging process S105c is a color image with opacity representing the image of the object O formed on the end face of the image guide plug portion 13c side of the image guide 13q. This image will be referred to as the super-resolution image I hereinafter. The pixel values {R(x,y), G(x,y), B(x,y), A(x,y)} of the pixel (x,y) of the super-resolution image I are given, for example, by the following equation: Specific examples of the super-resolution image I generated by averaging processing S105c will be described later with reference to different drawings.

D(x,y)=A1'(x,y)+A2'(x,y)+...+An'(x,y),
If D(x, y)≠0:
R(x,y)={R1'(x,y)A1'(x,y)+R2'(x,y)A2'(x,y)+...+Rn'(x,y)An'(x,y)}/D(x,y),
G(x,y)={G1'(x,y)A1'(x,y)+G2'(x,y)A2'(x,y)+...+Gn'(x,y)An'(x,y)}/D(x,y),
B(x,y)={B1'(x,y)A1'(x,y)+B2'(x,y)A2'(x,y)+...+Bn'(x,y)An'(x,y)}/D(x,y),
A (x, y) = [{A1' (x, y)} ² + {A2' (x, y)} ² +...+{An' (x, y)} ² ]/D (x, y),
If D(x,y)=0:
R(x, y)=0,
G(x, y)=0,
B(x, y)=0,
A(x, y) = 0.

(Example of execution of image processing method according to modified example)
An example of execution of the image processing method S100 according to the above-mentioned modified example will be described with reference to Figures 13 to 18. In this example, for simplicity, the number n of target images I1, I2, ..., In is set to 4.

Figure 13 is a diagram illustrating a transparent target image I1 obtained by the transparency processing S112. Figure 14 is a diagram illustrating a transparent target image I2 obtained by the transparency processing S112. Figure 15 is a diagram illustrating a transparent target image I3 obtained by the transparency processing S112. Figure 16 is a diagram illustrating a transparent target image I4 obtained by the transparency processing S112. It can be seen that in each transparent target image Ii, areas other than the area corresponding to the core of the image guide 13q have been made transparent.

Figure 17 is a diagram illustrating a super-resolution image I obtained by super-resolution processing S105. In target images I1 to I4, the alphabet and letters contained as subjects are difficult to read. In contrast, in super-resolution image I, the alphabet and letters contained as subjects are easy to read. In other words, it was confirmed that by performing super-resolution processing S105, a super-resolution image I with higher visibility than target images I1 to I4 was obtained.

Furthermore, comparing super-resolution image I (without transparency processing) shown in Figure 10 with super-resolution image I (with transparency processing) shown in Figure 17, it can be seen that the latter has a higher contrast between the alphabet and letters and their background, and as a result, the visibility of the alphabet and letters is higher in the latter. This is because in each target image Ii, the brightness of areas other than those corresponding to the core of image guide 13q is low, so in super-resolution image I shown in Figure 10, a decrease in brightness within the field of view and image graininess are likely to occur. However, in each transparent target image Ii, the areas other than those corresponding to the core of image guide 13q are made transparent, so a decrease in brightness within the field of view and image graininess are less likely to occur.

Note that a decrease in brightness and image graininess occur, for example, when the pixel values of pixels belonging to low-brightness areas in target image Ij are averaging with the pixel values of pixels belonging to high-brightness areas in another target image Ij', or when the pixel values of pixels belonging to low-brightness areas in target image Ij are averaging with the pixel values of pixels belonging to low-brightness areas in another target image I'.

Figure 18 is a diagram illustrating an output image I' obtained by masking process S106. In super-resolution image I, jaggies (jagged edges caused by image guide 13q being composed of multiple cores) can be seen at the boundaries of the end faces of image guide 13q, which is included as a subject. On the other hand, in output image I', jaggies cannot be seen at the boundaries of the end faces of image guide 13q, which is included as a subject. In other words, it was confirmed that by performing masking process S106, an output image I' with fewer jaggies than super-resolution image I can be obtained.

(Further Modification 1 of Image Processing Method)
As pre-processing for the super-resolution processing S105 described above, the image processing device 10 may perform core drawing processing on each transparent target image Ij.

In the core drawing process, the image processing device 10 first identifies an area in the transparent target image Ij that corresponds to the core of the image guide 13q (hereinafter also referred to as the "core area"). For example, a set of pixels in the mesh image I0' whose pixel values are equal to or greater than a predetermined threshold is identified as the core area.

Next, the image processing device 10 resets the pixel values of each pixel included in the core region in the transparent target image Ij for each of the red, green, and blue components so that they match either (a1) the pixel value of the pixel located at the center of the core region (hereinafter also referred to as the "center pixel value") or (a2) the average value of the pixel values of each pixel included in the core region (hereinafter also referred to as the "average pixel value"). The image processing device 10 also sets the opacity of the pixel located at the center of the core region in the transparent target image Ij to 100%. The image processing device 10 then resets the opacity of each pixel in the transparent target image Ij whose distance r from the center of the core region is equal to or less than a predetermined threshold R so that (b1) the opacity decreases linearly as the distance r increases, or (b2) the opacity decreases according to a Gaussian distribution as the distance r increases. If the opacity is not reset, the opacity of each pixel included in the core region will match the pixel value of the pixel having the same coordinates as that pixel in the mesh image I0', as described above.

(a) in Figure 19 is an enlarged view of the transparent target image Ij and super-resolution image I obtained when the pixel values of each pixel in the core region are reset to the average pixel value for the red, green, and blue components, but the opacity is not reset.

(b) in Figure 19 is an enlarged view of a transparent object image Ij and a super-resolution image I obtained when the pixel values of each pixel included in the core region for the red, green, and blue components are reset to the average pixel value, and the opacity of each pixel whose distance r from the center of the core region is less than a predetermined threshold R is reset to linearly decrease as the distance r increases. Comparing (a) and (b) in Figure 19 shows that resetting the opacity further reduces the decrease in brightness and image graininess in the super-resolution image I. Note that a similar effect can be achieved even when the pixel values of each pixel included in the core region for each of the red, green, and blue components are set to the center pixel value.

(c) of Figure 19 is an enlarged view of a transparent object image Ij and a super-resolution image I obtained when the pixel values of each pixel included in the core region are reset to the average pixel value for the red, green, and blue components, and the opacity of each pixel whose distance r from the center of the core region is equal to or less than a predetermined threshold R is reset to decrease according to a Gaussian distribution as the distance r increases. Comparing (a) and (c) of Figure 19 reveals that resetting the opacity further reduces brightness loss and image graininess in the super-resolution image I. Comparing (b) and (c) of Figure 19 reveals that resetting the opacity so that it decreases according to a Gaussian distribution as the distance r increases is more advantageous in further reducing brightness loss and image graininess in the super-resolution image I than resetting the opacity so that it decreases linearly as the distance r increases. Note that the same effect can be achieved by setting the pixel value of each pixel included in the core region as the central pixel value for each of the red, green, and blue components.

(Further Modification 2 of Image Processing Method)
In the above-mentioned super-resolution processing S105, the image processing device 10 may perform averaging processing S105c on all of the transparent object images I1, I2, ..., In, or may perform averaging processing S105c on only a portion of the transparent object images I1, I2, ..., In.

An example of the latter is a configuration in which transparent object images I1, I2, ..., In other than those with large amounts of positional deviation are subjected to the averaging process S105c. Transparent object images with large amounts of positional deviation have little overlap with the transparent object image In used as the reference image in the alignment process S105a, so their contribution to improving the visibility of the super-resolution image I is limited, and they may even cause noise to occur in the super-resolution image I. Therefore, by excluding transparent object images with large amounts of positional deviation from the averaging process S105c, the occurrence of noise in the super-resolution image I can be suppressed without impeding the effect of improving the visibility of the super-resolution image I.

Note that, among the transparent target images I1, I2, ..., In, a method for selecting a transparent target image with a large amount of positional deviation may include, for example, selecting a transparent target image Ii whose positional deviation vector vi exceeds a predetermined threshold value among the transparent target images I1, I2, ..., In.

Alternatively, a configuration is conceivable in which transparent object images I1, I2, ..., In other than those that are blurred or blurred are subjected to the averaging process S105c. Transparent object images that are significantly blurred or blurred are difficult to accurately align with the alignment process S105a, and the contours of the object O included as a subject are unclear, so their contribution to improving the visibility of the super-resolution image I is limited. Conversely, they may even cause noise to occur in the super-resolution image I. For this reason, by excluding blurred or blurred transparent object images from the averaging process S105c, it is possible to reduce the occurrence of noise in the super-resolution image I without impeding the effect of improving the visibility of the super-resolution image I.

Note that, among the transparent target images I1, I2, ..., In, one method for selecting a blurred or shaky transparent target image Ii is to select a transparent target image Ii for which the difference in positional deviation vector |(vi-1)-(vi)|, |(vi+1)-(vi)| from the adjacent transparent target images Ii-1, Ii+1 exceeds a predetermined threshold. However, the method for selecting a blurred or shaky transparent target image Ii is not limited to this, and any known method can be used.

(Further Modification 3 of Image Processing Method)
As post-processing of the super-resolution processing S105 described above, the image processing device 10 may perform a base image generation process in which a base image is generated by smoothing the last acquired target image In, and an output image generation process in which an output image is generated by superimposing the super-resolution image I on the generated base image. Here, the smoothing method is not particularly limited. The base image may be generated by smoothing using an averaging filter, or may be generated by smoothing using a Gaussian filter.

If the opacity of a pixel is 0 in all transparent target images I1', I2', ..., In', the opacity of that image will also be 0 in super-resolution image I. In other words, that pixel will be transparent in super-resolution image I. When such a super-resolution image I is displayed, the background visible through the transparent pixels may be perceived as showing through, giving an unnatural impression. For example, if an all-black background is used, when such a super-resolution image I is displayed, the background visible through the transparent pixels will be perceived as scattered black dots, giving an unnatural impression. By superimposing super-resolution image I on a base image generated by smoothing target image In, concerns about the background being visible through the transparent pixels can be eliminated, making it possible to display super-resolution image I more naturally.

In this modified example, a configuration has been described in which a base image is generated by smoothing the last acquired target image In, but this is not limiting. In other words, a configuration may be adopted in which a base image is generated by smoothing a target image Ii other than the last acquired target image In among the target images I1, I2, ..., In.

(summary)
This embodiment includes the following aspects.

[Aspect 1]
an acquisition process for acquiring a plurality of target images obtained by imaging one end of an image guide having a plurality of cores and facing the target;
a super-resolution process for generating a super-resolution image by simply averaging or weighted averaging pixel values of pixels corresponding to the same point of the object in the plurality of object images;
Image processing methods.

This aspect makes it possible to obtain a super-resolution image with higher resolution or visibility than the target image.

[Aspect 2]
As a pre-processing of the super-resolution processing, a transparency processing is further included in which an area other than an area corresponding to a core of the image guide in each of the plurality of target images is made transparent or semi-transparent;
The super-resolution processing is a process of generating the super-resolution image by weighting pixel values of pixels corresponding to the same point of the object in the plurality of target images after the preprocessing, using opacity as a weight.
2. The image processing method according to aspect 1.

This aspect of the invention reduces the reduction in brightness and image graininess that can occur in super-resolution images, thereby enabling the production of super-resolution images with higher visibility.

[Aspect 3]
The pre-processing further includes a core drawing process for setting a pixel value of each pixel included in an area corresponding to the core of the image guide in each of the plurality of target images to a pixel value of a pixel located at the center of the area.
3. The image processing method according to aspect 2.

This aspect further reduces the reduction in brightness and image graininess that can occur in super-resolution images, thereby enabling the production of super-resolution images with higher visibility.

Aspect 4
The pre-processing further includes a core drawing process for setting the pixel value of each pixel included in a region corresponding to the core of the image guide in each of the plurality of target images to an average value of each pixel included in the region.
3. The image processing method according to aspect 2.

This aspect further reduces the reduction in brightness and image graininess that can occur in super-resolution images, thereby enabling the production of super-resolution images with higher visibility.

Aspect 5
In the core drawing process, the opacity of each pixel in each of the plurality of target images whose distance from the center of the region is equal to or less than a threshold is set so as to decrease according to a Gaussian distribution as the distance increases.
5. The image processing method according to aspect 3 or 4.

This aspect further reduces the reduction in brightness and image graininess that can occur in super-resolution images, thereby enabling the production of super-resolution images with higher visibility.

Aspect 6
The method further includes, as post-processing of the super-resolution processing, a base image generation process of generating a base image by smoothing any of the plurality of target images, and a superimposition process of generating an output image by superimposing the super-resolution image on the base image.
6. The image processing method according to any one of aspects 2 to 5.

This aspect eliminates the possibility of the background being visible through the super-resolution image, making it possible to obtain a more natural super-resolution image.

Aspect 7
The super-resolution processing is a process of generating the super-resolution image by taking a weighted average of pixel values of pixels corresponding to the same point of the object in a target image determined to have a small positional deviation among the plurality of target images.
7. The image processing method according to any one of aspects 2 to 6.

According to this aspect, by reducing noise that may occur in super-resolution images, it is possible to obtain super-resolution images with higher visibility.

Aspect 8
The super-resolution processing is a process of generating the super-resolution image by taking a weighted average of pixel values of pixels corresponding to the same point of the object in a target image determined to be free of blur or shaking among the plurality of target images.
7. The image processing method according to any one of aspects 2 to 6.

According to this aspect, by reducing noise that may occur in super-resolution images, it is possible to obtain super-resolution images with higher visibility.

Aspect 9
An image processing device comprising at least one processor,
The processor:
an acquisition process for acquiring a plurality of target images obtained by imaging one end of an image guide having a plurality of cores and facing the target;
and performing a super-resolution process to generate a super-resolution image by averaging or weighted averaging pixel values of pixels corresponding to the same point of the object in the plurality of object images.
Image processing device.

This aspect makes it possible to obtain a super-resolution image with higher resolution or visibility than the target image.

Aspect 10
The processor further performs a transparency process as a preprocessing of the super-resolution process, which makes transparent or semi-transparent an area other than an area corresponding to a core of the image guide in each of the plurality of target images;
The super-resolution processing is a process of generating the super-resolution image by weighting pixel values of pixels corresponding to the same point of the object in the plurality of target images after the preprocessing, using opacity as a weight.
10. The image processing device according to aspect 9.

This aspect of the invention reduces the reduction in brightness and image graininess that can occur in super-resolution images, thereby enabling the production of super-resolution images with higher visibility.

Aspect 11
The processor further performs, as the pre-processing, a core drawing process of setting a pixel value of each pixel included in an area corresponding to a core of the image guide in each of the plurality of target images to a pixel value of a pixel located at the center of the area.
11. The image processing device according to aspect 10.

This aspect further reduces the reduction in brightness and image graininess that can occur in super-resolution images, thereby enabling the production of super-resolution images with higher visibility.

Aspect 12
An image processing program for operating at least one processor, the program comprising:
an acquisition process for acquiring a plurality of target images obtained by imaging one end of an image guide having a plurality of cores and facing the target;
a super-resolution process for generating a super-resolution image by averaging or weighted averaging pixel values of pixels corresponding to the same point of the object in the plurality of object images;
Image processing program.

This aspect makes it possible to obtain a super-resolution image with higher resolution or visibility than the target image.

Aspect 13
The processor further executes a transparency process as a preprocessing of the super-resolution process, which makes transparent or semi-transparent an area other than an area corresponding to a core of the image guide in each of the plurality of target images;
The super-resolution processing is a process of generating the super-resolution image by weighting pixel values of pixels corresponding to the same point of the object in the plurality of target images after the preprocessing, using opacity as a weight.
13. The image processing program according to aspect 12.

This aspect further reduces the reduction in brightness and image graininess that can occur in super-resolution images, thereby enabling the production of super-resolution images with higher visibility.

Aspect 14
and causing the processor to further execute, as the pre-processing, a core drawing process of setting a pixel value of each pixel included in an area corresponding to a core of the image guide in each of the plurality of target images to a pixel value of a pixel located at the center of the area.
14. The image processing program according to aspect 13.

This aspect further reduces the reduction in brightness and image graininess that can occur in super-resolution images, thereby enabling the production of super-resolution images with higher visibility.

Aspect 15
The image processing device according to any one of aspects 9 to 11,
an endoscopic probe including the image guide,
Endoscopy system.

This aspect makes it possible to realize an endoscope system that can obtain super-resolution images with higher resolution or visibility than the target image.

(Additional Notes)
The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in the above-described embodiments are also included in the technical scope of the present invention.

REFERENCE SIGNS LIST 1 Endoscope system 10 Image processing device 101 Processor 102 Memory 103 Storage 104 Input/output interface 105 Bus 11 Light source device 12 Camera device 13 Endoscope probe 13p Light guide 13q Image guide

Claims (15)

an acquisition process for acquiring a plurality of target images obtained by imaging one end of an image guide having a plurality of cores and facing the target;
a super-resolution process for generating a super-resolution image by simply averaging or weighted averaging pixel values of pixels corresponding to the same point of the object in the plurality of object images;
Image processing methods. As a pre-processing of the super-resolution processing, a transparency processing is further included in which an area other than an area corresponding to a core of the image guide in each of the plurality of target images is made transparent or semi-transparent;
The super-resolution processing is a process of generating the super-resolution image by weighting pixel values of pixels corresponding to the same point of the object in the plurality of target images after the preprocessing, using opacity as a weight.
The image processing method according to claim 1 . The pre-processing further includes a core drawing process for setting a pixel value of each pixel included in an area corresponding to the core of the image guide in each of the plurality of target images to a pixel value of a pixel located at the center of the area.
The image processing method according to claim 2 . The pre-processing further includes a core drawing process for setting the pixel value of each pixel included in a region corresponding to the core of the image guide in each of the plurality of target images to an average value of each pixel included in the region.
The image processing method according to claim 2 . In the core drawing process, the opacity of each pixel in each of the plurality of target images whose distance from the center of the region is equal to or less than a threshold is set so as to decrease according to a Gaussian distribution as the distance increases.
5. The image processing method according to claim 3 or 4. The method further includes, as post-processing of the super-resolution processing, a base image generation process of generating a base image by smoothing any of the plurality of target images, and a superimposition process of generating an output image by superimposing the super-resolution image on the base image.
The image processing method according to any one of claims 2 to 4. The super-resolution processing is a process of generating the super-resolution image by taking a weighted average of pixel values of pixels corresponding to the same point of the object in a target image determined to have a small positional deviation among the plurality of target images.
The image processing method according to any one of claims 2 to 4. The super-resolution processing is a process of generating the super-resolution image by taking a weighted average of pixel values of pixels corresponding to the same point of the object in a target image determined to be free of blur or shaking among the plurality of target images.
The image processing method according to any one of claims 2 to 4. An image processing device comprising at least one processor,
The processor:
an acquisition process for acquiring a plurality of target images obtained by imaging one end of an image guide having a plurality of cores and facing the target;
and performing a super-resolution process to generate a super-resolution image by averaging or weighted averaging pixel values of pixels corresponding to the same point of the object in the plurality of object images.
Image processing device. The processor further performs a transparency process as a preprocessing of the super-resolution process, which makes transparent or semi-transparent an area other than an area corresponding to a core of the image guide in each of the plurality of target images;
The super-resolution processing is a process of generating the super-resolution image by weighting pixel values of pixels corresponding to the same point of the object in the plurality of target images after the preprocessing, using opacity as a weight.
The image processing device according to claim 9 . The processor further performs, as the pre-processing, a core drawing process of setting a pixel value of each pixel included in an area corresponding to a core of the image guide in each of the plurality of target images to a pixel value of a pixel located at the center of the area.
The image processing device according to claim 10. An image processing program for operating at least one processor, the program comprising:
an acquisition process for acquiring a plurality of target images obtained by imaging one end of an image guide having a plurality of cores and facing the target;
a super-resolution process for generating a super-resolution image by averaging or weighted averaging pixel values of pixels corresponding to the same point of the object in the plurality of object images;
Image processing program. The processor further executes a transparency process as a preprocessing of the super-resolution process, which makes transparent or semi-transparent an area other than an area corresponding to a core of the image guide in each of the plurality of target images;
The super-resolution processing is a process of generating the super-resolution image by weighting pixel values of pixels corresponding to the same point of the object in the plurality of target images after the preprocessing, using opacity as a weight.
The image processing program according to claim 12. and causing the processor to further execute, as the pre-processing, a core drawing process of setting a pixel value of each pixel included in an area corresponding to a core of the image guide in each of the plurality of target images to a pixel value of a pixel located at the center of the area.
The image processing program according to claim 13. An image processing device according to any one of claims 9 to 11;
an endoscopic probe including the image guide,
Endoscopy system.