JP7544358B2 - Imaging device, inspection device, and imaging method - Google Patents

Description

The present invention relates to an imaging device for imaging a subject, an inspection device, and an imaging method.

For example, Patent Documents 1 to 3 disclose imaging elements or camera modules capable of capturing both visible light and near-infrared light. For example, Patent Document 1 discloses a camera module having a structure in which a filter section with IR-cutting properties and a structure that also functions as an aperture is arranged in front of the lens. Patent Document 2 discloses a solid-state imaging device in which a pixel sensor for detecting a color image of a document to be read and a pixel sensor for detecting near-infrared light recorded as invisible information are formed on the same substrate.

Patent document 3 discloses an imaging device that outputs a color image signal, an infrared image signal, or a mixed image signal that combines both. The imaging device captures an optical image representing a subject that includes infrared light using a photographing lens, captures the infrared image (near-infrared optical image) and the visible light image (visible optical image) in the imaging section without separating them, and performs predetermined signal processing after converting the infrared image and the visible light image into video signals by an imaging signal processing section. The imaging section has a color filter group including three filters for visible light and one filter for near-infrared light, and a solid-state imaging element including three pixels that detect visible light and one pixel that detects near-infrared light.

JP 2002-182270 A Japanese Patent Application Publication No. 10-65135 JP 2007-53731 A

However, devices that output color image signals, infrared image signals, or mixed image signals that combine both have the problem that they require the use of special imaging elements and special cameras that can capture visible light images and near-infrared images. In addition, when image processing is required for color images in visible light and black-and-white images in near-infrared light in image recognition and inspection of a subject, two sets of imaging systems are usually required: a color camera and white lighting, and a near-infrared black-and-white camera and near-infrared lighting. In addition, in the case of the above-mentioned two-camera configuration, the images obtained from each camera have different subject positions, so in order to share and process the images from the two cameras, processing such as position and angle of view correction is required. Note that the special imaging elements (special image sensors) and special cameras that can capture visible light and near-infrared images, as described in Patent Documents 1 to 3, are expensive and have very little freedom in selecting specifications.

The object of the present invention is to provide an imaging device and imaging method that can acquire visible light images and near-infrared images of a subject captured at the same angle of view with a simple configuration. It is also an object of the present invention to provide an inspection device that inspects a subject based on visible light images and near-infrared images acquired by the imaging device.

The means for solving the above problems and their effects will be described below.
An imaging device that solves the above problem includes a camera having an image sensor having N-band (N is a natural number greater than or equal to 3) spectral optical filters with different spectral transmittance characteristics and sensitive to the visible light region and near-infrared region, an optical bandpass filter arranged on an optical path between the image sensor and a subject, a white light source that irradiates the subject with white light having an emission spectrum in the visible light region, a near-infrared light source that irradiates the subject with near-infrared light that has an emission spectrum in the near-infrared light region and does not have an emission spectrum in the emission spectrum region of the white light source, and a conversion unit that separates image signals captured by the image sensor into signals of the N bands and performs a matrix operation on the separated N-band image signals to generate a first image signal in a P-band (P is a natural number less than N) having spectral sensitivity in the visible light region and a second image signal in a Q-band (Q is Q=N-P) having spectral sensitivity in the near-infrared region.

According to this configuration, a visible light image and a near-infrared image of a subject captured at the same angle of view can be obtained with a simple configuration.
In the imaging device, the camera may be a general-purpose color camera from which an infrared light blocking filter is removed.

According to this configuration, since a general-purpose color camera is used as the camera, the configuration of the imaging device can be simple.
In the above imaging device, for at least one of the image signals having multiple bands out of the P band and the Q band, a color image signal may be generated using pseudo-color based on the image signal having the multiple bands.

According to this configuration, since image signals of multiple bands can be displayed as a color image using pseudo colors, it is easy for an inspector to look at the image and check the inspection results, for example.
In the above imaging device, the surface of the subject facing the camera is defined as the front surface, and the surface opposite the front surface is defined as the back surface. The white light source may illuminate the front surface of the subject with white light, the near-infrared light source may irradiate near-infrared light from the back surface of the subject, and the camera may receive light of the white light reflected by the surface of the subject and near-infrared light that has passed through the subject.

According to this configuration, the imaging device can obtain an image of the surface of the subject and a transmitted light image of near-infrared light transmitted through the subject.
In the above imaging device, an optical filter member having a surface that forms a background of the subject is disposed between the near-infrared light source and the subject, and the optical filter member may reflect light of at least a specific wavelength in the visible light range of the white light irradiated from the white light source to the surface or absorb light in the visible light range, and transmit near-infrared light irradiated from the near-infrared light source to a back surface opposite the surface.

According to this configuration, the background color when the subject is imaged can be selected, so that visible light images and near-infrared images suitable for inspections, etc. can be provided.
In the above imaging device, the white light source and the near-infrared light source may be configured by a single light source that irradiates the white light and the near-infrared light toward the front, which is the surface of the subject facing the camera, and the light source may be configured in an array that mixes a plurality of light-emitting units that irradiate white light and a plurality of light-emitting units that irradiate near-infrared light.

According to this configuration, a visible light image and a near-infrared image of the front of the subject can be obtained.
An inspection apparatus that solves the above problem includes the above imaging device, and an inspection processing unit that inspects the subject based on the first image signal in the P band having spectral sensitivity in the visible light region output by the imaging device, and the second image signal in the Q band having spectral sensitivity in the near-infrared region.

According to this configuration, the subject can be inspected using both visible light images and near-infrared images, making it possible to inspect the subject in a way that cannot be achieved by visible light inspection alone.
The imaging method for solving the above problem is an imaging method for capturing an image of a subject with a camera to generate an image signal, the imaging method comprising: a light irradiation step in which a white light source irradiates the subject with white light having an emission spectrum in the visible light region and a near-infrared light source irradiates the subject with near-infrared light having an emission spectrum in the near-infrared light region and not in the emission spectrum region of the white light source; an imaging step in which a camera having N bands (where N is a natural number of 3 or more) of spectral optical filters with different spectral transmittance characteristics and an image sensor having sensitivity in the visible light region and the near-infrared region captures the subject through an optical bandpass filter arranged on an optical path between the image sensor and the subject; and a conversion step in which the image signal captured by the image sensor is separated into signals of the N bands and a matrix operation is performed on the separated N band image signals to generate a first image signal of a P band (where P is a natural number less than N) having spectral sensitivity in the visible light region and a second image signal of a Q band (where Q is Q=N-P) having spectral sensitivity in the near-infrared region. According to this method, the same effects as those of the imaging device can be obtained.

According to the present invention, it is possible to obtain visible light images and near-infrared images of a subject captured at the same angle of view with a simple configuration.

FIG. 1 is a schematic diagram showing an inspection system including an inspection device according to an embodiment. FIG. 1 is a schematic diagram showing a first configuration of an imaging device. FIG. 13 is a schematic diagram showing a second configuration of the imaging device. 1A is a diagram showing the configuration of a general-purpose color camera and a graph showing the relationship between wavelength and relative sensitivity, and FIG. 1B is a diagram showing the configuration of a color camera without an IR cut filter and a graph showing the relationship between wavelength and relative sensitivity. 3A and 3B are a diagram showing the configuration of a camera according to a first embodiment and a graph showing the relationship between wavelength and relative sensitivity. 4A is a graph showing the light transmittance characteristics of the optical bandpass filter F1, and FIG. 4B is a graph showing the relative sensitivity for each color of the image sensor. 6 is a graph showing relative sensitivity for each band of a second imaging signal obtained by performing a matrix operation on a first imaging signal for each color of the image sensor; 1A is a graph showing the emission spectrum of a white light source and a near-infrared light source combined, and FIG. 1B is a graph showing the relative sensitivity for each color of an image sensor via an optical bandpass filter. 13A is a graph showing the relative sensitivity for each color of an image sensor using a white light source and a near-infrared light source as light sources, and FIG. 13B is a graph showing the relative sensitivity for each band of a second imaging signal. 13 is a diagram showing the configuration of a camera according to a second embodiment, and a graph showing the relationship between wavelength and relative sensitivity. 4A is a graph showing the light transmittance characteristics of the optical bandpass filter F2, and FIG. 4B is a graph showing the relative sensitivity for each color of the image sensor. 6 is a graph showing relative sensitivity for each band of a second imaging signal obtained by performing a matrix operation on a first imaging signal for each color of the image sensor; 1A is a graph showing the combined emission spectrum of a white light source and a near-infrared light source, and FIG. 1B is a graph showing the relative sensitivity for each color of an image sensor via an optical bandpass filter. 13A is a graph showing the relative sensitivity for each color of an image sensor using a white light source and a near-infrared light source as light sources, and FIG. 13B is a graph showing the relative sensitivity for each band of a second imaging signal. FIG. 2 is a block diagram showing the functional configuration of an inspection apparatus. (a) is a two-channel pseudo-color image, and (b) is a near-infrared image. FIG. 11 is a schematic diagram showing a configuration of an imaging device according to a modified example. 10A and 10B are graphs showing the relative sensitivity of the second imaging signal for each band in a modified example.

An inspection system including an imaging device will be described below with reference to the drawings.
The inspection system 10 shown in FIG. 1 inspects the quality of the object 12 using an image of the object 12 as a subject. The inspection system 10 includes a conveying device 13 to which the object 12 is conveyed, and an inspection device 11 that inspects the quality of the object 12 based on an image captured by a camera 30 of the object 12 conveyed by the conveying device 13. The inspection device 11 includes an imaging device 15 that generates a multi-band multispectral image, and an inspection processing unit 70 that performs inspection using the multispectral image. The imaging device 15 includes a camera 30 that captures an image of the object 12 as an example of a subject conveyed by the conveying device 13, and a control processing unit 40 electrically connected to the camera 30. At least a part of the control processing unit 40 is configured by a computer. The computer includes an input device and a display unit. In this embodiment, the conveying device 13 is driven by a control unit of a conveying system that is communicatively connected to a control system of the inspection device 11. The control processing unit 40 may control the conveying device 13.

As shown in FIG. 1, the transport device 13 includes a conveyor 16 for transporting an object 12, which is an example of a subject, a sensor 17 for detecting the object 12 placed on the conveyor 16 and transported, and a discharge device (not shown) for removing the object 12 from the line of non-defective items if it is determined to be a defective item based on the inspection results of the inspection device 11. The conveyor 16 may be a belt conveyor, a roller conveyor, etc., as long as it can transport the object 12, or may be a device that grasps and transports the object 12 or transports the object 12 in a suspended state. The discharge device may be configured to push out the object 12 and remove it, or to blow the object 12 away with the force of air and remove it.

The inspection device 11 includes an imaging device 15 that generates a multi-band multispectral image, an inspection processing unit 70 that inspects the quality of the item 12 using the multi-band multispectral image, and a display unit 41 that displays the multispectral image and the inspection results. The inspection device 11 inspects the item 12 based on an image obtained by signal processing a multi-band second imaging signal S2 output when the imaging device 15 images the item 12, which is the subject of the inspection target. The display unit 41 may be a monitor connected to a computer or a display provided on an operation panel.

The imaging device 15 includes a white light source 20 that irradiates the object 12 with white light having an emission spectrum in the visible light region, a near-infrared light source 25 that irradiates the object 12 with near-infrared light having an emission spectrum in the near-infrared light region, a camera 30 that images the object 12, and a control processing unit 40. The control processing unit 40 includes a control unit 50 that controls the white light source 20, the near-infrared light source 25, and the camera 30, a conversion unit 60 that converts the image captured by the camera 30 into a multi-band multispectral image, and a signal processing unit 65 that performs signal processing on the second imaging signal S2 from the conversion unit 60. The conversion unit 60 converts the first imaging signal S1 obtained by imaging the camera 30 into a second imaging signal S2 representing a multi-band multispectral image. In this example, the second imaging signal S2 includes a three-band multispectral image. The signal processing unit 65 converts two bands of the second imaging signal S2 into a first image signal IS1 expressed in pseudo-color, and converts one band into a second image signal IS2 with adjusted brightness.

The inspection processing unit 70 inspects the quality of the item 12 using the 2-band first image signal IS1 and 1-band second image signal IS2 generated by the conversion unit 60 and signal processing unit 65 constituting the imaging device 15, and displays the 2-band first image IS1 and 1-band second image IS2 used in the inspection and the inspection results on the display unit 41. In this case, the first image IS1 and the second image IS2 may be superimposed with an emphasis on the inspection results. Furthermore, the imaging device 15 may display the 2-band first image IS1 and 1-band second image IS2 generated by the conversion unit 60 and signal processing unit 65 on the display unit 41 as the imaging results.

As shown in Figs. 1 to 3, the camera 30 includes an optical bandpass filter 31, a lens 32, and a color image sensor 33 (hereinafter simply referred to as "image sensor 33"). The optical bandpass filter 31 is disposed on the optical path between the object 12, which is the subject, and the image sensor 33. In the example shown in Fig. 1, the optical bandpass filter 31 is disposed between the object 12, which is the subject, and the lens 32, but it may also be disposed between the lens 32 and the image sensor 33.

The image sensor 33 receives an image of the object 12, which is the subject, through the optical bandpass filter 31 and the lens 32, and outputs a first imaging signal S1 according to the light reception result. The first imaging signal S1 output by the image sensor 33 is input to the conversion unit 60. The conversion unit 60 converts the first imaging signal S1 into a second imaging signal S2 that represents a multi-band multispectral image.

The conversion unit 60 separates the image signal captured by the image sensor 33 into N-band (where N is a natural number equal to or greater than 3) signals, and performs a matrix operation on the separated N-band image signals to generate a first image signal in a P-band (where P is a natural number less than N) that has spectral sensitivity in the visible light region, and a second image signal in a Q-band (where Q is Q=N-P) that has spectral sensitivity in the near-infrared region.

The signal processing unit 65 converts M bands (where M is a natural number less than N and equal to or greater than 2) of the second imaging signal S2 representing a multispectral image of N bands (where N is a natural number equal to or greater than 3) into a first image signal IS1 expressed in pseudo-color. Furthermore, the signal processing unit 65 adjusts the luminance if there is one remaining band in the second imaging signal S2, and performs processing to express the image in pseudo-color if there are two or more remaining bands. In this way, the signal processing unit 65 converts the second imaging signal S2 into a first image signal IS1 and a second image signal IS2. In this embodiment, since there are three N bands, the signal processing unit 65 converts the remaining one band into a second image signal IS2 by adjusting the luminance.

The inspection processing unit 70 shown in FIG. 1 inspects the quality of the object 12, which is the subject, using the first image signal IS1 and the second image signal IS2 converted by the conversion unit 60 and the signal processing unit 65. In this example, the first image signal IS1 is a two-channel visible light image, and the second image signal IS2 is a near-infrared image. The inspection processing unit 70 inspects the object 12 using two-channel visible light images and near-infrared images captured simultaneously by one camera 30 for one object 12. Since the two-channel visible light images and near-infrared images are captured simultaneously by one camera 30, the imaging timing and angle of view are also the same.

In the imaging device 15 of this embodiment, two types of light sources, a white light source 20 and a near-infrared light source 25, are prepared in order to simultaneously perform visible light imaging and near-infrared imaging. The following three conditions are satisfied as light source characteristic conditions for the two types of light sources 20, 25. That is, (a) the light emission characteristics of the two light sources 20, 25 do not overlap, (b) one light source has light emission characteristics in the visible light region, and (c) the other light source has light emission characteristics in the near-infrared light region. Note that since the white light source 20 is made of a white LED, the white light irradiated from the white light source 20 does not include near-infrared light.

The white light source 20 shown in FIG. 1 is a light source having an emission spectrum in the visible light region. The white light source 20 includes a plurality of light-emitting units 21 to 23 having different emission spectra in the visible light wavelength region VA. The white light source 20 in this example includes a first light-emitting unit 21, a second light-emitting unit 22, and a third light-emitting unit 23. The white light source 20 is configured by combining a plurality of light-emitting units 21 to 23 having different emission spectra. The plurality of light-emitting units 21 to 23 are configured, for example, by LEDs. In this example, the first light-emitting unit 21, the second light-emitting unit 22, and the third light-emitting unit 23 are respectively composed of a red LED, a green LED, and a blue LED. In other words, the white light source 20 irradiates white light having an emission spectrum in the red wavelength region, an emission spectrum in the green wavelength region, and an emission spectrum in the blue wavelength region. The control unit 50 causes the plurality of light-emitting units 21 to 23 to emit light, thereby irradiating the white light toward the object 12, which is the subject of the image. The white light source 20 may be a single light-emitting unit.

The near-infrared light source 25 irradiates near-infrared light having an emission spectrum in the near-infrared wavelength range NIRA and not in the emission spectrum range of the white light source 20 (see FIG. 8). In the example shown in FIG. 1, the near-infrared light source 25 includes a first light-emitting unit 26 and a second light-emitting unit 27. The near-infrared light source 25 is configured by combining multiple types of light-emitting units 26, 27 each having a different emission spectrum in the near-infrared wavelength range NIRA. The multiple types of light-emitting units 26, 27 are configured, for example, by LEDs. Note that the near-infrared light source 25 may be configured by one type of LED having an emission spectrum in the near-infrared light range and not in the emission spectrum range of the white light source 20.

The combined emission spectrum LS (see FIG. 8) of the white light source 20 and the near-infrared light source 25 has multiple peaks in the transmission wavelength range of the optical bandpass filter 31. Here, the graph shown in FIG. 6(a) shows the transmission wavelength range of the optical bandpass filter 31 (F1). Also, the graph shown in FIG. 11(a) shows the transmission wavelength range of the optical bandpass filter 31 (F2). The graph shown in FIG. 8(a) shows the combined emission spectrum LS of the white light source 20 and the near-infrared light source 25. As can be seen from the graph shown in FIG. 8(a), the combined emission spectrum LS of the white light source 20 and the near-infrared light source 25 has two peaks that overlap in the visible light wavelength range of about 430 to 720 nm, and one peak in the near-infrared wavelength range of about 880 to 1000 nm. As can be seen from the graph in FIG. 6(a), the transmission wavelength range of the optical bandpass filter F1 is in the visible light wavelength range of about 510 to 570 nm, the visible light wavelength range of about 600 to 720 nm, and the near-infrared wavelength range of about 850 to 1000 nm or more. Therefore, the emission spectrum LS combining the white light source 20 and the near-infrared light source 25 has a peak in the emission spectrum of the white light source 20 in the visible light wavelength range of about 510 to 570 nm within the transmission wavelength range of the optical bandpass filter 31, and has a peak in the emission spectrum of the near-infrared light source 25 in the transmission wavelength range of about 850 to 1000 nm or more.

The imaging device 15 shown in FIG. 1 has a color filter 34 (see FIG. 5) as an example of a spectral optical filter having N bands (where N is a natural number of 3 or more) with different spectral transmittance characteristics on a pixel-by-pixel basis, and is equipped with an image sensor 33 that is sensitive to the visible light region and the near-infrared region. The color filter 34 is configured by arranging an R filter 34R, a G filter 34G, and a B filter 34B, each having a plurality of different transmittance characteristics, on a pixel-by-pixel basis.

The conversion unit 60 separates the image signal S1 captured by the image sensor 33 into N-band signals. The conversion unit 60 then performs a matrix operation on the separated N-band image signals to generate a first image signal MS1 in a P-band (where P is a natural number less than N) that has spectral sensitivity in the visible light region, and a second image signal MS2 in a Q-band (where Q is Q=N-P) that has spectral sensitivity in the near-infrared region.

The imaging device 15 employs either a first configuration shown in FIG. 2 or a second configuration shown in FIG.
As shown in FIG. 2, the white light source 20 irradiates the surface of the object 12, which is the subject. The camera 30 captures the visible light reflected from the white light irradiated to the front of the object. The near-infrared light source 25 is disposed on the opposite side of the object 12, which is the subject, with respect to the camera 30. The camera 30 captures the transmitted light of the near-infrared light irradiated from the rear of the object 12 by the near-infrared light source 25 and transmitted through the object 12. That is, the camera 30 captures a visible light image, which is an image of visible light reflected from the surface of the object, and a near-infrared image (transmitted image), which is an image of near-infrared light transmitted through the object 12. The surface of the near-infrared light source 25 facing the object 12 exhibits a single predetermined color, and serves as the background color when the object 12 is imaged.

As shown in FIG. 3, an optical filter member 29 is disposed between the object 12, which is the subject, and the near-infrared light source 25. Even if the surface of the near-infrared light source 25 is a predetermined monochromatic color, it is not possible to select that monochromatic color as the background color when imaging the object 12. For this reason, the optical filter member 29 is disposed between the object 12 and the near-infrared light source 25. The surface of the optical filter member 29 facing the object 12 exhibits a monochromatic color that is the desired background color. For this reason, an image suitable for inspection can be captured in which the object 12 stands out from the background.

The optical filter member 29 reflects at least a specific wavelength of the visible light range of the white light irradiated onto the front surface from the white light source 20, or absorbs light in the visible light range, and transmits near-infrared light irradiated from the near-infrared light source 25 onto the back surface, which is the surface opposite the front surface.

The optical filter member 29 is made of an optical filter or film that transmits at least infrared light from the back surface and has the property of reflecting light of a specific wavelength in the illumination light from the white light source 20 from the front surface. The optical filter member 29 is, for example, a visible light cutting member. Specific configurations of the optical filter member 29 made of a visible light cutting member include a configuration of only a visible light cutting film, and a configuration of a visible light cutting film and a film.

The optical filter member 29 has a low transmittance of visible light and a reflectance in the range of 90 to 100%, and has a transmittance of infrared light in the range of 90 to 100%.
The optical filter member 29 reflects only visible light of a specific wavelength in the visible light region from the white light from the white light source 20, and the background is a specific color so that the light source for illumination and the illumination jig are not reflected in the background of the subject, the article 12. Examples of the specific color include single colors such as red, blue, and green. The specific color of the background may be white, which reflects light of all wavelengths of visible light, or black, which does not reflect (absorbs) all wavelengths.

The surface of the optical filter member 29 may reflect at least light of a specific wavelength. For example, the surface of the optical filter member 29 may be white. That is, the surface of the optical filter member 29 may reflect light of all wavelengths in the visible light range of the white light emitted by the white light source 20. The surface of the optical filter member 29 may be black. That is, the surface of the optical filter member 29 may have the property of absorbing light of all wavelengths in the visible light range.

Next, the configuration of the image sensor 33 provided in the camera 30 will be described with reference to Figures 4(a) and (b). Figure 4(a) shows a general-purpose color camera 200 that captures RGB images. The color camera 200 includes a lens 32 attached to the lens barrel 30a, a near-infrared light cut filter 201 (hereinafter also referred to as the IR cut filter 201) that blocks near-infrared light, and an image sensor 33. The image sensor 33 includes an R light receiving element 33R that receives red light transmitted through the R filter 34R and outputs an R image signal according to the amount of received light, a G light receiving element 33G that receives green light transmitted through the G filter 34G and outputs a G image signal according to the amount of received light, and a B light receiving element 33B that receives blue light transmitted through the B filter 34B and outputs a B image signal according to the amount of received light. In the image sensor 33, the R light receiving element 33R, the G light receiving element 33G, and the B light receiving element 33B are arranged in a predetermined array.

This image sensor 33 has RGB imaging characteristics with near-infrared light cut off. The R light receiving element 33R, the G light receiving element 33G, and the B light receiving element 33B are sensitive to light in the respective wavelength bands shown in the graph of FIG. 4(a). In this graph, the horizontal axis indicates wavelength and the vertical axis indicates relative sensitivity. The R light receiving element 33R is highly sensitive to light in the red (R) wavelength band shown in the graph of FIG. 4(a). The G light receiving element 33G is highly sensitive to light in the green (G) wavelength band shown in the graph of FIG. 4(a). The B light receiving element 33B is highly sensitive to light in the blue (B) wavelength band shown in the graph of FIG. 4(a).

Figure 4(b) shows a color camera 250 in which the IR cut filter 201 has been removed from the general-purpose color camera 200 shown in Figure 4(a). The image sensor 33 built into the color camera 250 does not cut near-infrared light and has RGB imaging characteristics that include the wavelength band of near-infrared light. The R light receiving element 33R, the G light receiving element 33G, and the B light receiving element 33B are sensitive to light in the visible light wavelength region VA and the near-infrared wavelength region NIRA (particularly the near-infrared wavelength region) shown in the graph in Figure 4(b).

Figure 5 shows a schematic configuration of the camera 30 of this embodiment. As shown in Figure 5, the camera 30 has an optical bandpass filter 31 on the optical path between the image sensor 33 and the object 12, which is the subject. The camera 30 does not have the IR cut filter 201 shown in Figure 4 (a). The image sensor 33 has the same configuration as that of the general-purpose color camera 200 shown in Figure 4 (a).

As shown in the graph of FIG. 4(b), the image sensor 33 itself has sensitivity in the visible light wavelength region VA and the near-infrared wavelength region NIRA. The camera 30 is a general-purpose color camera from which the infrared cut filter has been removed. In detail, the camera 30 is configured, for example, by removing the IR cut filter 201 from the general-purpose color camera 200 shown in FIG. 4(a) and attaching an optical bandpass filter 31 to the optical path. Note that the camera 30 is not limited to a configuration based on the general-purpose color camera 200.

The color filter 34 constituting the image sensor 33 is an RGB primary color filter, but may also be a complementary color filter of Mg, Ye, or Cy. In addition to the RGB or complementary color filters, an NIR filter that selectively transmits near-infrared light may also be mixed. Furthermore, the RGB filters may be R, G1, G2, and B filters, or the color filter 34 may be a combination of a complementary color filter and a primary color filter. Furthermore, the filters to be combined may be three or more types.

The optical bandpass filter 31 has a spectral transmittance characteristic having one or more blocking regions in the visible light wavelength region VA and one or more transmission regions in the near-infrared wavelength region NIRA. In order to obtain such a spectral transmittance characteristic, the optical bandpass filter 31 is composed of one or two optical bandpass filters 31A and 31B as shown in FIG. 5. The optical bandpass filter 31 shown in FIG. 5 has one or more transmission regions in the near-infrared wavelength region NIRA, so that the light of the non-visible light wavelength transmitted through the optical bandpass filter 31 from the object 12, which is the subject, is received by the image sensor 33 shown in FIG. The optical bandpass filter 31 of this embodiment transmits at least light of a part of the wavelength band in the near-infrared wavelength region NIRA. The near-infrared wavelength region NIRA is a region with wavelengths of about 800 to about 2500 nm. Although one optical bandpass filter 31 may be made up of three or more filters, the light transmittance decreases and it is necessary to increase the amount of light from the white light source 20, so from the viewpoint of power saving, it is preferable to have a small number of filters.

The optical bandpass filter 31 has a spectral transmittance characteristic suitable for the inspection depending on the difference in the spectral reflectance characteristic of the object to be identified in the subject to be inspected. In this embodiment, the imaging device 15 of the first embodiment in which the optical bandpass filter 31 is the optical bandpass filter F1 (FIG. 6(a)) and the imaging device 15 of the second embodiment in which the optical bandpass filter 31 is the optical bandpass filter F2 (FIG. 11(a)) are shown as examples.

The first and second embodiments will be described below.
(First embodiment)
First, the first embodiment will be described with reference to Fig. 5 to Fig. 9. The optical bandpass filter F1 shown in Fig. 6(a) has a spectral transmittance characteristic having a blocking region with a transmittance of 10% or less in a plurality of wavelength bands, and a transmission region with a transmittance of 70% or more in a plurality of wavelength bands. In other words, the optical bandpass filter F1 has a spectral transmittance characteristic having a blocking region in the visible light wavelength region VA, and a transmission region in the near-infrared wavelength region NIRA.

Thus, even if the image sensor 33 itself has the relative sensitivity of the three bands of RGB shown in FIG. 6(b), the image sensor 33 has the substantial relative sensitivity shown in the graph of FIG. 5 due to the spectral transmittance characteristics of the optical bandpass filter F1 shown in FIG. 6(a).

The R light receiving element 33R constituting the image sensor 33 shown in FIG. 5 has a sensitivity in the band indicated by R in the graph of the same figure, the G light receiving element 33G has a sensitivity in the band indicated by G, and the B light receiving element 33B has a sensitivity in the band indicated by B. Each light receiving element 33R, 33G, 33B receives light that has passed through the optical bandpass filter F1 and each filter 34R, 34G, 34B of the color filter 34 according to its respective sensitivity. The image sensor 33 outputs a first imaging signal S1 in which each imaging signal having an R value, G value, and B value according to the amount of light received by each light receiving element 33R, 33G, 33B is arranged serially in the order of a predetermined array pattern.

Figure 7 shows the relative sensitivity characteristic of the second imaging signal S2 obtained by converting the first imaging signal S1 by the conversion unit 60. It corresponds to the relative sensitivity characteristic of the second imaging signal S2 when irradiated with an ideal light source whose emission spectrum has a relative value = 1 in the entire wavelength range. The conversion unit 60 shown in Figure 7 generates the second imaging signal S2 by converting the RGB values of the first imaging signal S1 input from the image sensor 33 shown in Figure 5 into XYZ values. In other words, the conversion unit 60 converts the three-band RGB signal shown in the graph of Figure 5 into a three-band XYZ signal (Figure 7) that has sensitivity to a wavelength band different from the RGB signal. The graph shown in Figure 7 shows the three bands represented by the XYZ values that make up the second imaging signal S2. The horizontal axis of the graph indicates wavelength, and the vertical axis indicates relative sensitivity.

As shown in FIG. 7, one of the three bands XYZ constituting the second imaging signal S2 has a peak in the near-infrared wavelength region NIRA. The peaks of the other two bands are in the visible light wavelength region VA. More specifically, as shown in the graph in FIG. 7, the X band has a peak at approximately 890 nm, the Y band has a peak at approximately 620 nm, and the Z band has a peak at approximately 550 nm. Each peak of relative sensitivity shown by the three bands XYZ is higher than the relative sensitivity value of the other two bands in the wavelength range of each peak, and the XYZ bands are separated. However, although the X band has a peak in the near-infrared wavelength region NIRA, the peak is small.

When light with the combined emission spectrum of the white light source 20 and the near-infrared light source 25 shown in FIG. 8(a) is incident, the image sensor 33 outputs the three-band RGB signal shown in the graph of FIG. 9(a) as the first imaging signal S1 as a result of combining the emission intensity characteristic shown in FIG. 8(a) and the relative sensitivity characteristic of the first imaging signal S1 shown in FIG. 8(b). The conversion unit 60 converts the three-band RGB signal shown in the graph of FIG. 9(a) into a three-band XYZ signal shown in FIG. 9(b) that has sensitivity to a wavelength band different from the RGB signal.

As shown in the graph in Figure 9 (b), the X band has a peak at approximately 930 nm, the Y band has a peak at approximately 620 nm, and the Z band has a peak at approximately 550 nm. Each peak of sensitivity shown by the three bands XYZ is higher than the sensitivity of the other two bands, and each band of XYZ is separated. The two bands Y and Z are visible light bands, and the X band is a near-infrared light band. The two visible light bands Y and Z and the one near-infrared light band X are separated.

Second Example
Next, a second embodiment will be described with reference to Figures 10 to 14. The optical bandpass filter F2 shown in Figure 11(a) has a cutoff region in the visible light wavelength region VA at wavelengths of about 520 nm or less, and has a spectral transmittance characteristic with a transmission region in the wavelength region exceeding about 520 nm. In the near-infrared wavelength region NIRA, the entire region shown in the figure is a transmission region. The optical bandpass filter F2 has a transmission region at least in the near-infrared wavelength region.

In the camera 30 shown in FIG. 10, the optical bandpass filter F2 having the spectral transmittance characteristics shown in FIG. 11(a) blocks light from the object 12, which is the subject, with wavelengths of approximately 520 nm or less, and transmits light with wavelengths greater than approximately 520 nm. Therefore, even if the image sensor 33 itself has the sensitivity shown in FIG. 11(b), the image sensor 33 does not receive light with wavelengths of approximately 520 nm or less that is blocked by the optical bandpass filter 31. Therefore, as shown in the graph in FIG. 10, the image sensor 33 has substantial sensitivity in the wavelength range greater than approximately 520 nm. In other words, the image sensor 33 has no sensitivity in the wavelength range less than approximately 520 nm, but has sensitivity in the wavelength range greater than approximately 520 nm.

The R light receiving element 33R constituting the image sensor 33 shown in FIG. 5 has a sensitivity in the band indicated by R in the graph of the same figure, the G light receiving element 33G has a sensitivity in the band indicated by G, and the B light receiving element 33B has a sensitivity in the band indicated by B. Each light receiving element 33R, 33G, 33B receives light that has passed through the optical bandpass filter F2 and each filter 34R, 34G, 34B of the color filter 34 according to its respective sensitivity. The image sensor 33 outputs a first imaging signal S1 in which each imaging signal having an R value, G value, and B value according to the amount of light received by each light receiving element 33R, 33G, 33B is arranged serially in the order of a predetermined array pattern.

Figure 12 shows the relative sensitivity characteristics of the second imaging signal S2 obtained by converting the first imaging signal S1 by the conversion unit 60. The conversion unit 60 shown in Figure 12 generates the second imaging signal S2 by converting the RGB values of the first imaging signal S1 input from the image sensor 33 shown in Figure 10 into XYZ values. That is, the conversion unit 60 converts the three-band RGB signal shown in the graph in Figure 10 into a three-band XYZ signal that has sensitivity to a wavelength band different from that of the RGB signal. The graph in Figure 12 shows the three bands represented by the XYZ values that make up the second imaging signal S2. The horizontal axis of the graph indicates wavelength, and the vertical axis indicates relative sensitivity.

As shown in FIG. 12, the peak of at least one of the three bands of the second imaging signal S2 is in the near-infrared wavelength region NIRA. In particular, in this example, the peak of one band is in the near-infrared wavelength region NIRA. In detail, as shown in the graph of FIG. 12, the X band has a peak at about 830 nm, the Y band has a peak at about 630 nm, and the Z band has a peak at about 550 nm. Each peak of sensitivity shown in the three bands XYZ is higher than the sensitivity of the other two bands at the position of the peak wavelength, and each band of XYZ is separated. Although the X band has a peak in the near-infrared region, each of the Y and Z bands also has a portion sensitive to the near-infrared wavelength region NIRA. In this case, the second imaging signal S2 cannot be separated into a visible light image and a near-infrared image. In other words, a near-infrared image is partially mixed in the visible light image, and a visible light image is partially mixed in the near-infrared image. Such images are not suitable for applications where visible light and near-infrared images need to be separated for inspection.

Therefore, in this second embodiment, the light sources are the white light source 20 (FIG. 1) and the near-infrared light source 25 (FIG. 1), both of which have the emission spectrum shown in FIG. 13(a). As a result, the image sensor 33 outputs the three-band RGB signal shown in the graph of FIG. 14(a) as the first imaging signal S1 based on the incident light having the emission spectrum shown in FIG. 13(a) and the RGB sensitivity of the image sensor 33 shown in the graph of FIG. 13(b). The conversion unit 60 converts the three-band RGB signal shown in the graph of FIG. 14(a) into the three-band XYZ signal shown in FIG. 14(b), which has sensitivity to wavelength bands different from the RGB signal.

As shown in the graph of FIG. 14(b), the X band has a peak at about 860 nm, the Y band has a peak at about 620 nm, and the Z band has a peak at about 550 nm. Each peak of sensitivity shown in the three bands XYZ is higher than the sensitivity of the other two bands at the position of the wavelength of the peak, and each band of XYZ is separated. The X band is a band with a peak in the near-infrared region, and the two bands Y and Z are bands with peaks in the visible light region. The two bands Y and Z in the visible light region and the one band X in the near-infrared region are separated. Therefore, the first image signal MS1, which is a visible light image signal output from the imaging device, and the second image signal MS2, which is a near-infrared image signal, are separated so that their wavelength bands do not overlap.

Next, the configurations of the conversion unit 60 and the inspection processing unit 70 will be described with reference to FIG.
15, an image of the article 12 is formed on the imaging surface of the image sensor 33 through the optical bandpass filter 31 and the lens 32. The image sensor 33 outputs a first imaging signal S1 as an imaging result of the article 12 to the conversion unit 60. The first imaging signal S1 is a serial signal including an R imaging signal (red signal), a G imaging signal (green signal) and a B imaging signal (blue signal) from the light receiving elements 33R, 33G and 33B. The R imaging signal, the G imaging signal and the B imaging signal are also simply referred to as the R signal, the G signal and the B signal.

As shown in FIG. 15, the conversion unit 60 includes an RGB separation unit 61, an XYZ conversion unit 62, an amplifier unit 63, and a high-luminance color suppression unit (not shown). The RGB separation unit 61 separates the first imaging signal S1 input from the image sensor 33 into an R imaging signal, a G imaging signal, and a B imaging signal.

The XYZ conversion unit 62 converts the R, G, and B signals input from the RGB separation unit 61 into an X, Y, and Z signal. In detail, the XYZ conversion unit 62 converts the R, G, and B signals, which are the signal values of the R, G, and B signals, into an X, Y, and Z signal by performing a matrix operation on the RGB values. Matrix coefficients are provided to the XYZ conversion unit 62. Here, the matrix used in the matrix operation is a 3x3 matrix. The XYZ conversion unit 62 is provided with the coefficients of the 3x3 matrix.

The XYZ conversion unit 62 performs a matrix operation to multiply the RGB values of the first imaging signal S1 by a 3x3 matrix determined using the matrix coefficients, and converts the signal into a second imaging signal S2 expressed in XYZ, which has different spectral characteristics from the RGB of the first imaging signal S1. The matrix coefficients are coefficients for dispersing the RGB of the first imaging signal S1 into multiple bands of the XYZ of the second imaging signal S2.

Here, the calculation formula for converting the RGB signals, which are the first imaging signal S1, into the XYZ signals, which are the second imaging signal S2, is given by the following formula (1).

Here, a1 to a3, b1 to b3, and c1 to c3 are matrix coefficients, and Gx, Gy, and Gz are amplification factors.

The XYZ conversion unit 62 performs the calculation process of multiplying the RGB values by a 3x3 matrix in the above formula (1). The XYZ conversion unit 62 outputs the XYZ values before being multiplied by the amplification factor to the amplification unit 63.

Here, the number of colors of the color filter 34 in the image sensor 33 is n (where n is a natural number of 3 or more). The matrix calculation performed between n imaging signals is an m×n matrix calculation (where m is a natural number of 2 or more). In the m×n matrix, matrix coefficients are set that can separate the imaging signals for each color of the first imaging signal S1 into n bands of wavelength regions. In this example, the imaging signals for each color of the first imaging signal S1 are R signals, G signals, and B signals, and the number of colors n is "3" (n=3). The second imaging signal S2 has three bands of X signals, Y signals, and Z signals, and m=3. In other words, the m×n matrix is a 3×3 matrix. And, in the 3×3 matrix, matrix coefficients are set that can increase the separation of the three bands.

For example, when an optical bandpass filter F1 having the transmission characteristics shown in FIG. 6(a) is used, the following 3×3 matrix coefficients are given to the matrix calculation for the conversion from FIG. 5 to FIG. 7, and the matrix calculation for the conversion from FIG. 9(a) to FIG. 9(b). That is, the following 3×3 matrix coefficients are given: a1=0, a2=-0.13, a3=1, b1=1, b2=0.1, b3=-1.15, c1=-0.2, c2=1, c3=-0.7.

Note that m×n (where m ≠ n) matrix calculations may also be used. If the number of colors is "3", the calculation is not limited to 3×3 matrix calculations, and a 3×4 matrix calculation may be performed to generate a 4-band multispectral image, or a 3×2 matrix calculation may be performed to generate a multispectral image with fewer bands than the number of colors n.

The amplifier 63 multiplies the XYZ values from the XYZ conversion unit 62 by the given X amplification factor Gx, Y amplification factor Gy, and Z amplification factor Gz. The amplifier 63 multiplies the X value after XYZ conversion by the X amplification factor Gx, the Y value by the Y amplification factor Gy, and the Z value by the Z amplification factor Gz. That is, the amplifier 63 normalizes the XYZ bands by multiplying a 1×3 matrix in the above formula (1) with the amplification factors Gx, Gy, and Gz as matrix coefficients. The amplifier 63 outputs the normalized XYZ values as the imaging signal S2. Note that the normalization process may be a method of fixing one signal level and adjusting the other two signal levels. For example, the Y signal may be fixed and the X signal and Z signal may be adjusted.

In this way, the conversion unit 60 sequentially performs RGB separation processing, XYZ conversion processing, and normalization processing on the input first imaging signal S1 to output the second imaging signal S2. The conversion unit 60 outputs a three-band multispectral image of the object 12 to the signal processing unit 65. In other words, the conversion unit 60 converts the first imaging signal S1 into the second imaging signal S2. The first imaging signal S1 is an N-band image signal. The second imaging signal S2 is an N-band image signal. In other words, the conversion unit 60 converts the N-band image signal into an N-band image signal. In this example, the three-band RGB image is converted into a three-band XYZ image. In this way, a three-band multispectral image is generated. In other words, the conversion unit 60 generates a first image signal MS1 in the P band (where P is a natural number less than N) that has spectral sensitivity in the visible light region, and a second image signal MS2 in the Q band (where Q is Q=N-P) that has spectral sensitivity in the near-infrared region, from the first imaging signal S1.

The imaging device 15 includes a signal processor 65 that generates a pseudo-color multi-channel color image signal based on the image signals of the multiple bands for at least one of the P band and Q band image signals. The signal processor 65 inputs the N-band (3-band) XYZ image constituting the second imaging signal S2 by dividing it into the P band and the Q band (where Q=N-P). When the second imaging signal S2 is 3-band, the signal processor 65 inputs it by dividing it into a 2-band first image signal MS1 and a 1-band second image signal MS2. The 2-band first image signal MS1 is a visible light image signal, and the 1-band second image signal MS2 is a near-infrared image.

The signal processing unit 65 has a pseudo-color signal processing unit 66 that converts the first image signal MS1 of the P band, which is a multi-band signal, into a pseudo-color image signal, and a luminance signal processing unit 67 that adjusts the luminance value of the second image signal MS2 of one band. The pseudo-color signal processing unit 66 outputs the first image signal IS1, which represents the first image signal MS1 in pseudo-color, to the inspection processing unit 70. The luminance signal processing unit 67 outputs the second image signal IS2 after the luminance value of the second image signal MS2 has been adjusted to the inspection processing unit 70. The luminance signal processing unit 67 may perform luminance adjustment by nonlinear processing such as gamma correction, edge enhancement processing, etc. as necessary.

16(a) and (b) show images captured by the imaging device 15. In the example of FIG. 16, the objects in the image are two containers. The container on the left is a translucent synthetic resin container containing liquid. The container on the right is a transparent glass container containing a dark-colored (e.g., black) liquid. As shown in FIG. 16(a), the first image signal IS1 is a signal of a two-channel pseudo-color image. Therefore, the first image is displayed in two-channel pseudo-color on the display unit 41. Since the inspector can visually recognize defects and the like by the difference in color by looking at the first image on the display unit 41, it is easy to inspect the object 12. Also, as shown in FIG. 16(b), the second image signal IS2 is a signal of a transmitted image in which near-infrared light has passed through the object 12. The second image is displayed on the display unit 41 as a transmitted image of the object 12 as shown in the same figure. Therefore, the contents of the container and foreign objects mixed in the liquid in the container can be inspected. A wire has been mixed into the liquid in the container on the right, and foreign objects such as the wire, which are not visible in the visible light image in Figure 16(a), can be seen in the near-infrared image shown in Figure 16(b).

Next, the inspection processing unit 70 will be described.
The inspection processing unit 70 inspects the subject, that is, the item 12, based on a first image signal IS1 in the P band having spectral sensitivity in the visible light region output by the imaging device 15, and a second image signal IS2 in the Q band having spectral sensitivity in the near-infrared region.

The inspection processing section 70 includes a visible light inspection processing section 71, a near-infrared inspection processing section 72, and a mixed inspection processing section 73. A first image signal IS1 is input to the visible light inspection processing section 71, a second image signal IS2 is input to the near-infrared inspection processing section 72, and the first image signal IS1 and the second image signal IS2 are input to the mixed inspection processing section 73. The first image signal IS1 and the second image signal IS2 input to the mixed inspection processing section 73 are a two-band visible light image signal and a one-band near-infrared image signal.

The visible light inspection processing unit 71 inspects the item 12 based on the first image signal IS1, which is a visible light image. The near-infrared inspection processing unit 72 inspects the item 12 based on the second image signal IS2, which is a near-infrared image. The mixed inspection processing unit 73 inspects the item 12 based on both the first image signal IS1 and the second image signal IS2, which are images of visible light and near-infrared light. For example, the visible light inspection processing unit 71 specifies a color coordinate range from pseudo-color for the first image signal IS1, which is a pseudo-color image signal, to extract specific color areas, and judges the quality of the item 12 based on the size and number of the specific color areas. The near-infrared inspection processing unit 72 binarizes the second image signal IS2, which is a luminance image signal, by judging the upper and lower threshold values of the luminance level, and judges the quality of the item 12 based on the size and number of the inspection target areas extracted as a result of the binarization. The mixed inspection processing unit 73 judges the quality of the item 12 based on the size and number of common areas obtained as a result of an AND operation between a specific color area separately extracted from the first image signal IS1, which is a pseudo-color image signal, and a threshold area separately extracted from the second image signal IS2, which is a luminance image signal.

Next, the operation of the imaging device 15 and the inspection device 11 will be described.
1, an object 12, which is a subject, is transported by a conveyor 16 of a transport device 13. When a sensor 17 detects the object 12, a trigger signal is input to a control unit 50. The control unit 50, which has input the trigger signal, outputs a light emission command signal to cause the white light source 20 and the near-infrared light source 25 to emit light, and outputs an image capture command signal to the image sensor 33 to cause the image sensor 33 to capture an image.

Visible light reflected by the surface of the article 12 and near-infrared light transmitted through the article 12 are incident on the camera 30 after passing through an optical bandpass filter 31 and a lens 32. An image of the visible light reflected by the surface of the article 12 and an image of the near-infrared light transmitted through the article 12 are formed on the imaging surface of the image sensor 33 in the camera 30. The image sensor 33 captures one frame of the article 12 through the optical bandpass filter 31. The image sensor 33 outputs a first imaging signal S1 to the conversion unit 60.

The converter 60 separates the first imaging signal S1 into RGB, and converts the RGB values into XYZ values by multiplying the separated RGB values by a 3x3 matrix. Normalized XYZ values are generated by multiplying the XYZ values by amplification factors Gx, Gy, and Gz. In this way, the converter 60 outputs a second imaging signal S2 including a three-band multispectral image. The second imaging signal S2 is output to the signal processor 65. In other words, the converter 60 generates, from the first imaging signal S1, a first image signal MS1 in the P band (where P is a natural number less than N) that has spectral sensitivity in the visible light region, and a second image signal MS2 in the Q band (where Q is Q=N-P) that has spectral sensitivity in the near-infrared region.

The signal processing unit 65 generates a pseudo-color multi-channel color image signal based on the multi-band image signal for at least one of the P band and Q band image signals that are multi-band. The signal processing unit 65 inputs the N-band (3-band) XYZ image that constitutes the second imaging signal S2, separated into the P band and the Q band (where Q=N-P). When the second imaging signal S2 is 3-band, the 2-band first image signal MS1 is input to the pseudo-color signal processing unit 66, and the 1-band second image signal MS2 is input to the luminance signal processing unit 67. In this example, the first image signal MS1 is a visible light image signal, and the second image signal MS2 is a near-infrared image.

The pseudo-color signal processing unit 66 converts the two-band first image signal MS1 into a pseudo-color image signal. The luminance signal processing unit 67 adjusts the luminance value of the one-band second image signal MS2. As a result, the first image signal IS1 and the second image signal IS2 are output from the signal processing unit 65 to the inspection processing unit 70. The first image signal IS1 is a two-channel pseudo-color image signal shown in FIG. 16(a). The second image signal IS2 is a near-infrared image signal shown in FIG. 16(b).

In the inspection processing unit 70, the visible light inspection processing unit 71 inspects the item 12 based on the first image signal IS1, which is a visible light image. The near-infrared inspection processing unit 72 inspects the item 12 based on the second image signal IS2, which is a near-infrared image. The mixed inspection processing unit 73 inspects the item 12 based on both the first image signal IS1 and the second image signal IS2.

Each of the inspection processing units 71 to 73 judges the quality of the object 12. If the inspection result indicates that the object 12 is defective, the ejection device is driven to remove the defective object 12 from the conveyor 16.
In this embodiment, an imaging method is adopted. The imaging method includes an irradiation step, an imaging step, and a conversion step. In the imaging step, the white light source 20 irradiates the object 12 as a subject with white light having an emission spectrum in the visible light region, and the near-infrared light source irradiates the object 12 as a subject with near-infrared light having an emission spectrum in the near-infrared light region and not having an emission spectrum in the emission spectrum region of the white light source 20. In the imaging step, a camera 30 having a color filter 34 as an example of a spectral optical filter of N bands (N is a natural number of 3 or more) having different spectral transmittance characteristics and an image sensor 33 having sensitivity in the visible light region and the near-infrared region images the object 12 through an optical bandpass filter 31 (F1, F2) arranged on the optical path between the image sensor 33 and the object 12. In the conversion step, the image signal captured by the image sensor 33 is separated into N-band signals, and a matrix operation is performed on the separated N-band image signals to generate a first image signal MS1 in a P-band (where P is a natural number less than N) having spectral sensitivity in the visible light region, and a second image signal MS2 in a Q-band (where Q is Q=N-P) having spectral sensitivity in the near-infrared region.

According to the first embodiment described above in detail, the following effects can be obtained.
(1) The imaging device 15 includes a camera 30 having an image sensor 33 having N bands (N is a natural number of 3 or more) of spectral optical filters with different spectral transmittance characteristics and sensitivity in the visible light region and the near-infrared region, and an optical bandpass filter 31 arranged on an optical path between the image sensor 33 and an object 12 as a subject. The imaging device 15 includes a white light source 20 that irradiates the object 12 with white light having an emission spectrum in the visible light region, and a near-infrared light source 25 that irradiates the object 12 with near-infrared light having an emission spectrum in the near-infrared light region and not having an emission spectrum in the emission spectrum region of the white light source 20. The imaging device 15 also includes a conversion unit 60 that separates an image signal captured by the image sensor 33 into N band signals and performs a matrix operation on the separated N band image signals to generate a first image signal in a P band having spectral sensitivity in the visible light region and a second image signal in a Q band having spectral sensitivity in the near-infrared region. Therefore, a visible light image and a near-infrared image of the article 12 captured at the same angle of view can be simultaneously acquired by a single camera 30 with a simple configuration.

(2) The camera 30 is a general-purpose color camera with the infrared cut filter removed. Therefore, since the camera 30 uses a general-purpose color camera, the configuration of the imaging device 15 can be simple. Incidentally, in imaging devices that output a color image signal, an infrared image signal, or a mixed image signal that combines both, a special imaging element or a special camera capable of capturing visible light images and near-infrared images is used. In contrast, according to the imaging device 15 of this embodiment, the camera 30 uses a general-purpose color camera, so there is no need to use a special imaging element or a special camera. In addition, when image processing is required for a color image in visible light and a black-and-white image in near-infrared in image recognition and inspection of a subject, two sets of imaging systems are usually required: a color camera and white lighting, and a near-infrared black-and-white camera and near-infrared lighting. In the case of a two-camera configuration, the images obtained from each camera have different positions of the subject, so in order to share and process the images from the two cameras, processing such as position and angle of view correction is required. In contrast, with the imaging device 15 of this embodiment, visible light images and near-infrared images are acquired with a single camera 30, eliminating the need for processing such as position and angle of view correction. This allows the imaging device 15 to have a simple configuration and relatively simple processing. Furthermore, conventional imaging devices equipped with special imaging elements or special cameras capable of visible light imaging and near-infrared imaging are expensive and offer very little freedom in selecting specifications. In contrast, the imaging device 15 of this embodiment uses a general-purpose color camera, making the imaging device 15 relatively inexpensive and offering a high degree of freedom in selecting specifications.

(3) For at least one of the image signals having multiple bands, either the P band or the Q band, a pseudo-color, multi-channel color image signal is generated based on the image signal of the multiple bands. Therefore, for the image signal of the multiple bands, a pseudo-color, multi-channel color image can be displayed, making it easier for an inspector to perform an inspection by looking at the image and to confirm the inspection results, for example.

(4) The white light source 20 is configured by combining multiple visible light light sources with different visible light wavelength ranges, or the near-infrared light source 25 is configured by combining multiple near-infrared light sources 25 with different infrared light wavelength ranges. The combined emission spectrum of the white light source 20 and the near-infrared light source 25 has multiple peaks in the transmission wavelength range of the optical bandpass filter 31. This makes it easier to separate the visible light image and the near-infrared image, and provides an image suitable for inspection, etc.

(5) If the surface of the object 12 facing the camera 30 is the front surface and the surface opposite the front surface is the back surface, the white light source 20 illuminates the front surface of the object 12 with white light, and the near-infrared light source 25 irradiates near-infrared light from the back surface of the object 12. The camera 30 receives the white light reflected by the surface of the object 12 and the near-infrared light transmitted through the object 12. Thus, the imaging device 15 can obtain an image of the surface of the object 12 and an image of the transmitted light transmitted through the object 12 by the near-infrared light.

(6) Between the near-infrared light source 25 and the article 12, an optical filter member 29 is placed, with the surface of the article 12 serving as the background. The optical filter member 29 reflects at least light of a specific wavelength in the visible light range of the white light irradiated onto the surface from the white light source 20, or absorbs light in the visible light range, and transmits near-infrared light irradiated from the near-infrared light source 25 onto the back surface opposite the front surface. This allows the background color when the article is imaged to be selected, making it possible to provide visible light images and near-infrared images suitable for inspection, etc.

(7) The inspection device 11 includes an imaging device 15 and an inspection processing unit that inspects the item 12 based on a first image signal in the P band having spectral sensitivity in the visible light region and a second image signal in the Q band having spectral sensitivity in the near-infrared region output by the imaging device 15. Therefore, since the item 12 can be inspected using visible light images and near-infrared images, it is possible to inspect the item 12 in a way that cannot be obtained by only visible light inspection that inspects using visible light images.

(8) The imaging method for capturing an image of an item 12 with a camera 30 to generate an image signal includes a light irradiation step, an imaging step, and a conversion step, and therefore provides the same effects as the imaging device 15 described in (1) above.

The embodiment is not limited to the above and may be modified in the following manner.
The configuration of the imaging device 15 is not limited to the configuration shown in FIG. 2 and FIG. 3. For example, as shown in FIG. 17, the imaging device 15 may be configured to include a light source 80 that irradiates visible light and near-infrared light from the front side of the object 12, which is the subject, toward the front of the object 12. The light source 80 includes a plurality of light-emitting units 21 to 23 that have different emission spectra in the visible light wavelength region VA. The plurality of light-emitting units 21 to 23 are the same as the first light-emitting unit 21, the second light-emitting unit 22, and the third light-emitting unit 23 that constitute the white light source 20 in the above embodiment. For example, the plurality of light-emitting units 21 to 23 are made of LEDs, and are made of red LEDs, green LEDs, and blue LEDs, respectively. Furthermore, the light source 80 includes a first light-emitting unit 26 and a second light-emitting unit 27 that irradiate near-infrared light that has an emission spectrum in the near-infrared wavelength region NIRA and does not have an emission spectrum in the emission spectrum region of the light-emitting units 21 to 23. The plurality of types of light-emitting units 26 and 27 are made of LEDs, for example.

In this modified example, the white light source and the near-infrared light source are configured from a single light source 80 that irradiates white light and near-infrared light onto the front surface of the object 12 as the subject, which is the surface facing the camera 30. The light source 80 is configured in a mixed arrangement of multiple light-emitting units 21-23 that irradiate white light and multiple light-emitting units 26, 27 that irradiate near-infrared light.

In the example shown in FIG. 17, light-emitting units 21-23 consisting of white LEDs and light-emitting units 26, 27 consisting of near-infrared LEDs are evenly arranged in one light source 80. Therefore, light-emitting units 26, 27 consisting of near-infrared LEDs are arranged next to light-emitting units 21-23 consisting of white LEDs. Note that in this modified example, light-emitting units 21-23 of light source 80 that irradiate white light constitute an example of a white light source, and light-emitting units 26, 27 that irradiate near-infrared light constitute an example of a near-infrared light source. In this configuration in which white light and near-infrared light are irradiated to the front of object 12, the white light source and the visible light source do not necessarily need to be separate entities, and may be constituted by a single light source 80. However, in FIG. 17, instead of the light source 80, a configuration in which the white light source 20 and the near-infrared light source 25 are adjacently arranged, or a configuration in which the white light source 20 and the near-infrared light source 25 are arranged as far apart as possible so long as the white light and the near-infrared light can be irradiated from the front of the article 12, may be adopted.

The surface of the article 12 is printed with ink. The ink used for printing includes ink that reflects infrared light. The inspection device 11 having the imaging device 15 shown in FIG. 17 inspects the ink that reflects infrared light printed on the subject. The article 12 may have a printed area on its surface printed with ink that is a mixture of ink that reflects visible light and ink that reflects infrared light, or may have a first printed area printed with ink that reflects visible light and a second printed area printed with ink that reflects near-infrared light. The article 12 may also have a film having a portion that reflects near-infrared light attached to its surface by heat welding or adhesive.

Visible light irradiated from the light source 80 onto the front of the article 12 is reflected by the surface of the article 12 and enters the camera 30. Also, near-infrared light irradiated from the light source 80 onto the surface of the article 12 is reflected by the surface of the article 12 and enters the camera 30. In this modified example, the near-infrared light reflected by the surface of the article 12 enters the camera 30, making it possible to inspect parts that do not reflect visible light but reflect near-infrared light, which are present on the surface of the article 12 or inside the article 12 that has been penetrated.

In this way, even in the imaging device 15 that captures the near-infrared light reflected on the surface of the article 12, the conversion unit 60 and the signal processing unit 65 connected to the camera 30 perform the same processing as in the above embodiment. Then, the inspection processing unit 70 can inspect the surface of the article 12 or the part inside the article 12 that reflects near-infrared light based on the first image signal IS1 and the second image signal IS2 output by the imaging device 15. In addition, since one light source 80 is used in this modified example, the visible light image and the near-infrared image are acquired as images in which light is irradiated from the same direction on the article 12. For example, when inspecting using two images with different light irradiation angles, the difference in the irradiation angle may prevent appropriate inspection from being performed. In contrast, in this modified example, when the visible light image and the near-infrared image are used together or compared to obtain the inspection result of the article 12, appropriate inspection can be performed.

The coefficients of the 3x3 matrix used in the matrix calculation are not limited to the values in the above embodiment. For example, in the second embodiment, the coefficients of the 3x3 matrix are changed to a1 = 0.05, a2 = -0.2, a3 = 1, b1 = 1, b2 = 0.2, b3 = -1, c1 = 0.15, c2 = 0.8, c3 = -0.9.

In this case, the XYZ imaging characteristics shown in FIG. 18(a) are obtained instead of FIG. 12, and the XYZ output characteristics shown in FIG. 18(b) are obtained instead of FIG. 14(b). As shown in FIG. 18(b), a first image IS1 in which a visible light image and a near-infrared image are mixed, and a second image IS2 which is a near-infrared image are obtained. In this way, the configuration is not limited to one in which the image is divided into a visible light image and a near-infrared image, and at least one of the first image IS1 and the second image IS2 may be an image in which a visible light image and a near-infrared image are mixed.

- In the above embodiment and the modified example shown in FIG. 17, the types of light-emitting units constituting the white light source are not limited to three types, and may be four or more types, two types, or one type, as long as they are capable of emitting white light. Also, the types of light-emitting units constituting the near-infrared light source are not limited to two types, and may be three or more types, or one type, as long as they are capable of emitting near-infrared light.

- In the above embodiment, two bands of visible light and one band of infrared light are used, but the reverse is also possible. In other words, one band of visible light and two bands of infrared light are also possible. In other words, in the above embodiment, when N=3, P=2 and Q=1 are used, but P=1 and Q=2 may also be used. In this case, a pseudo-color two-channel color image signal may be generated based on the two-band second image signal.

- The value of N is not limited to N=3, and may be N=4. For example, in the case of a color camera with primary color filters, the four colors may be R, G1, G2, and B. Also, it may be a color camera with complementary color filters, and the complementary colors may be yellow, cyan, magenta, and green.

-When N colors is 4 colors, P=2 and Q=2, a pseudo-color two-channel color image signal may be generated based on a two-band first image signal, and a pseudo-color two-channel color image signal may be generated based on a two-band second image signal.

- In the above embodiment, pseudo colors are created, but it is also possible to configure the system without creating pseudo colors. For example, if the system is not configured to display an inspection image on a monitor, it is not necessary to create pseudo colors.

The white light source 20 and the near-infrared light source 25 may be configured to emit light in a time-division manner rather than simultaneously.
Image data (e.g., RGB image data) based on a first imaging signal captured by the image sensor 33 through the optical bandpass filter 31 using the camera 30 is stored in a removable memory such as a USB memory. The image data stored in the removable memory may be read by a personal computer, and a CPU (conversion unit 60) of the personal computer may perform a conversion process including a matrix operation to generate a multi-band multispectral image. In other words, the device that performs the imaging step and the device that performs the conversion step may be separate devices. A multi-band multispectral image can also be obtained by such an imaging method.

- At least one of the amplification rate and the parameters may be calculated by the inspection processing unit 70, or values calculated in advance by another computer may be written to the memory of the imaging device 15.

The imaging device 15 is not limited to being used for inspection. For example, the imaging device 15 may be used as the eyes of a robot, and the robot may distinguish between visible light and infrared light images and recognize them.
The image output by the imaging device 15 may be visually recognized by an inspector for inspection.

Visible light (for example, white light) may be irradiated from behind the subject.
The number of colors of the color filters 34 constituting the image sensor 33 is not limited to three or four, and may be five, six, seven, or eight. At least one of the colors may be a filter that transmits invisible light but not visible light. For example, the image sensor 33 may be provided with a color filter including an NIR filter that transmits near-infrared light.

The item 12, which is an example of a subject to be imaged or inspected, is not particularly limited. The item 12 may be, for example, a container such as a plastic bottle or a glass bottle, food, a beverage, an electronic component, an electrical appliance, daily necessities, a part, a component, a raw material such as a powder or liquid, etc. It is sufficient that the item 12 can be inspected for quality using a multispectral image.

The subject 12 may be any object that can be inspected using a multi-band multispectral image with at least one band in the near-infrared wavelength range (NIRA) to inspect an object that is difficult to inspect using an image captured under visible light alone. The object 12 may be, for example, a container containing liquid, foodstuffs such as fruits and vegetables, plants such as flowers, semi-processed or processed foods of plant or animal origin, or living things.

The imaging subject or inspection subject is not limited to the item 12. The inspection subject may be a photograph of a building, an aerial photograph of a terrain, a photograph of the sky, an astronomical photograph, a microscope photograph, or the like.
The imaging device 15 may be configured as a device separate from the inspection processing unit 70. The imaging device 15 may be used for purposes other than inspection.

The arrangement pattern of the color filters constituting the image sensor 33 is not limited to the RGB Bayer arrangement, and may be any arrangement pattern such as a stripe arrangement.
At least one of the control unit 50, the conversion unit 60, and the inspection processing unit 70 may be configured in whole or in part by software consisting of a computer executing a program, or may be configured by hardware such as an electronic circuit.

10... inspection system, 11... inspection device, 12... object as an example of a subject, 13... conveying device, 15... imaging device, 16... conveyor, 17... sensor, 20... white light source, 21... first light-emitting unit, 22... second light-emitting unit, 23... third light-emitting unit, 25... near-infrared light source, 26... first light-emitting unit, 27... second light-emitting unit, 29... optical filter member, 30... camera, 30a... lens barrel, 31, F1, F2... optical band-pass filter, 31A, 32B... optical band-pass filter, 32... lens, 33... color image sensor (image sensor), 33R... R light-receiving element, 33G... G light-receiving element, 33B... B light-receiving element, 34... as an example of a spectral optical filter color filters, 34R...R filter, 34G...G filter, 34B...B filter, 40...control processing unit, 41...display unit, 50...control unit, 60...conversion unit, 61...RGB separation unit, 62...XYZ conversion unit, 63...amplification unit, 70...inspection processing unit, 71...visible light inspection processing unit, 72...near-infrared inspection processing unit, 73...mixed inspection processing unit, 80...light source, 201...infrared light cut filter (IR cut filter), LS...emission spectrum, S1...first imaging signal, S2...second imaging signal, MS1...first image signal, MS2...second image signal, IS1...first image signal, IS2...second image signal, VA...visible light wavelength region, NIRA...near-infrared wavelength region.

Claims (8)

a camera having an image sensor having N bands (N is a natural number of 3 or more) of spectral optical filters with different spectral transmittance characteristics and having sensitivity in the visible light region and the near infrared region;
a filter disposed on an optical path between the image sensor and a subject;
a white light source that irradiates the subject with white light having an emission spectrum in the visible light region;
a near-infrared light source that irradiates the subject with near-infrared light having an emission spectrum in a near-infrared light region and not having an emission spectrum in an emission spectrum region of the white light source;
a conversion unit that separates an image signal captured by the image sensor into signals of the N bands and performs a matrix operation on the separated N-band image signals to generate a first image signal of a P band (where P is a natural number less than N) having a spectral sensitivity in a visible light region and a second image signal of a Q band (where Q is Q=N−P) having a spectral sensitivity in a near-infrared region ,
The image sensor has a plurality of light receiving elements having sensitivity in the visible light region and the near infrared region,
the filter has a blocking region that blocks light having a specific wavelength or less in the visible light region so that some of the light receiving elements among the plurality of light receiving elements have a sensitivity peak in the near-infrared region;
The conversion unit is
Separating an image signal captured by the image sensor into an R signal, a G signal, and a B signal;
generating an X signal having a spectral sensitivity in a near-infrared region by performing a matrix operation on the R signal, the G signal, and the B signal using matrix coefficients determined so as to leave a near-infrared region component of the B signal and to cancel out a visible light region component of the B signal, a R signal component, and a G signal component;
generating a Y signal having spectral sensitivity in the visible light region by performing a matrix operation on the R signal, the G signal, and the B signal using matrix coefficients determined so as to leave a visible light region component of the G signal and to cancel out a near-infrared region component of the G signal, a component of the R signal, and a component of the B signal;
an imaging device that generates a Z signal having spectral sensitivity in the visible light range by performing a matrix operation on the R signal, the G signal, and the B signal using matrix coefficients determined so as to leave the visible light range components of the R signal and to cancel out the near-infrared range components of the R signal, the G signal components, and the B signal components . The imaging device according to claim 1, characterized in that the camera is a general-purpose color camera with an infrared cut filter removed. The imaging device according to claim 1 or 2, characterized in that for at least one of the image signals of the P band and the Q band, which is a multiple band, a color image signal is generated using pseudo-color based on the image signal of the multiple bands. If the surface of the subject facing the camera is the front surface, and the surface opposite the front surface is the back surface,
the white light source illuminates the front of the subject with white light, and the near-infrared light source irradiates the back of the subject with near-infrared light;
4. The imaging device according to claim 1, wherein the camera receives light reflected by a surface of the subject from the white light and near-infrared light transmitted through the subject. an optical filter member having a surface that is a background of the subject disposed between the near-infrared light source and the subject;
5. The imaging device according to claim 4, wherein the optical filter member reflects at least a specific wavelength of light in the visible light range of the white light irradiated from the white light source onto the front surface or absorbs light in the visible light range, and transmits near-infrared light irradiated from the near-infrared light source onto a back surface opposite the front surface. the white light source and the near-infrared light source are configured by a single light source that irradiates the white light and the near-infrared light onto a front surface of the subject that faces the camera,
4. The imaging device according to claim 1, wherein the light source is configured in an array in which a plurality of light-emitting units that irradiate white light and a plurality of light-emitting units that irradiate near-infrared light are mixed. An imaging device according to any one of claims 1 to 6,
an inspection processing unit that inspects the subject based on the first image signal of a P band having spectral sensitivity in the visible light region and the second image signal of a Q band having spectral sensitivity in the near-infrared region output by the imaging device. An imaging method for capturing an image of a subject with a camera to generate an image signal, comprising:
a light irradiation step in which a white light source irradiates the subject with white light having an emission spectrum in a visible light region, and a near-infrared light source irradiates the subject with near-infrared light having an emission spectrum in a near-infrared light region but not in the emission spectrum region of the white light source;
an imaging step in which a camera having an image sensor with a plurality of light receiving elements having sensitivity to the visible light region and the near infrared region , the camera having a spectral optical filter with N bands (N is a natural number of 3 or more) with different spectral transmittance characteristics and a filter having a blocking region that blocks light having a specific wavelength or less in the visible light region so that some of the light receiving elements have a sensitivity peak in the near infrared region, and an image of the subject is captured through the filter;
a conversion step of separating an image signal captured by the image sensor into signals of the N bands, and performing a matrix operation on the separated image signals of the N bands to generate a first image signal of a P band (where P is a natural number less than N) having a spectral sensitivity in a visible light region, and a second image signal of a Q band (where Q is Q=N−P) having a spectral sensitivity in a near-infrared region ,
In the conversion step,
Separating an image signal captured by the image sensor into an R signal, a G signal, and a B signal;
generating an X signal having a spectral sensitivity in a near-infrared region by performing a matrix operation on the R signal, the G signal, and the B signal using matrix coefficients determined so as to leave a near-infrared region component of the B signal and to cancel out a visible light region component of the B signal, a R signal component, and a G signal component;
generating a Y signal having spectral sensitivity in the visible light region by performing a matrix operation on the R signal, the G signal, and the B signal using matrix coefficients determined so as to leave a visible light region component of the G signal and to cancel out a near-infrared region component of the G signal, a component of the R signal, and a component of the B signal;
an imaging method comprising: generating a Z signal having spectral sensitivity in the visible light range by performing a matrix operation on the R signal, the G signal, and the B signal using matrix coefficients determined so as to leave the visible light range components of the R signal and to cancel out the near-infrared range components of the R signal, the G signal components, and the B signal components .