JP7118318B2 - Imaging device failure diagnosis system - Google Patents

Description

The present disclosure relates to an imaging device failure diagnosis system.

Japanese Patent Laid-Open No. 2002-200002 describes a color imaging device with a test pattern projector. The color imaging device includes a test pattern, a light source for illuminating the test pattern, and reflects the light that has passed through the test pattern and transmits the light from the subject, thereby causing these lights to enter the imaging tube. Equipped with a half mirror. The color imaging device detects the color shift of the image caused by the failure of the imaging device based on the photographed image of the test pattern.

JP-A-61-71789

In the technique described above, when photographing a subject, preparation for photographing the subject (preparation of lighting for the subject, etc.) and when photographing a test pattern, preparation for photographing the test pattern is required. Therefore, for example, there is a problem that it is not possible to frequently perform photographing for fault diagnosis of the image pickup device during photographing of an object.

The present disclosure has been made to solve the above-described problems, and aims to provide a technique capable of frequently performing imaging for fault diagnosis of an imaging device.

An imaging device failure diagnosis system according to the present disclosure includes a subject illumination light emitting device for illuminating a subject, a test chart for imaging device failure diagnosis, a test chart illumination light emitting device for illuminating the test chart, and a subject. One surface from which the light emitting device for illumination emits and transmits the first light reflected by the subject, and the other surface from which the light emitting device for illuminating the test chart emits and reflects the second light that has passed through the test chart. an imaging device that captures an image of a subject by receiving first light transmitted through the half mirror and captures a test chart by receiving second light reflected by the half mirror; a switching control unit that controls switching between a first mode in which the subject illumination light emitting device illuminates the subject and a second mode in which the test chart illumination light emitting device illuminates the test chart; I have.

According to the present disclosure, it is possible to frequently perform imaging for fault diagnosis of an imaging device.

1 is a block diagram showing the configuration of an imaging device failure diagnosis system according to Embodiment 1; FIG. 4 is a flow chart showing an imaging device failure diagnosis method by the imaging device failure diagnosis system according to Embodiment 1 during production and inspection. 5 is a flowchart showing an imaging device fault diagnosis method by the imaging device fault diagnosis system 100 according to Embodiment 1 at startup. 4 is a flowchart showing an imaging device failure diagnosis method by the imaging device failure diagnosis system according to Embodiment 1 during normal operation; FIG. 4 is a schematic diagram for explaining a specific example of the half mirror according to the first embodiment; FIG. 2 is a schematic diagram for explaining an imaging device failure diagnosis method by the imaging device failure diagnosis system according to Embodiment 1; 3 is a schematic diagram showing a configuration in which the image pickup device failure diagnosis system according to Embodiment 1 further includes a specific wavelength blocking device; FIG. FIG. 10 is a diagram for explaining reflection of a test chart in a subject image in a specific example of Embodiment 1; FIG. 4 is a diagram for explaining reflection data in a specific example of Embodiment 1; FIG. 4 is a schematic diagram showing a configuration in which the test chart according to Embodiment 1 has alignment markers; FIG. FIG. 4 is a diagram for explaining a reflection correction method according to a specific example of Embodiment 1; FIG. 1 is a schematic diagram for explaining a configuration in which an imaging device failure diagnosis system 100 according to Embodiment 1 includes a plurality of test charts, a plurality of test chart illuminating light emitting devices, and a plurality of half mirrors; FIG. 13A is a block diagram showing a hardware configuration that implements the functions of the imaging device fault diagnosis control unit according to Embodiment 1. FIG. 13B is a block diagram showing a hardware configuration for executing software that implements the functions of the imaging device fault diagnosis control unit according to Embodiment 1. FIG.

Hereinafter, in order to describe the present disclosure in more detail, embodiments for carrying out the present disclosure will be described with reference to the accompanying drawings.
Embodiment 1.
FIG. 1 is a block diagram showing the configuration of an imaging device failure diagnosis system 100 according to Embodiment 1. As shown in FIG. As shown in FIG. 1, the imaging device failure diagnosis system 100 includes a subject illumination light emitting device 1, a test chart 2, a test chart illumination light emitting device 3, a half mirror 4, an imaging device 5, a storage device 6, and an imaging device failure diagnosis system. A diagnosis control unit 7 is provided.

The object illumination light emitting device 1 irradiates the object with light. The light emitted by the subject illumination light emitting device 1 is reflected by the subject and propagates toward the half mirror 4 . More specifically, in Embodiment 1, the subject illumination light emitting device 1 is an LED that illuminates the driver of the vehicle.

A test chart 2 is a test chart for diagnosing failures of the imaging device. More specifically, in the first embodiment, the test chart 2 is a transmissive test chart for fault diagnosis of the imaging device. Examples of the test chart 2 include a test chart for resolution check, a test chart for uniformity check, a test chart for color reproducibility check (for example, Macbeth chart), a test chart for gradation check, or a human figure. Alternatively, a test chart in which a face is shown can be used.

The test chart irradiation light emitting device 3 irradiates the test chart 2 with light. Light emitted from the test chart illumination light emitting device 3 passes through the test chart 2 and propagates toward the half mirror 4 . In Embodiment 1, the test chart irradiation light emitting device 3 is an LED that irradiates the test chart 2 with light.

The half mirror 4 has one surface through which the first light emitted by the light emitting device 1 for illuminating the object and reflected by the object is transmitted, and the light emitting device 3 for illuminating the test chart, from which the light passes through the test chart 2. and a second surface that reflects the second light. The first light and the second light emitted from the half mirror 4 propagate toward the imaging device 5, respectively. A detailed structure of the half mirror 4 will be described later.

The imaging device 5 images the subject by receiving the first light transmitted through the half mirror 4 , and images the test chart 2 by receiving the second light reflected by the half mirror 4 . Examples of the imaging device 5 include a camera module and the like. The imaging device 5 includes, for example, a lens or an image sensor. That is, in that case, the failure diagnosis of the imaging device 5 is, for example, the failure diagnosis of the lens, image sensor, or the like. The subject image obtained by imaging the subject by the imaging device 5 and the test chart image obtained by imaging the test chart 2 by the imaging device 5 are output to the imaging device failure diagnosis control unit 7 . In addition, in this specification, an image is a moving image or a still image.
The storage device 6 stores expected values of test chart images. The storage device 6 outputs the expected value of the stored test chart image to the imaging device fault diagnosis control section 7 .

The imaging device failure diagnosis control section 7 includes a switching control section 10 , a failure diagnosis section 11 and a subject image correction section 12 . In Embodiment 1, the imaging device failure diagnosis control section 7 is a CPU.
The switching control unit 10 controls light emission and extinguishment of the light emitting device 1 for illuminating the subject, and controls light emission and extinguishing of the light emitting device 3 for illuminating the test chart. The switching control unit 10 switches between a first mode in which the subject illumination light emitting device 1 illuminates the subject and a second mode in which the test chart illumination light emitting device 3 illuminates the test chart 2 . to control. Details will be described later.

The failure diagnosis unit 11 diagnoses a failure of the imaging device 5 based on the test chart image obtained by imaging the test chart 2 with the imaging device 5 . More specifically, in the first embodiment, the fault diagnosis unit 11 compares the test chart image obtained by imaging the test chart 2 with the imaging device 5 and the expected value stored in the storage device 6. is used to diagnose the failure of the imaging device 5 . Details will be described later.
The subject image correction unit 12 corrects reflection of the test chart 2 in the subject image obtained by imaging the subject with the imaging device 5 . Details will be described later.

The operation of the imaging device failure diagnosis system 100 according to Embodiment 1 will be described below with reference to the drawings. FIG. 2 is a flow chart showing an imaging device failure diagnosis method by the imaging device failure diagnosis system 100 during production and inspection. FIG. 3 is a flowchart showing an imaging device fault diagnosis method by the imaging device fault diagnosis system 100 at startup. FIG. 4 is a flow chart showing an imaging device failure diagnosis method by the imaging device failure diagnosis system 100 during normal operation (during steady state). 2, 3 and 4 indicates the function of the imaging device failure diagnosis control unit 7 or the function of the switching control unit 10 of the imaging device failure diagnosis control unit 7, and the Test Chart LED (Test chart LED) indicates the function of the test chart illumination light emitting device 3, Driver LED indicates the function of the subject illumination light emitting device 1, Expect Data (expected value data), Initial Data (initial data). Data), Startup Data, and Cyclic Data indicate data stored in the storage device 6, respectively, and Initial Check and Cyclic Check indicate failure diagnosis, respectively. The function of the unit 11 or the function of the subject image correction unit 12, Reflect Correction (reflection correction) indicates the function of the subject image correction unit 12, and the Sub CPU (sub CPU) is a sub CPU (not shown). indicates the function of

First, the imaging device failure diagnosis method by the imaging device failure diagnosis system 100 at the time of production and inspection will be described below.
As shown in FIG. 2, at the time of production of the imaging device failure diagnosis system 100, expected values of test chart images are written in advance in the storage device 6 from an external device (not shown). Alternatively, the storage device 6 stores in advance the expected values of the test chart images. The expected value is, for example, a theoretical value, digital data on which the test chart 2 is based, or data obtained by photographing the test chart 2 with a reference machine.

During inspection of the imaging device fault diagnosis system 100, the switching control unit 10 of the imaging device fault diagnosis control unit 7 controls the test chart irradiation light emitting device 3 to irradiate the test chart 2 with light (step ST1). As a result, light (second light) emitted by the test chart irradiation light emitting device 3 passes through the test chart 2 , propagates toward the half mirror 4 , and is reflected by the half mirror 4 . Then, the imaging device 5 captures an image of the test chart 2 by receiving the light. When the light emitting device 3 for illuminating the test chart emits light, the front side of the surface of the half mirror 4 on the subject side (for example, the screen surface of the driver monitoring system) does not receive light from the outside of the imaging device failure diagnosis system 100. It may be light-shielded to prevent light from entering, or it may be unlight-shielded, assuming normal operation.

Next, the imaging device fault diagnosis control unit 7 writes the test chart image obtained by imaging the test chart 2 with the imaging device 5 in the storage device 6 as the test chart imaging data at the time of shipping inspection (step ST2). .

Next, the failure diagnosis section 11 reads the expected values of the test chart image stored in the storage device 6 (step ST3).
Next, the failure diagnosis unit 11 reads the test chart imaging data stored in the storage device 6 in step ST2 (step ST4).

Next, the fault diagnosis unit 11 diagnoses a fault of the imaging device 5 by comparing the read test chart imaging data (test chart image) with the read expected value (step ST5). In the first embodiment, the failure diagnosis section 11 diagnoses the soundness of the image sensor of the imaging device 5 in step ST5.

When the failure diagnosis unit 11 diagnoses that the imaging device 5 is normal in step ST5, the expected value of the test chart image stored in the storage device 6 is used as the expected value of the test chart image in the imaging device 5. It is replaced with imaging data (step ST6). That is, the failure diagnosis unit 11 updates the expected values of the test chart 2 stored in the storage device 6 .

When the failure diagnosis unit 11 diagnoses that the imaging device 5 is abnormal in step ST5, it notifies the sub CPU (not shown) that the imaging device 5 is abnormal (step ST7). The sub CPU notifies an external ECU (Electronic Control Unit) (not shown) of the abnormality of the imaging device 5 .

Further, when the function of correcting the reflection of the test chart 2 in the subject image is turned on, the imaging device failure diagnosis control section 7 executes the following steps.
First, the switching control unit 10 performs control so that the test chart irradiation light emitting device 3 is turned off (step ST8). A faint light emitted by the test chart irradiation light emitting device 3 not being completely extinguished passes through the test chart 2 , propagates toward the half mirror 4 , and is reflected by the half mirror 4 . The imaging device 5 images the test chart 2 by receiving the light. The test chart image thus obtained is used as reflection data of the test chart 2 in the subject image.

Next, the imaging device fault diagnosis control unit 7 writes the reflection data obtained by the imaging by the imaging device 5 into the storage device 6 (step ST9).
Next, the subject image correction unit 12 reads the reflection data stored in the storage device 6 (step ST10).

Next, the subject image correction unit 12 creates reflection mask data based on the read reflection data (step ST11). The reflection mask data is data used when the subject image correction unit 12 corrects the reflection of the test chart 2 in the subject image.
Next, the subject image correction unit 12 writes the created reflection mask data into the storage device 6 (step ST12).

The imaging device failure diagnosis method by the imaging device failure diagnosis system 100 at startup will be described below.
As shown in FIG. 3, when the imaging device failure diagnosis system 100 is activated, the switching control unit 10 of the imaging device failure diagnosis control unit 7 causes the test chart irradiation light emitting device 3 to irradiate the test chart 2 with light. control (step ST20). As a result, light (second light) emitted by the test chart irradiation light emitting device 3 passes through the test chart 2 , propagates toward the half mirror 4 , and is reflected by the half mirror 4 . Then, the imaging device 5 captures an image of the test chart 2 by receiving the light.

Next, the imaging device failure diagnosis control unit 7 writes the test chart image obtained by imaging the test chart 2 with the imaging device 5 into the storage device 6 as test chart imaging data at the time of activation (step ST21).
Next, the failure diagnosis section 11 reads the test chart imaging data stored in the storage device 6 in step ST21 (step ST22).

Next, the failure diagnosis section 11 reads the expected value of the test chart image stored in the storage device 6 (step ST23). The expected value may be the expected value updated by the failure diagnosis unit 11 in step ST6, or as described above, the theoretical value, the digital data that is the basis of the test chart 2, or the test chart of the reference machine. Data obtained by photographing 2 may also be used.

Next, the fault diagnosis unit 11 diagnoses a fault of the imaging device 5 by comparing the read test chart imaging data (test chart image) with the read expected value (step ST24). Also here, the failure diagnosis unit 11 diagnoses the soundness of the image sensor of the imaging device 5 .

If the imaging device failure diagnosis control unit 7 diagnoses that the imaging device 5 is normal in step ST24 and the function of correcting the reflection of the test chart 2 in the subject image is turned on, the following steps from step ST25 are performed. Each step up to ST29 is executed.

First, the switching control unit 10 performs control so that the test chart irradiation light emitting device 3 is turned off (step ST25). A faint light emitted by the test chart irradiation light emitting device 3 not being completely extinguished passes through the test chart 2 , propagates toward the half mirror 4 , and is reflected by the half mirror 4 . The imaging device 5 images the test chart 2 by receiving the light. The test chart image thus obtained is used as reflection data of the test chart 2 in the subject image.

Next, the imaging device failure diagnosis control unit 7 writes the reflection data obtained by the imaging by the imaging device 5 into the storage device 6 (step ST26).
Next, the subject image correction unit 12 reads the reflection data stored in the storage device 6 in step ST26 (step ST27).

Next, the subject image correction unit 12 creates reflection mask data based on the read reflection data (step ST28).
Next, the subject image correction section 12 writes the created reflection mask data into the storage device 6 (step ST29). As described above, the subject image correction unit 12 may create the reflection mask data each time it is activated.

On the other hand, when the fault diagnosis unit 11 diagnoses that the imaging device 5 is abnormal in step ST24, it notifies the sub CPU (not shown) that the imaging device 5 is abnormal (step ST30). The sub CPU notifies an external ECU (Electronic Control Unit) (not shown) of the abnormality of the imaging device 5 .

The imaging device failure diagnosis method by the imaging device failure diagnosis system 100 during normal operation will be described below.
As shown in FIG. 4, during normal operation of the imaging device failure diagnosis system 100, the switching control unit 10 of the imaging device failure diagnosis control unit 7 controls the subject illumination light emitting device 1 to irradiate the subject with light. (Step ST40). As a result, the light (first light) emitted by the subject illumination light emitting device 1 is reflected by the subject, propagates toward the half mirror 4 , and passes through the half mirror 4 .
Next, the imaging device fault diagnosis control unit 7 controls the imaging device 5 to capture an image of the subject by receiving the light (step ST41).

Next, when the function for correcting the reflection of the test chart 2 in the subject image is turned on, the following steps ST42 to ST45 are executed.
First, the subject image correction unit 12 acquires a subject image (image data) obtained by imaging the subject with the imaging device 5 in step ST41 (step ST42).

Next, the subject image correction unit 12 acquires the reflection mask data written in the storage device 6 in step ST29 (step ST43).
Next, the subject image correction unit 12 corrects the reflection of the test chart 2 in the subject image based on the obtained reflection mask data (step ST44).

Next, the subject image correction unit 12 outputs the subject image after reflection correction to a mechanism (not shown) that detects the state of the subject (driver) (step ST45). The mechanism detects the state of the subject based on the subject image after reflection correction.

As a step subsequent to step ST41 or step ST45, the switching control section 10 controls the test chart irradiation light emitting device 3 to irradiate the test chart 2 with light (step ST46). As a result, light (second light) emitted by the test chart irradiation light emitting device 3 passes through the test chart 2 , propagates toward the half mirror 4 , and is reflected by the half mirror 4 . Then, the imaging device 5 captures an image of the test chart 2 by receiving the light. In step ST46, the subject irradiation light emitting device 1 is turned off.

Next, the imaging device failure diagnosis control unit 7 writes the test chart image obtained by imaging the test chart 2 with the imaging device 5 in the storage device 6 as test chart imaged data (step ST47).

Next, the failure diagnosis section 11 reads the test chart image data stored in the storage device 6 in step ST47 (step ST48).
Next, the failure diagnosis section 11 reads the expected value of the test chart image stored in the storage device 6 (step ST49). The expected value may be the expected value updated by the failure diagnosis unit 11 in step ST6, or as described above, the theoretical value, the digital data that is the basis of the test chart 2, or the test chart of the reference machine. Data obtained by photographing 2 may also be used.

Next, the fault diagnosis section 11 diagnoses a fault of the imaging device 5 by comparing the read test chart imaging data (test chart image) with the read expected value (step ST50). Also here, the failure diagnosis unit 11 diagnoses the soundness of the image sensor of the imaging device 5 .

When the image pickup device failure diagnosis control section 7 diagnoses that the image pickup device 5 is normal in step ST50, the process returns to step ST40, and the steps from step ST40 to step ST50 are repeatedly executed.

When the imaging device failure diagnosis control unit 7 diagnoses that the imaging device 5 is abnormal in step ST50, it notifies the sub CPU (not shown) that the imaging device 5 is abnormal (step ST51). The sub CPU notifies an external ECU (Electronic Control Unit) (not shown) of the abnormality of the imaging device 5 .

A specific example of the half mirror 4 according to the first embodiment will be described below with reference to the drawings. FIG. 5 is a schematic diagram for explaining a specific example of the half mirror 4. As shown in FIG. The diagram on the left side of FIG. 5 shows the function of one side of the light emitting device 1 for illuminating a subject and transmitting the first light reflected by the subject, and the diagram on the right side of FIG. The function of the other surface from which the light-emitting device 3 emits and reflects the second light that has passed through the test chart 2 is shown.

As shown in FIG. 5, the direction of incidence of the first light on the half mirror 4 and the direction of incidence of the second light on the half mirror 4 are orthogonal, and the half mirror 4 has one surface and the first surface. They are arranged so that the angle formed by the incident direction of light is an acute angle and the angle formed by the other surface and the incident direction of the second light is an acute angle.

In the example shown on the left side of FIG. 5 , the first light emitted from the subject illumination light emitting device 1 (not shown) and reflected by the subject enters one surface of the half mirror 4 , and part of the light is reflected by the half mirror 4 . part of the light is reflected by the half mirror 4 and most of the light is transmitted through the half mirror 4 .

In the example of the diagram on the right side of FIG. 5, the second light emitted from the test chart irradiation light emitting device 3 (not shown) and passed through the test chart 2 (not shown) is incident on the other surface of the half mirror 4 and partially is absorbed by the half mirror 4, part of it is transmitted through the half mirror 4, and most of it is reflected by the half mirror 4.
Thereby, the first light transmitted through the half mirror 4 and the second light reflected by the half mirror 4 propagate toward the imaging device 5 .

In this way, by using the respective characteristics of one surface and the other surface of the half mirror 4 to control the light emission and extinguishment of the light emitting device 1 for illuminating the object and the light emission and extinguishment of the test chart 2, respectively, the object or the test chart can be illuminated. Either one of the two can be selectively photographed.

A specific example of the imaging device failure diagnosis method by the imaging device failure diagnosis system 100 according to the first embodiment will be described below with reference to the drawings. FIG. 6 is a schematic diagram for explaining the imaging device failure diagnosis method by the imaging device failure diagnosis system 100 (illustration of the storage device 6 and the imaging device failure diagnosis control unit 7 is omitted). Note that, in the specific example, the imaging device 5 includes a lens 20 and an image sensor 21 . Also, in this specific example, the function for correcting reflection of the test chart 2 in the subject image is turned off.

As shown in FIG. 6, the test chart 2 is installed in the direction in which the first light emitted from the light emitting device 3 for test chart irradiation is emitted. The half mirror 4 is installed at a position where the first light that has passed through the test chart 2 and the second light that is emitted from the subject illumination light emitting device 1 and reflected by the subject (not shown) intersects at right angles.

The lens 20 of the imaging device 5 is installed at a position in the common propagation direction of the first light transmitted through one surface of the half mirror 4 and the second light reflected by the other surface of the half mirror 4. there is The image sensor 21 of the imaging device 5 is installed at a position in the propagation direction of the first light and the second light that have passed through the lens 20 .

First, in step ST40 described above, the switching control unit 10 controls the subject illumination light emitting device 1 to irradiate the subject with light. As a result, the first light emitted by the subject illumination light emitting device 1 is reflected by the subject (not shown), propagates toward the half mirror 4 , and passes through the half mirror 4 . The first light that has passed through the half mirror 4 passes through the lens 20 and is received by the image sensor 21 . Then, in step ST41 described above, the imaging device failure diagnosis control unit 7 controls the imaging device 5 to capture an image of the subject. In this specific example, the imaging device failure diagnosis control unit 7 monitors whether the driver is looking away or dozing off based on the subject image obtained by imaging the driver as the subject by the imaging device 5. , to issue a warning as necessary via a speaker (not shown). Here, since the light-emitting device 3 for illuminating the test chart is turned off, the test chart 2 is not reflected in the subject image.

Next, in step ST<b>46 described above, the switching control unit 10 controls the test chart irradiation light emitting device 3 to irradiate the test chart 2 with light. As a result, the second light emitted by the test chart irradiation light emitting device 3 passes through the test chart 2 , propagates toward the half mirror 4 , and is reflected by the half mirror 4 . Then, the imaging device 5 captures an image of the test chart 2 by receiving the light. In this case, since the light-emitting device 1 for illuminating the object is turned off, the object is not reflected in the test chart image obtained by imaging the test chart 2 with the imaging device 5 .

Next, in step ST47 described above, the imaging device failure diagnosis control unit 7 writes the test chart image obtained by imaging the test chart 2 with the imaging device 5 in the storage device 6 as test chart imaging data. Next, at step ST48, the fault diagnosis unit 11 reads the test chart imaging data stored in the storage device 6 at step ST47. Next, in step ST49, the failure diagnosis section 11 reads the expected values of the test chart image stored in the storage device 6. FIG.

Next, in step ST50, the fault diagnosis unit 11 diagnoses a fault of the imaging device 5 by comparing the read test chart image data (test chart image) with the read expected value.

According to the configuration of the specific example, it is possible to frequently perform failure diagnosis of the imaging device 5 including the lens 20 and the image sensor 21 (photoelectric conversion element). The reliability of failure diagnosis can be ensured without the need to spend a large cost such as installing two sensors.

A configuration in which the imaging device failure diagnosis system 100 according to Embodiment 1 further includes the specific wavelength blocking device 30 will be described below with reference to the drawings. FIG. 7 is a schematic diagram showing a configuration in which the imaging device failure diagnosis system 100 according to Embodiment 1 further includes a specific wavelength blocking device 30 (illustration of the storage device 6 and the imaging device failure diagnosis control unit 7 is omitted). ).

The specific wavelength blocking device 30 is installed in front of one surface of the half mirror 4 on which the first light emitted by the subject irradiation light emitting device 1 and reflected by the subject is incident. The specific wavelength blocking device 30 transmits the first light and blocks light of wavelengths other than the wavelength of the first light. Examples of the specific wavelength blocking device 30 include a bandpass filter (optical filter) and the like.

In this example, for example, the subject irradiation light emitting device 1 irradiates the subject with light of a specific wavelength other than visible light, such as near-infrared light, which is not often contained in sunlight. In that case, the specific wavelength blocking device 30 transmits light of the specific wavelength and blocks light of wavelengths other than the specific wavelength. This makes it possible to eliminate the influence of outside light (sunlight, streetlights, etc.) while the vehicle is running. In addition, there is an effect that the driver is not conscious of light emission and photographing.

When photographing the test chart, the light emitting device 3 for illuminating the test chart irradiates the test chart 2 with light, passes through the test chart 2, and the second light reflected by the half mirror 4 is captured by the image sensor 21 as an image. be At this time, the subject irradiation light emitting device 1 does not emit light, light of wavelengths other than a specific wavelength in the sunlight is blocked by the specific wavelength blocking device 30 , and the light reflected on the subject is reflected by the half mirror 4 . not reach. This makes it possible to reduce the reflection of the subject in the test chart image.

A specific example of the reflection correction method in the imaging device failure diagnosis method by the imaging device failure diagnosis system 100 according to the first embodiment will be described below with reference to the drawings. FIG. 8 is a diagram for explaining how the test chart 2 is reflected in the subject image in the specific example of the first embodiment. In FIG. 8, from the top, a graph showing the time course of the light emission amount of the light emitting device 3 for illuminating the test chart, a graph showing the time course of the light emission amount of the subject illumination light emitting device 1, and the light emission amount of the test chart illumination light emitting device 3 , and the graph showing the time course of the amount of light emitted from the light emitting device 1 for illuminating the object, and the time course of the image captured by the imaging device 5 are shown in order.

First, in step ST46 described above, the switching control section 10 controls the test chart irradiation light emitting device 3 to irradiate the test chart 2 with light. Then, after steps ST47 to ST50 are executed, the process returns to step ST40, and the switching control section 10 controls the subject irradiation light emitting device 1 to irradiate the subject with light.

As a result, as indicated by the arrow in A in FIG. 8, the light emitting device 3 for illuminating the subject may start emitting light before the light emitting device 3 for illuminating the test chart is turned off in time. In such a case, or in a case where the light from the outside of the imaging device failure diagnosis system 100 cannot be completely blocked, there is a possibility that the test chart 2 will be reflected in the subject image obtained by imaging by the imaging device 5. be. Therefore, the subject image correction unit 12 corrects the subject image as follows.

FIG. 9 is a diagram for explaining reflection data (reference image) in the specific example. The solid line in the upper graph of FIG. 9 indicates the time course of the light emission amount of the light emitting device 3 for illuminating the test chart (the dotted line indicates the time course of the light emission amount when the light emitting device 1 for subject irradiation emits light), and the lower graph. 2 shows the passage of time of images obtained by imaging by the imaging device 5. FIG.

In step ST8 or step ST25 described above, the switching control unit 10 controls the light emitting device 3 for test chart irradiation to turn off while the light emitting device 3 for test chart irradiation is emitting light. At this time, the subject illumination light emitting device 1 does not emit light. A faint light emitted when the test chart illumination light emitting device 3 is not completely extinguished passes through the test chart 2 , propagates toward the half mirror 4 , and is reflected by the half mirror 4 . The imaging device 5 captures an image of the test chart by receiving the light. The test chart image thus obtained is used as reflection data (reference image) of the test chart 2 in the subject image (image on the lower right side of FIG. 9).

Then, in step ST11 or step ST28 described above, the subject image correction unit 12 creates reflection mask data based on the reflection data. Then, in step ST42 described above, the subject image correction unit 12 corrects the reflection of the test chart 2 in the subject image based on the reflection mask data.

It should be noted that there is a possibility that the positional relationship between the test chart 2 and the image sensor 21 is slightly displaced due to the vibration of the vehicle when photographing while the vehicle is running. Therefore, when the subject image correction unit 12 simply subtracts the reflection mask data from the subject image, the test chart 2 is reflected in the subject image, and the test chart 2 is not reflected in the subject image. The mask data will be subtracted, and the image quality will be degraded.

As a countermeasure against this, in the specific example, the test chart 2 has a marker 40 for alignment. FIG. 10 is a schematic diagram showing a configuration in which the test chart 2 has markers 40 for alignment. As shown in FIG. 10, in this specific example, the test chart 2 has a rectangular shape, and alignment markers 40 are provided at the four corners of the rectangular shape.

11A and 11B are diagrams for explaining a reflection correction method according to a specific example of Embodiment 1. FIG. In FIG. 11, from the top, a graph showing the time course of the light emission amount of the light emitting device 3 for illuminating the test chart, a graph showing the time course of the light emission amount of the subject illumination light emitting device 1, and the light emission amount of the test chart illumination light emitting device 3 and the time course of the amount of light emitted from the light emitting device 1 for illuminating the object, and the time course of the image obtained by imaging with the imaging device 5, the reflection data, and the reflection mask data. .

First, the subject image correction unit 12 corrects the test chart 2 based on the marker 40 of the test chart image obtained by capturing the test chart 2 with the imaging device 5 immediately before the reflection correction (immediately before step ST42). is specified (displacement detection in FIG. 11). Next, the subject image correction unit 12 aligns the reflection mask data based on the specified installation position of the test chart 2, and aligns the subject image from the subject image in which the reflection of the test chart 2 occurs. Subtract the reflection mask data (reflection mask data subtraction in FIG. 11). As a result, it is possible to reduce the influence of the misalignment of the installation position of the test chart 2 on the reflection correction.

A configuration in which the imaging device failure diagnosis system 100 according to Embodiment 1 includes a plurality of test charts 2, a plurality of test chart irradiation light emitting devices 3, and a plurality of half mirrors 4 will be described below with reference to the drawings. . FIG. 12 is a schematic diagram for explaining a configuration in which the imaging device failure diagnosis system 100 according to Embodiment 1 includes a plurality of test charts 2, a plurality of test chart irradiation light emitting devices 3, and a plurality of half mirrors 4. (illustration of the storage device 6 and the imaging device failure diagnosis control unit 7 is omitted).

The configuration of the imaging device failure diagnosis system 100 shown in FIG. 12B is similar to the configuration of the imaging device failure diagnosis system 100 further including the specific wavelength blocking device 30 in FIG. The configuration of the imaging device failure diagnosis system 100 shown in FIG. 12C is a configuration in which the light shielding plate 53 is installed in the imaging device 5 in the imaging device failure diagnosis system 100 shown in FIG. 12B. A light shielding plate 53 indicated by C in FIG. 12 blocks light other than the first light transmitted through the half mirror 4 and the second light reflected by the half mirror 4 . More specifically, for example, the light shielding plate 53 shown by C in FIG.

In the example of FIG. 12C , the light shielding plate 53 is configured to reduce the influence of light other than the first light transmitted through the half mirror 4 and the second light reflected by the half mirror 4 in the imaging by the imaging device 5 . can be suppressed.

Each configuration of the imaging device failure diagnosis system 100 indicated by D, E, and F in FIG. It is the configuration provided.

More specifically, in each example of D, E, and F in FIG. 12 , the imaging device failure diagnosis system 100 further includes a test chart 50 different from the test chart 2 . In addition, the imaging device failure diagnosis system 100 further includes another test chart irradiation light emitting device 51 for irradiating another test chart 50 with light. In addition, the imaging device failure diagnosis system 100 emits light emitting device 3 for illuminating the test chart, one surface for transmitting the second light that has passed through the test chart 2 , and another light emitting device 51 for illuminating test chart. and the other half mirror 52 that reflects the third light that has passed through another test chart 50 .

Also, the other surface of the half mirror 4 in each example of D, E and F in FIG. are reflected respectively. Further, the imaging device 5 in each example of D, E, and F in FIG. 12 further captures another test chart 50 by further receiving the third light reflected by the half mirror 4 .

Further, the switching control unit 10 (not shown) in each example of D, E, and F of FIG. 3 controls switching between a second mode in which the test chart 2 is illuminated with light and a third mode in which another test chart illumination light emitting device 51 illuminates another test chart 50 .

As shown in each example of D, E, and F in FIG. 12, the arrangement of the combination of another test chart 50, another test chart irradiation light emitting device 51, and another half mirror is the above-mentioned combination. It is not particularly limited as long as it functions.

Further, in each example of D, E, and F in FIG. 12, the imaging device failure diagnosis system 100 sets a combination of another test chart 50, another test chart irradiation light emitting device 51, and another half mirror 52. Only the configuration provided is described. However, the imaging device failure diagnosis system 100 may include a plurality of combinations of different test charts 50 , different test chart illuminating light emitting devices 51 , and different half mirrors 52 . In that case, yet another test chart irradiation light emitting device irradiates still another test chart with further light, and yet another half mirror emits the test chart irradiation light emitting device 3 and passes through the test chart 2. A further test chart illumination light emitting device has one surface for transmitting the second light and another surface for reflecting the further light that has passed through the further test chart. Moreover, in that case, the half mirror 4 further reflects further light reflected by another half mirror. Moreover, in that case, the imaging device 5 further captures another test chart by receiving further light reflected by the half mirror 4 . Further, in that case, the switching control unit 10 (not shown) switches from or to a further mode in which a further test chart irradiation light emitting device irradiates a further test chart with light. More control.

According to the above configuration, by appropriately using a plurality of types of test charts as the above test charts, it is possible to change the contents of the image quality evaluation, and use the optimum chart according to the item to be evaluated. can improve the performance of diagnostic functions.

Note that in the configuration in which the imaging device failure diagnosis system 100 includes a plurality of light emitting devices as described above, it is conceivable that the light emitted from each light emitting device may be reflected on the image sensor along a different optical path than the intended optical path. be done. Therefore, the imaging device failure diagnosis system 100 in each of the examples of D, E, and F in FIG. 12 may further include the light shielding plate 53 described above. The configuration of the imaging device failure diagnosis system 100 indicated by G in FIG. 12 is a configuration in which the light blocking plate 53 is installed in the imaging device 5 in the imaging device failure diagnosis system 100 indicated by D in FIG. 12 . The configuration of the imaging device failure diagnosis system 100 indicated by H in FIG. 12 is a configuration in which the light blocking plate 53 is installed in the imaging device 5 in the imaging device failure diagnosis system 100 indicated by E in FIG. 12 .

A light shielding plate 53 indicated by G and H in FIG. 12 blocks light other than the first light transmitted through the half mirror 4 and the second light and third light reflected by the half mirror 4 . More specifically, for example, the light shielding plate 53 indicated by G and H in FIG. Note that when the imaging device failure diagnosis system 100 includes a plurality of combinations of different test charts 50, different test chart irradiation light emitting devices 51, and different half mirrors 52, the imaging device failure diagnosis system 100 can A light shielding plate other than the plate 53 may be further provided. In that case, for example, the other light blocking plate blocks the light emitted from the still another test chart illumination light-emitting device that has not passed through the still another test chart.
According to the above configuration, it is possible to suppress reflection in the subject image or the test chart image due to unintended light leakage.

Each function of the switching control unit 10, the failure diagnosis unit 11, and the subject image correction unit 12 in the imaging device failure diagnosis control unit 7 is realized by a processing circuit. That is, the imaging device failure diagnosis control section 7 includes a processing circuit for executing the processing of each step shown in FIGS. This processing circuit may be dedicated hardware, or may be a CPU (Central Processing Unit) that executes a program stored in a memory.

FIG. 13A is a block diagram showing a hardware configuration that implements the functions of the imaging device fault diagnosis control section 7. As shown in FIG. FIG. 13B is a block diagram showing a hardware configuration for executing software that implements the functions of the imaging device fault diagnosis control section 7. As shown in FIG.

If the processing circuit is the dedicated hardware processing circuit 101 shown in FIG. 13A, the processing circuit 101 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC Circuit), FPGA (Field-Programmable Gate Array), or a combination thereof.

The functions of the switching control section 10, the fault diagnosis section 11, and the object image correction section 12 in the imaging apparatus fault diagnosis control section 7 may be realized by separate processing circuits, or these functions may be integrated into one processing circuit. can be realized with

When the processing circuit is the processor 102 shown in FIG. 13B, each function of the switching control unit 10, the failure diagnosis unit 11, and the subject image correction unit 12 in the imaging device failure diagnosis control unit 7 is software, firmware, or software and firmware. realized by a combination of
Software or firmware is written as a program and stored in the memory 103 .

The processor 102 implements the functions of the switching control section 10 , the fault diagnosis section 11 and the object image correction section 12 in the imaging device fault diagnosis control section 7 by reading and executing the programs stored in the memory 103 . That is, the imaging device failure diagnosis control unit 7 stores a program that results in the processing of each step shown in FIGS. 2, 3, and 4 when these functions are executed by the processor 102. A memory 103 is provided for

These programs cause a computer to execute each procedure or method of the switching control section 10, the fault diagnosis section 11, and the subject image correction section 12 in the imaging apparatus fault diagnosis control section 7. FIG. The memory 103 may be a computer-readable storage medium storing a program for causing a computer to function as the switching control section 10, the fault diagnosis section 11, and the subject image correction section 12 in the imaging apparatus fault diagnosis control section 7. .

The processor 102 corresponds to, for example, a CPU (Central Processing Unit), a processing device, an arithmetic device, a processor, a microprocessor, a microcomputer, or a DSP (Digital Signal Processor).

The memory 103 includes, for example, non-volatile or volatile semiconductor memories such as RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically-EPROM), Magnetic discs such as hard discs and flexible discs, flexible discs, optical discs, compact discs, mini discs, CDs (Compact Discs), DVDs (Digital Versatile Discs), and the like.

Some of the functions of the switching control unit 10, the failure diagnosis unit 11, and the subject image correction unit 12 in the imaging device failure diagnosis control unit 7 may be implemented by dedicated hardware, and some may be implemented by software or firmware. good.

For example, the function of the switching control unit 10 is realized by a processing circuit as dedicated hardware. The functions of the fault diagnosis section 11 and the subject image correction section 12 may be implemented by the processor 102 reading and executing programs stored in the memory 103 .
As such, the processing circuitry may implement each of the above functions in hardware, software, firmware, or a combination thereof.

As described above, the imaging device failure diagnosis system 100 according to the first embodiment includes the object irradiation light emitting device 1 for irradiating the object with light, the test chart 2 for imaging device failure diagnosis, and the test chart 2. The light-emitting device 3 for irradiating the test chart and the light-emitting device 1 for illuminating the subject emit one surface through which the first light reflected by the subject is transmitted, and the light-emitting device 3 for illuminating the test chart is emitted, A half mirror 4 having the other surface that reflects the second light that has passed through the test chart 2; an imaging device 5 for capturing an image of a test chart 2 by receiving second light; and a switching control unit 10 for controlling switching between a second mode for irradiating light on the light.

According to the above configuration, by switching between the illumination of the subject and the illumination of the test chart 2, it is possible to switch between the photographing of the subject and the photographing of the imaging device 5 for failure diagnosis. Switching can be done. Therefore, photography for failure diagnosis of the imaging device 5 can be frequently performed.

Further, for example, in the above configuration, by taking images of the test chart 2 at regular intervals between shootings of the subject, shooting for fault diagnosis of the imaging device 5 can be performed frequently without affecting the shooting of the subject. It can be carried out.

Further, for example, one of the sensors required for automatic driving vehicles is an image sensor (CMOS, CCD, etc.). In order to establish automatic driving, the image sensor is required to perform highly accurate photographing processing. Therefore, by adopting the above-described configuration, it is possible to frequently perform image capturing for failure diagnosis of the image sensor. It is also possible to maintain

The imaging device failure diagnosis system 100 according to the first embodiment further includes a failure diagnosis unit 11 that diagnoses a failure of the imaging device 5 based on the test chart image obtained by imaging the test chart 2 with the imaging device 5. I have.
According to the above configuration, failure diagnosis of the imaging device 5 can be frequently performed for the reasons described above.

The imaging device failure diagnosis system 100 according to Embodiment 1 further includes a storage device 6 that stores the expected values of the test chart images. The failure of the imaging device 5 is diagnosed by comparing the .
According to the above configuration, failure diagnosis of the imaging device 5 can be preferably performed by comparing the test chart image and the expected value.

In the imaging device failure diagnosis system 100 according to the first embodiment, in front of one surface of the half mirror 4, there is a specific wavelength light that transmits the first light and blocks light with a wavelength other than the wavelength of the first light. A blocking device 30 is installed.
According to the above configuration, it is possible to suppress the influence of light having a wavelength other than the wavelength of the first light when photographing an object or photographing the test chart 2 .

The imaging device failure diagnosis system 100 according to the first embodiment further includes a subject image correction unit 12 that corrects reflection of the test chart 2 in the subject image obtained by imaging the subject with the imaging device 5. .
According to the above configuration, reflection of the test chart 2 in the subject image can be removed.

The test chart 2 in the imaging device failure diagnosis system 100 according to the first embodiment has markers 40 for alignment.
According to the above configuration, in glare correction, by appropriately performing alignment based on the markers 40, the influence of the displacement of the installation position of the test chart 2 on the glare correction can be reduced.

The imaging device failure diagnosis system 100 according to the first embodiment includes a test chart 50 different from the test chart 2, a separate test chart irradiation light emitting device 51 for irradiating the separate test chart 50 with light, and a test chart irradiation system. one surface that transmits the second light that has passed through the test chart 2 from which the light emitting device 3 for emitting light is emitted; and the other half mirror 52 having the other surface that reflects light, the other surface of the half mirror 4 being the second light transmitted through the other half mirror 52 and the other half mirror 52 The imaging device 5 further captures another test chart 50 by further receiving the third light reflected by the half mirror 4, and the switching control unit 10 selects the subject. A first mode in which the illumination light emitting device 1 illuminates the object, a second mode in which the test chart illumination light emitting device 3 illuminates the test chart 2, and another test chart illumination light emitting device 3. It controls switching between a third mode of illuminating another test chart 50 .

According to the above configuration, by appropriately using two types of test charts, it is possible to further switch to photography for another failure diagnosis. Therefore, two types of failure diagnosis can be frequently performed.

In the imaging device failure diagnosis system 100 according to the first embodiment, the imaging device 5 receives light other than the first light transmitted through the half mirror 4 and the second light and third light reflected by the half mirror 4. A light shielding plate 53 is installed to block the
According to the above configuration, it is possible to suppress reflection in the subject image or the test chart image due to unintended light leakage.

The imaging device failure diagnosis system 100 according to the first embodiment includes a plurality of combinations of different test charts 50 , different test chart illuminating light emitting devices 51 , and different half mirrors 52 .
According to the above configuration, it is possible to switch to photography for failure diagnosis of three or more types by using three or more types of test charts as appropriate. Therefore, three or more types of fault diagnosis can be frequently performed.
It should be noted that any component of the embodiment can be modified, or any component of the embodiment can be omitted.

Since the imaging device failure diagnosis system according to the present disclosure can frequently perform imaging for failure diagnosis of the imaging device, it can be used for techniques for diagnosing failures of the imaging device.

1 light-emitting device for object illumination 2 test chart 3 light-emitting device for test chart illumination 4 half mirror 5 imaging device 6 storage device 7 imaging device failure diagnosis control section 10 switching control section 11 failure diagnosis section 12 Subject image corrector, 20 lens, 21 image sensor, 30 specific wavelength blocking device, 40 marker, 50 another test chart, 51 another light emitting device for illuminating test chart, 52 another half mirror, 53 light shielding plate, 100 imaging device Fault diagnosis system, 101 processing circuit, 102 processor, 103 memory.

Claims (9)

a subject irradiation light emitting device for irradiating a subject with light;
a test chart for imaging device failure diagnosis;
a light emitting device for irradiating a test chart that irradiates the test chart with light;
one surface through which the first light emitted from the subject illumination light emitting device and reflected by the subject is transmitted; and a second light emitted from the test chart illumination light emitting device and passing through the test chart. a half mirror having a second surface that reflects the
an imaging device that captures an image of the subject by receiving first light transmitted through the half mirror, and captures an image of the test chart by receiving second light reflected by the half mirror;
Switching control for controlling switching between a first mode in which the subject illumination light emitting device illuminates the subject and a second mode in which the test chart illumination light emitting device illuminates the test chart. A failure diagnosis system for an imaging device, comprising: 2. The apparatus according to claim 1, further comprising a failure diagnosis unit that diagnoses a failure of said imaging device based on a test chart image obtained by imaging said test chart by said imaging device. Imaging device fault diagnosis system. Further comprising a storage device for storing the expected value of the test chart image,
3. The imaging device failure according to claim 2, wherein said failure diagnosis unit diagnoses a failure of said imaging device by comparing said test chart image with an expected value stored in said storage device. diagnostic system. A specific wavelength blocking device is installed in front of the one surface of the half mirror for transmitting the first light and blocking light having a wavelength other than the wavelength of the first light. 2. The image pickup device failure diagnosis system according to claim 1, wherein: 2. The imaging apparatus according to claim 1, further comprising a subject image correction unit that corrects reflection of said test chart in a subject image obtained by said imaging apparatus capturing said subject. Fault diagnosis system. 6. The imaging device failure diagnosis system according to claim 5, wherein said test chart has a marker for alignment. a test chart different from the test chart;
another test chart irradiation light emitting device for irradiating the another test chart with light;
One surface from which the light emitting device for illuminating the test chart is emitted and transmits the second light that has passed through the test chart, and the light emitting device for illuminating the another test chart is emitted and passed through the another test chart. and another half mirror having the other surface that reflects the third light,
the other surface of the half mirror reflects the second light transmitted through the other half mirror and the third light reflected by the other half mirror;
The imaging device further captures an image of the another test chart by further receiving third light reflected by the half mirror;
The switching control unit selects a first mode in which the light emitting device for illuminating the subject irradiates the subject with light, a second mode in which the light emitting device for illuminating the test chart irradiates the test chart with light, and the other modes. 2. The imaging device failure diagnosis system according to claim 1, wherein said light emitting device for illuminating said test chart controls switching between a third mode and a third mode for illuminating said another test chart. The imaging device is provided with a light shielding plate for blocking light other than the first light transmitted through the half mirror and the second light and third light reflected by the half mirror. 8. The image pickup device failure diagnosis system according to claim 7, wherein: 9. The imaging device fault diagnosis system according to claim 8, comprising a plurality of combinations of said another test chart, said another test chart illumination light emitting device, and said another half mirror.

Images