JP5041170B2 - Image processing device - Google Patents

Description

  The present invention relates to an image processing apparatus, and more particularly to an image processing apparatus that acquires a partial image of a subject.

2. Description of the Related Art Conventionally, there is a video image transmitting apparatus that searches for an entire image captured by a camera, extracts only a necessary portion from the entire image, and transmits the extracted portion to a receiving side. For example, in the apparatus described in Patent Document 1, only a necessary part is extracted from an image taken by a camera, only the part is encoded using an encoder, and transmitted by a transmitter. Specifically, the face is detected by searching for an image including a human face photographed by the camera. Next, a face area including the detected face is extracted, and only the extracted area is encoded using an encoder. The encoded image data is transmitted by the transmitter. Thereby, the amount of data to be encoded can be reduced.
JP 2001-218179 A

  As described above, in the apparatus described in Patent Document 1, a necessary partial area is extracted by searching an image photographed by a camera, and only a necessary partial area is encoded. The partial area is preferably a high-quality image in order to perform image recognition processing in detail. For example, when processing an image that captures a person's face and recognizing the open / closed state of eyes or the state of line of sight, the eye position is detected from an image that captures a wide area including the person's face, and the detected part of the eye A detailed image of the area is required. However, if a high-quality image of a wide area including the partial area is captured by a camera in order to obtain a high-quality image of the partial area, and the searched image is searched, the necessary partial area is extracted from the high-quality image. Search and detect. As a result, the amount of data to be subjected to detection processing increases and the processing load on the apparatus increases. Therefore, there is a problem that it takes time to detect the partial area and a problem that the cost is increased because hardware with high processing capability and a large-capacity memory are required. Usually, a relatively low-quality image is sufficient if only a necessary partial area is detected from an image captured by a camera. Therefore, in the method of searching for a high-quality image and extracting a partial region as in the apparatus described in Patent Document 1, an unnecessarily high load is applied during the search. Further, in the conventional method, a wide area image is searched and processed by an image processing unit provided in the imaging unit. However, because the image size is large, the load of the wide area image search process is large, or There may be a case where the imaging unit and the image processing unit cannot be provided together due to a request such as processing the previous raw image by another method. In such a case, a large-capacity transmission cable or a large-capacity buffer memory for transmitting these large images is necessary due to the image size and the like, which is a problem.

  SUMMARY OF THE INVENTION Therefore, an object of the present invention is to provide an image processing apparatus that can extract an image of a necessary partial region from an image including a subject with low load and low cost.

The present invention employs the following configuration in order to solve the above problems. That is, the image processing apparatus according to the present invention includes an image acquisition unit and a target identification unit. The image acquisition means acquires a first image obtained by imaging a predetermined area including a subject and a second image obtained by imaging an area having a resolution higher than that of the first image and narrower than the predetermined area. The target specifying unit specifies a target region to be subjected to image recognition processing from the first image by searching for the first image acquired by the image acquisition unit. The image acquisition unit acquires a second image obtained by imaging the target area specified by the target specifying unit. In addition, after the target specifying unit specifies the target region, the image processing apparatus acquires the first image by the image acquiring unit and stores the coordinate position and the size stored for the target region specified by the target specifying unit. Using the information, the second image is acquired by the image acquiring unit, the first image is acquired by the image acquiring unit, the second image is acquired by the image acquiring unit, and the image recognition process of the second image is performed. May be repeated.

  According to the present invention, when a necessary partial area is searched and extracted from an image obtained by imaging a wide area including a subject, a search can be performed with low load, and a high quality image obtained by imaging the necessary partial area. A partial image can be acquired. Specifically, the image acquisition unit of the image processing apparatus according to the present invention acquires a first image obtained by imaging a wide area including a subject. The object specifying unit specifies a necessary partial area in the wide area corresponding to the first image by searching the first image. And an image acquisition means acquires the 2nd image which imaged the specified partial area as an image with higher resolution than a 1st image. That is, by searching for a first image that captures a relatively low-quality wide area, and acquiring a second image that captures a necessary partial area, the search for an image that captures a high-quality wide area is performed. In comparison, it is possible to search with a low load. Thereby, the load of the apparatus can be reduced. Further, since the data amount of the wide area image to be searched is small, it can be searched with hardware having low processing capability, and the memory capacity can be reduced, so that cost reduction can be realized.

  In the present invention, an imaging means for imaging a subject may further be provided. In this configuration, the image acquisition unit acquires a first image having a resolution lower than that of the captured image by performing a thinning process on the image captured by the imaging unit, and the target is obtained from the image captured by the imaging unit. The second image may be acquired by extracting an area corresponding to the area.

  According to this configuration, the image acquisition unit can acquire a first image having a resolution lower than that of the captured image by performing a thinning process on the image captured by the imaging unit. The target specifying means can specify the target area by searching the first image that has been subjected to the thinning process. And the image acquisition means can acquire the 2nd image whose resolution is higher than a 1st image by extracting the specified area | region from the image imaged by the imaging means. Thereby, when searching for a necessary partial region with a simple configuration, it is possible to search with a low load, and to acquire a partial image with high resolution.

  The present invention may further include an imaging unit that captures an image of an object and an optical zoom unit that enlarges or reduces the object by optical zoom. In this configuration, the image acquisition unit may acquire the second image by capturing an image of the subject enlarged by the optical zoom unit than the first image by the imaging unit.

  According to this configuration, the first image obtained by imaging the wide area including the subject is acquired by widening the imaging range by the optical zoom means, and the necessary partial area is specified by searching the acquired first image. Can do. Then, the second image obtained by imaging the partial area of the subject can be acquired by optically enlarging the specified partial area by the optical zoom unit. As a result, the second image obtained by capturing the details of the necessary partial region can be acquired even using a relatively low-resolution imaging means, and the search can be performed with a low load when searching for the first image. it can.

  In this invention, you may further provide the control means which controls an image acquisition means so that either one of the 2nd image specified by the object specific means or the 1st image may be output. In this configuration, the image acquisition unit is disposed in the imaging unit in which the imaging unit is disposed. The object specifying unit and the control unit are arranged in a camera ECU unit separate from the imaging unit. Then, based on a control signal from the control means, either the first image or the second image may be transmitted from the imaging unit to the camera ECU unit.

  According to this configuration, it is possible to physically separate the imaging unit that images the subject and the camera ECU unit that performs image processing. The camera ECU unit can acquire either the first image obtained by imaging a wide area including the subject or the second image obtained by imaging a partial region of the subject as required for image processing.

  In the present invention, the imaging unit and the camera ECU unit are connected by a transmission cable and a control line, transmit a control signal using the control line, and use the transmission cable from the imaging unit to the camera ECU unit. Either the first image or the second image may be transmitted.

  According to this configuration, an image can be transmitted from the imaging unit to the camera ECU unit using the transmission cable. In addition, a control signal can be transmitted from the camera ECU unit to the imaging unit using the control line.

  In the present invention, the image acquisition unit and the target specifying unit may be disposed in an imaging unit in which the imaging unit is disposed. Then, only the second image may be transmitted from the imaging unit to the camera ECU unit that performs image recognition processing separate from the imaging unit.

  According to this configuration, it is possible to physically separate the imaging unit that captures a partial region of the subject and the camera ECU unit that performs image processing. In addition, since the image transmitted from the imaging unit to the camera ECU unit is only the second image obtained by capturing the partial area of the subject, a delay that occurs during image transmission can be prevented.

  In the present invention, the imaging unit and the camera ECU unit may be connected by a transmission cable, and the second image may be transmitted using the transmission cable.

  According to this configuration, the second image can be transferred from the imaging unit to the camera ECU unit using the transmission cable.

  In the present invention, the image acquisition means may transmit a combined image obtained by combining a plurality of second images into one as the second image to the camera ECU unit.

  According to this configuration, it is possible to efficiently perform image processing performed by the camera ECU unit. That is, an image obtained by capturing a predetermined partial area is arranged at a predetermined position in the combined image, so that the camera ECU unit specifies the position and size of the predetermined partial area and is acquired from the imaging unit. Since the processing to be performed can be omitted, the image processing can be performed efficiently.

  In the present invention, the signal transmitted through the transmission cable may be NTSC.

  According to this configuration, a relatively inexpensive and proven NTSC cable can be used.

  In the present invention, the signal transmitted through the transmission cable may be GVIF or LVDS.

  According to this configuration, the acquired image can be digitally transmitted. In addition, since high-speed transmission is possible, a higher quality image can be transmitted.

  In the present invention, the transmission cable is a differential signal transmission cable with a shield line, and the shield line may be used as a control line.

  According to this configuration, it is possible to omit a control line that connects the imaging unit and the camera ECU unit, and thus it is possible to reduce the cost. Further, by omitting the control line, the wiring space of the cable can be omitted.

  In the present invention, the target specifying means is capable of specifying a plurality of predetermined target areas, selectively searching and specifying a target area necessary for image recognition processing from the plurality of target areas. Also good.

  According to this configuration, when there are a plurality of target areas necessary for the image recognition process, the necessary target areas can be selected and searched. Thereby, only a necessary partial image can be acquired according to the situation.

  In the present invention, the image acquisition means may acquire an image of the driver's face.

  According to this configuration, the driver's face can be estimated by acquiring an image of the driver's face.

  In the present invention, the image acquisition means may acquire an image of the environment around the vehicle.

  According to this structure, it can be used when estimating the situation around the vehicle by acquiring an image of the environment around the vehicle.

  According to the present invention, when a necessary partial area is searched and extracted from an image obtained by imaging a wide area including a subject, a search can be performed with low load, and a high quality image obtained by imaging the necessary partial area. A partial image can be acquired. Thereby, an image obtained by capturing a necessary partial region of the subject can be extracted with low load and low cost.

(First embodiment)
The image processing apparatus according to the first embodiment of the present invention will be described below with reference to FIGS. FIG. 1 is a block diagram illustrating a configuration of an image processing apparatus according to the first embodiment. The image processing apparatus according to the first embodiment is mounted on a vehicle, for example, and is used for photographing a driver's face.

  As shown in FIG. 1, the image processing apparatus 1 includes an imaging sensor 2, a region selection / thinning unit 3, a target specifying unit 4, and a control unit 5. The imaging sensor 2 and the area selection / thinning means 3 are disposed in the imaging unit 11. The object specifying unit 4 and the control unit 5 are disposed in the camera ECU unit 12. The imaging unit 11 is provided with a buffer 6 for temporarily storing images. Similarly, the camera ECU unit 12 is provided with a buffer 7. An image processing unit 10 is provided in the camera ECU unit 12. Further, the imaging unit 11 and the camera ECU unit 12 are connected by a transmission cable 8 and a control line 9. Hereinafter, each part will be described.

(Description of each part of the image processing apparatus 1)
The imaging sensor 2 is imaging means for imaging a subject, and converts light incident from a lens (not shown) into an electrical signal. The imaging sensor 2 captures a wide area around the driver's seat of the vehicle including the face of the driver as the subject. The imaging sensor 2 captures an image with a predetermined resolution. An image captured by the image sensor 2 is temporarily stored in the buffer 6. Specifically, the imaging sensor 2 is a CCD image sensor, a CMOS image sensor, or the like.

  The area selection / thinning means 3 is an image acquisition means for acquiring a wide area image obtained by imaging a predetermined area including a subject and a partial image obtained by imaging an area having a higher resolution than the wide area image and narrower than the predetermined area. . Specifically, the area selection / thinning means 3 is a circuit that executes two processes. In other words, the area selection / thinning means 3 generates an image having a lower resolution than the image captured by the imaging sensor 2 as a wide area image by thinning out an image obtained by imaging the wide area including the driver's face by the imaging sensor 2. To do. Further, only a partial region of the image captured by the image sensor 2 is selected and extracted, and is generated as a partial image.

  Since the wide area image is generated by thinning out the image captured by the image sensor 2, the data amount is smaller than the image captured by the image sensor 2 regardless of the image captured of the wide area. Yes. Here, the thinning-out process is performed, for example, by extracting some pixels from the pixels of the entire image captured by the image sensor 2. For example, there are the following methods. That is, only one pixel is extracted from the four adjacent pixels, and the remaining three pixels are discarded. Then, this process is performed on the entire image picked up by the image sensor 2, and a thinning process is performed by combining the extracted pixels to generate one image. In another method, an average of four adjacent pixels is taken as one pixel, and the entire image captured by the image sensor 2 is processed, and the thinning process is performed by reducing the number of pixels in the entire image. Is called. By the method as described above, an image having a resolution lower than that of the image captured by the image sensor 2 can be generated by a simple process.

  Since the partial image is an image obtained by cutting out a partial region of the image captured by the imaging sensor 2, the partial image is an image having the same image quality (same resolution) as the image captured by the imaging sensor 2. However, since only the partial area is cut out, the image size of the partial image is smaller than the image captured by the imaging sensor 2. The partial image is acquired by, for example, extracting an image of a predetermined size centered on the driver's face center from the image captured by the image sensor 2.

  The object specifying means 4 specifies a desired partial area of the subject to be subjected to image processing by searching the wide area image thinned by the area selecting / thinning means 3. Specifically, the object specifying unit 4 searches the wide area image that has been subjected to the thinning process by the region selection / thinning unit 3 and specifies the driver's face. The identification of the driver's face is performed by a conventional method such as pattern matching, edge detection, or a combination thereof. The target specifying unit 4 obtains the position and size of the driver's face in the thinned wide area image.

  When the object specifying unit 4 searches for a large image and determines the driver's face, a large load is applied. However, in the image processing apparatus 1 according to the present embodiment, since the wide area image is thinned out, the data amount of the image is small. Therefore, even when searching a wide area image and specifying a driver's face, a big load is not applied.

  The control means 5 is an area selection / thinning means for outputting either a partial image obtained by imaging the partial area specified by the target specifying means 4 or a wide area image thinned by the area selection / thinning means 3. 3 is controlled. When the wide area image is necessary, the control means 5 transmits a control signal to the area selection / thinning means 3 so as to output the wide area image via the control line 9. The area selection / thinning means 3 that has received the control signal for outputting the wide area image transmits the wide area image to the control means 5 via the transmission cable 8. On the other hand, when a partial image (driver's face image) is required, the control means 5 sends a control signal to the area selection / thinning means 3 so as to output the partial image via the control line 9. Send. Specifically, the control unit 5 first acquires information on the position and size of the driver's face determined by the target specifying unit 4. Next, a control signal including information on the position and size of the driver's face is transmitted to the region selection / thinning means 3 via the control line 9. The area selection / thinning means 3 that has received the control signal for outputting the partial image transmits the driver's face image, which is a partial image, to the control means 5 via the transmission cable 8.

  The buffer 6 and the buffer 7 are configured by a memory. The buffer 6 temporarily stores the image captured by the image sensor 2 and outputs the stored image according to the request of the area selection / thinning means 3. The buffer 7 temporarily stores the thinned wide area image or partial image output from the area selection / thinning means 3. The buffer 7 outputs a wide area image stored in accordance with a request from the target specifying means 4 in order to specify the target area. Further, the stored partial image is output for image processing by the image processing unit 10.

  The imaging sensor 2 and the area selection / thinning means 3 are disposed in the imaging unit 11. Specifically, the imaging unit 11 is a camera body, and is disposed at a position where the driver's face can be photographed. The imaging unit 11 is disposed, for example, above the steering column. Further, the object specifying unit 4 and the control unit 5 are arranged in the camera ECU unit 12. The camera ECU unit 12 is disposed at a position away from the imaging unit 11.

  The image processing unit 10 of the camera ECU unit 12 performs an image process on the image transmitted from the imaging unit 11 and performs a statistical process, thereby estimating the state of the driver. For example, the image processing unit 10 recognizes the open / closed state of the driver's eyes from the partial image (driver's face image), and determines whether or not the driver is in a normal state. Then, the image processing unit 10 transmits a control signal to the control ECU that controls the vehicle so as to perform a necessary operation according to the driver's state estimation result. Note that the operation performed by the image processing unit 10 is not a feature of the present invention, and thus detailed description thereof is omitted.

  The imaging unit 11 and the camera ECU unit 12 are connected by a transmission cable 8 and a control line 9. The transmission cable 8 is a cable for transmitting an image from the imaging unit 11 to the camera ECU unit 12. The control line 9 is a cable for transmitting a control signal from the camera ECU unit 12 to the imaging unit 11. The transmission cable 8 is, for example, an NTSC cable, and a signal transmitted from the imaging unit 11 to the camera ECU unit 12 is an NTSC signal. The transmission cable 8 may be a differential signal transmission cable with a shield wire. In this case, the analog signal of the image is converted into a digital signal in the imaging unit 11 and transmitted from the imaging unit 11 to the camera ECU unit 12 by a GVIF or LVDS signal. The imaging unit 11 and the camera ECU unit 12 do not need to be physically connected by the transmission cable 8 and the control line 9, and an image and a control signal may be transmitted by wireless communication.

  In the present embodiment, the imaging sensor 2 and the region selection / thinning unit 3 acquire a wide-area image with a low resolution and a partial image with a high resolution. However, in other embodiments, the imaging unit 11 includes an optical sensor. A wide-area image and a partial image may be acquired by including a lens having a zoom function, an image sensor, and a zoom control unit that controls enlargement / reduction of a subject by the optical zoom lens. In this case, when a wide area image including the driver's face is acquired, the wide area including the driver's face is imaged by the imaging sensor by controlling the focal length of the lens to be short (widening the imaging range). Next, when acquiring a partial image that is a driver's face or a part of the driver's face (for example, eyes, nose, etc.), the captured wide area image is searched by the above-described target specifying means 4 and necessary. A partial area (driver's face or driver's face part) is determined. Then, the focal length of the lens is controlled to be longer (the imaging range is narrowed) with respect to the obtained partial area, thereby enlarging the partial area that is the driver's face or the driver's face, and the imaging sensor. Take an image. Thus, by performing optical zoom on the necessary part, the driver's face is enlarged even if the wide-area image and the partial image have the same resolution. The number of pixels per unit area of the face increases. That is, the partial image acquired by the optical zoom becomes an image with higher resolution than the wide area image. Therefore, it is possible to acquire a wide-area image having a lower image quality and a partial image having a higher image quality than the partial image. Although a high-quality partial image can be acquired, even when searching for a wide area image, a large load is not applied as compared to searching for a wide area image having the same image quality as the partial image. Since the position of the driver's face is within an approximate range, it is preferable to adjust the position so that the range can be imaged when zoomed in advance. Further, in the case where the orientation of the imaging unit 11 is variable and a partial image is captured, the orientation of the imaging unit 11 may be controlled so as to be directed toward the partial region to be imaged.

  In other embodiments, the imaging unit 11 including the imaging sensor 2 and the region selection / thinning unit 3 described above includes two imaging sensors having different resolutions and a switching unit that switches between images captured by the two imaging sensors. May be replaced with the imaging unit 11 having. In this configuration, a wide area including the driver is imaged simultaneously by two imaging sensors. Next, the position and size of a necessary partial area are obtained by searching for a wide area image acquired by an imaging sensor with a low resolution by the object specifying means 4. Then, in the image captured by the high-resolution imaging sensor, a partial image having a high resolution is obtained by selecting and extracting a region having the position and size of the necessary partial region obtained by the target specifying unit 4. can do.

(Description of operation of image processing apparatus 1)
Next, the operation of the image processing apparatus 1 will be described with reference to FIGS. FIG. 2 is a flowchart showing an operation flow of the image processing apparatus 1. The process flow according to the flowchart shown in FIG. 2 is controlled by the camera ECU unit 12. The process of the flowchart shown in FIG. 2 is repeatedly executed while the vehicle is running, for example.

  In step S <b> 101, a wide area image including a driver's face is captured by the imaging sensor 2. The imaging sensor 2 always captures a wide area image including a driver's face. The captured wide area image is temporarily stored in the buffer 6.

  FIG. 3 is a diagram illustrating a state in which a wide area image including a driver's face is captured by the image sensor 2. As shown in FIG. 3, the wide area image captured by the imaging sensor 2 is an image including the driver's face, neck, shoulders, and the like. In addition, a driver's seat, a seat belt, and the like may be imaged. Next, step S102 is executed.

  In step S102, the camera ECU unit 12 determines a required image type. If the partial area to be imaged is not fixed, it is determined to acquire a wide area image. If the partial area is fixed, it is determined to acquire the partial area. If a wide area image is necessary, the camera ECU 12 advances the process to step S103. If the partial image is necessary, the camera ECU 12 advances the process to step S106.

  In step S <b> 103, the control unit 5 transmits a control signal to the region selection / thinning unit 3 so that the wide area image captured in step S <b> 101 is thinned out. The area selection / thinning means 3 performs a thinning process on the wide area image captured by the imaging sensor 2.

  FIG. 4 is a diagram showing a wide area image that has been subjected to thinning processing. As shown in FIG. 4, the thinned wide area image is an image obtained by capturing a wide area similar to the image of FIG. 3, but has a smaller number of pixels than the wide area image shown in FIG. It is an image. Therefore, the thinned wide area image has a smaller amount of data than the wide area image of FIG. The wide area image subjected to the thinning process is temporarily stored in the buffer 7 of the camera ECU 12 via the transmission cable 8. Next, the camera ECU unit 12 proceeds to step S104.

  In step S <b> 104, the target specifying unit 4 performs partial area recognition processing. The object specifying unit 4 detects the position of the driver's face by taking out the thinned-out wide area image from the buffer 7 and searching for the thinned-out wide area image.

  FIG. 5 is a diagram illustrating a state in which a driver's face is recognized by searching for a wide area image that has been subjected to the thinning process. The object specifying unit 4 recognizes the driver's face from the thinned-out wide area image by a method such as pattern matching or edge detection, which is a conventional face recognition technique. The object specifying unit 4 is set in advance so as to recognize the driver's face. As illustrated in FIG. 5, when the target specifying unit 4 recognizes the driver's face, the position of the driver's face (position of the face center 13) and the size (partial region 14) in the thinned-out wide area image are recognized. (Diagonal length). Next, the camera ECU unit 12 proceeds to step S105.

  In step S105, the camera ECU 12 determines whether or not the partial image (driver's face image) has been successfully recognized in step S104. If the recognition of the partial image is successful, the camera ECU 12 advances the process to step S106. If the recognition of the partial image fails, the camera ECU unit 12 returns the process to step S101. Note that the processing shown in the entire flowchart is one cycle, and the processing from step S103 to step S105 is executed in the first cycle, and the position and size of the partial area are obtained. In the second and subsequent cycles, the processes from step S103 to step S105 are executed as necessary (the next process is branched in step S102).

  In step S106, information on the coordinate position and size of the partial image obtained in step S104 is stored as input data in the next step S107. Next, the process of step S107 is performed. If Steps S103 to S105 are not executed, that is, if the image type is determined to be a partial image in Step S102, the information on the coordinate position and size of the partial area is the cycle before this cycle. Already sought and saved. Accordingly, when it is determined in step S102 that the image type is a partial image, in step S106, information on the coordinate position and size of the partial image that has already been stored is used as input data in the next step S107.

  In step S <b> 107, the control unit 5 transmits a control signal to the region selection / thinning unit 3 so as to acquire a partial image. The transmitted control signal includes information on the position and size of the partial image obtained in step S106. The area selection / thinning means 3 takes out a wide area image captured by the imaging sensor 2 from the buffer 6. The retrieved wide area image may be a wide area image that has been subjected to thinning processing, or may be the latest wide area image captured by the imaging sensor 2. Next, the area selection / thinning means 3 extracts the partial image of the position and size transmitted from the control means 5 from the extracted wide area image.

  FIG. 6 is a diagram illustrating a state in which a partial image of the driver's face is extracted from the wide area image captured by the image sensor 2. In FIG. 6, the face center 13 and the partial region 14 are shown at the same position as the position shown in FIG. 5 on the wide area image captured by the imaging sensor 2. The area selection / thinning means 3 extracts the image of the partial area 14 as a partial image.

  FIG. 7 is a diagram illustrating an image of a driver's face as an extracted partial image. A partial image obtained by imaging the extracted partial region 14 shown in FIG. 7 is transmitted to the buffer 7 of the camera ECU unit 12 via the transmission cable 8. Next, the process of step S108 is performed.

  In step S108, the image processing unit 10 performs image recognition processing of the partial image extracted in step S107. Here, in order to estimate the driver's state, various processes are performed to estimate the driver's state. The process in the image processing unit 10 may be any process as long as it is a process for estimating the driver's state using a partial image. Thereafter, the process ends.

  In step S104 for recognizing the necessary partial area 14, the object specifying means 4 not only recognizes the entire face of the driver but also selects and recognizes a face portion such as the driver's eyes and mouth. Also good. Moreover, you may recognize the whole face of a driver | operator and face parts, such as a driver | operator's eyes and a mouth. FIG. 8 is a diagram illustrating a state where the driver's eyes, mouth, and face are selectively searched. The object specifying means 4 is controlled by the control means 5, for example, a partial area 14a of the entire driver's face, a partial area 14b of the driver's right eye, a partial area 14c of the driver's left eye, and a partial area of the driver's mouth. 14d is set to be detected. The object specifying unit 4 selectively searches, for example, the partial region 14b and the partial region 14c, which are partial regions of the driver's eyes, under the control of the control unit 5. In this way, only a necessary part can be extracted by selectively searching and recognizing the face part.

  Further, in step S107 for extracting the necessary partial area 14, when there are a plurality of necessary partial images, one combined image obtained by combining the plurality of partial images may be transmitted to the camera ECU unit 12 as a partial image. . FIG. 9 is a diagram illustrating an example of a combined image obtained by combining a plurality of necessary partial images. In FIG. 9, partial images obtained by imaging a plurality of partial regions (partial regions 14a to 14d) shown in FIG. 8 are combined as one image. In this way, the combined image makes it easy to perform image recognition processing in the subsequent image processing in step S108. That is, when the image processing is performed by dividing the combined image into a plurality of regions in advance and arranging a predetermined face portion (for example, the left eye in the upper left region) in each region, By processing only this area, a partial image of a predetermined face can be processed. In addition, when transmitting a plurality of partial images one by one, it is necessary to specify the position and size of the partial images every time before transmission, but when combining and transmitting a plurality of partial images, the position of the partial images And size information can be specified at a time. Therefore, transmission efficiency is improved.

  The operation of the image processing apparatus 1 has been described above. By thinning out the wide area image picked up by the image sensor 2 and searching for the wide area image that has been thinned out, it is possible to search at a low load and at a high speed and to recognize a necessary partial area. In addition, since it is not necessary to transmit a wide-area image with high resolution from the imaging unit 11 to the camera ECU unit 12, it is possible to prevent signal loss due to an increase in transmission amount. In addition, when transmitting a wide-area image with high resolution, it may be impossible to use an NTSC transmission cable that has been proven in-vehicle due to an increase in transmission amount. However, in this image processing apparatus, a wide-area image is transmitted. However, since the transmission amount does not increase, an NTSC transmission cable can be used. Furthermore, since the transmission amount does not increase, the memory capacity of the buffer 7 of the camera ECU unit 12 can be reduced, and the cost can be reduced.

  Note that the image processing apparatus 1 according to the present embodiment is used to image the driver's face and estimate the driver's state, but can also be used for other purposes. For example, the image processing apparatus 1 captures an environment around a vehicle and is used for recognizing pedestrians, other vehicles, and obstacles around the vehicle. In this case, for example, the imaging unit 11 is disposed at a position where the front of the vehicle can be imaged, and images the front of the vehicle. When used for recognition of other vehicles, the wide area image picked up by the image sensor 2 of the image pickup unit 11 is thinned by the area selection / thinning means 3 and transmitted to the target specifying means 4. The target specifying means 4 searches for the thinned wide area image to recognize an object around the vehicle and specifies the position and size of the object (specifies the target partial area). Further, the region selection / thinning means 3 extracts an image (partial image) of the identified object from the image captured by the imaging sensor 2 and transmits it to the camera ECU unit 12. The image processing unit 10 of the camera ECU unit 12 recognizes whether or not the object is another vehicle by processing the extracted image of the object. In addition, the size and direction of other vehicles are recognized from the extracted image. In accordance with the results of these recognition processes, it is possible to determine what kind of vehicle is present around the vehicle, which can be used for collision prediction and the like.

(Second Embodiment)
Next, an image processing apparatus according to a second embodiment of the present invention will be described with reference to FIG. FIG. 10 is a block diagram illustrating a configuration of an image processing apparatus according to the second embodiment.

  In the image processing device 20 shown in FIG. 10, the same components as those of the image processing device 1 according to the first embodiment are denoted by the same reference numerals, and description thereof is omitted. Hereinafter, the image processing apparatus 20 will be described.

  As shown in FIG. 10, in the image processing apparatus 20, the object specifying unit 4 is disposed in the imaging unit 21. Further, the control means 5 that existed in the first embodiment is deleted from the camera ECU unit 22. In addition, there is no control line connecting the imaging unit 21 and the camera ECU unit 22.

  The area selection / thinning means 3 acquires an image captured by the imaging sensor 2 via the buffer 6 and performs a thinning process. The thinned wide area image is transferred to the target specifying unit 4. As described above, the target specifying unit 4 searches the thinned wide area image and obtains the position and size of the necessary partial area. The object specifying unit 4 transmits information on the position and size of the necessary partial area to the region selection / thinning unit 3. The area selection / thinning means 3 extracts a partial image by the method described above, and transmits only the extracted partial image to the camera ECU unit 12 via the transmission cable 8. The transmitted partial image is subjected to image processing by the image processing unit 10 and estimation of the driver's state is performed.

  As described above, by disposing the target specifying unit 4 in the imaging unit 11, the time for transmitting the wide area image to the camera ECU unit 12 is shortened as compared with the image processing apparatus 1 according to the first embodiment. And delay due to transmission can be prevented. Further, by transmitting only the necessary partial images to the camera ECU unit 12, it is not necessary to control which of the wide area image or the partial image is transmitted. Therefore, the control line 9 of the image processing apparatus 1 according to the first embodiment is not necessary, and cost reduction can be realized.

  In the second embodiment, as in the first embodiment, the imaging unit 21 includes a lens having an optical zoom function, an imaging sensor, and zoom control means for controlling the enlargement / reduction of the subject using the optical zoom lens. The wide area image and the partial image may be acquired. Further, similarly to the first embodiment, the imaging unit 21 may be replaced with two imaging sensors having different resolutions and a switching unit that switches an image captured by the two imaging sensors. Furthermore, it is needless to say that the second embodiment can be used for other purposes described in the first embodiment.

(Third embodiment)
Next, an image processing apparatus according to a third embodiment of the present invention will be described with reference to FIG. FIG. 11 is a block diagram illustrating a configuration of an image processing apparatus according to the third embodiment.

  In the image processing apparatus 30 shown in FIG. 11, the same components as those of the image processing apparatus 1 according to the first embodiment are denoted by the same reference numerals and description thereof is omitted. Hereinafter, the image processing apparatus 30 will be described.

  As shown in FIG. 11, the image processing apparatus 30 has a configuration in which the transmission cable 8 is replaced with the transmission cable 18 and the control line 9 is not provided.

  The transmission cable 18 of the image processing apparatus 30 is a differential signal transmission cable with a shield wire. A signal transmitted from the imaging unit 11 to the camera ECU unit 12 is a differential signal, which is GVIF or LVDS. A control signal from the control unit 5 is transmitted to the region selection / thinning unit 3 of the imaging unit 11 via the shield line of the transmission cable 18. The transmission of the image from the area selection / thinning means 3 of the imaging unit 11 to the camera ECU unit 12 is performed via the transmission cable 18.

  Thereby, the control line 9 which connects the imaging part 11 and camera ECU part 12 can be abbreviate | omitted, cost reduction and a cable laying space can be abbreviate | omitted.

  Note that, in the third embodiment, as in the first embodiment, the wide-area image and the partial image may be acquired using the optical zoom function. Further, similarly to the first embodiment, a configuration having two imaging sensors having different resolutions may be used. Furthermore, it goes without saying that the third embodiment can also be used for other purposes described in the first embodiment.

  As described above, in the present invention, a high-quality image obtained by capturing a necessary partial area of a subject can be acquired with low load and low cost. For example, the present invention can be used as an image processing device mounted on a vehicle. Can do.

1 is a block diagram showing the configuration of an image processing apparatus according to a first embodiment. A flowchart showing a flow of operations of the image processing apparatus 1 according to the first embodiment. The figure which shows a mode that the wide area image including the driver | operator's face was imaged by the imaging sensor Figure showing a thinned wide area image The figure which showed a mode that a driver's face is recognized by searching the wide area image by which thinning processing was carried out. The figure which showed a mode that the partial image of a driver | operator's face was extracted from the wide area image imaged by the imaging sensor The figure which shows the image of the driver's face which is the extracted partial image A diagram showing how to selectively search for the driver's eyes, mouth, and face The figure which shows the example of the combined image which combined several required partial images FIG. 3 is a block diagram illustrating a configuration of an image processing apparatus according to a second embodiment. A block diagram showing composition of an image processing device concerning a 3rd embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1, 20, 30 Image processing apparatus 2 Imaging sensor 3 Area | region selection / thinning-out means 4 Target identification means 5 Control means 6, 7 Buffer 8, 18 Transmission cable 9 Control line 10 Image processing part 11, 21 Imaging part 12, 22 Camera ECU Part 13 Face center 14 Partial area

Claims (14)

Image acquisition means for acquiring a first image obtained by imaging a predetermined area including a subject and a second image obtained by imaging an area having a resolution higher than that of the first image and narrower than the predetermined area;
Searching the first image acquired by the image acquisition means, the target specifying means for specifying a target area to be subjected to image recognition processing from the first image,
The image acquisition means acquires the second image obtained by imaging the target area specified by the target specifying means ,
After the target area is specified by the target specifying means, when the first image is acquired by the image acquiring means, the image is obtained using information on the coordinate position and size stored for the target area specified by the target specifying means. The second image is acquired by the acquisition unit, and the acquisition of the first image by the image acquisition unit, the acquisition of the second image by the image acquisition unit, and the image recognition process of the second image are performed. And an output of the image processing apparatus. It further comprises imaging means for imaging the subject,
The image acquisition unit acquires the first image having a resolution lower than that of the captured image by performing a thinning process on the image captured by the imaging unit, and the image acquisition unit acquires the first image from the image captured by the imaging unit. The image processing apparatus according to claim 1, wherein the second image is acquired by extracting an area corresponding to the target area. Imaging means for imaging the subject;
Optical zoom means for enlarging and reducing the subject by optical zoom,
The said image acquisition means acquires the said 2nd image by imaging the said to-be-photographed object with the said imaging means from the said optical zoom means rather than the said 1st image, The said 2nd image is acquired. Image processing device. Control means for controlling the image acquisition means to output either one of the second image or the first image specified by the target specifying means;
The image acquisition unit is disposed in an imaging unit in which the imaging unit is disposed,
The target specifying unit and the control unit are arranged in a camera ECU unit separate from the imaging unit,
The image according to claim 2, wherein either the first image or the second image is transmitted from the imaging unit to the camera ECU unit based on a control signal from the control unit. The image processing apparatus according to 3.   The imaging unit and the camera ECU unit are connected by a transmission cable and a control line, transmit the control signal using the control line, and use the transmission cable from the imaging unit to the camera ECU unit. The image processing apparatus according to claim 4, wherein either the first image or the second image is transmitted. The image acquisition unit and the target specifying unit are disposed in an imaging unit in which the imaging unit is disposed,
4. The image processing apparatus according to claim 2, wherein only the second image is transmitted from the imaging unit to a camera ECU that performs an image recognition process separate from the imaging unit. 5. .   The image processing apparatus according to claim 6, wherein the imaging unit and the camera ECU unit are connected by a transmission cable, and the second image is transmitted using the transmission cable.   The said image acquisition means transmits the combined image which combined several said 2nd image into one to the said camera ECU part as said 2nd image, The any one of Claim 4 to 7 characterized by the above-mentioned. The image processing apparatus described.   The image processing apparatus according to claim 5, wherein the signal transmitted through the transmission cable is NTSC.   The image processing apparatus according to claim 5 or 7, wherein the signal transmitted through the transmission cable is GVIF or LVDS.   The image processing apparatus according to claim 5, wherein the transmission cable is a differential signal transmission cable with a shield line, and the shield line is used as the control line.   The target specifying unit is capable of specifying a plurality of predetermined target areas, and selectively searching and specifying a target area necessary for image recognition processing from the plurality of target areas. The image processing apparatus according to any one of claims 1 to 11.   The image processing apparatus according to claim 1, wherein the image acquisition unit acquires an image of a driver's face.   The image processing apparatus according to claim 1, wherein the image acquisition unit acquires an image of an environment around a vehicle.

Images