JP7690757B2 - Image Recording Device - Google Patents

Description

The present invention relates to an image recording device.

In recent years, there has been progress in the development of image recording devices that can accurately record the situation when an accident occurs.

Patent document 1 discloses a technology relating to an image processing device that sets a region of interest in an image frame of a moving image captured by an imaging means, detects a region of a moving object in this image frame, determines whether or not at least a part of the detected region of the moving object is included in the region of interest, and controls the encoding of the inside and outside of the region of interest of the moving image depending on the result of this determination.

JP 2019-134323 A

Image recording devices such as drive recorders constantly capture images of the vehicle's surroundings and record images and videos that are collections of images (hereafter simply referred to as images). In recent years, images from image recording devices have become more highly detailed, and as a result, the capacity of image files has also increased.

In the technology disclosed in Patent Document 1, when a moving object is included in a region of interest (ROI), the region of interest is encoded so that the image quality is high. In this way, by selectively setting the image quality of the region of interest to high and setting the image quality of areas other than the region of interest to relatively low quality, it is possible to reduce the size of the image file while improving the image quality of the necessary areas. However, in an image recording device such as a drive recorder, if all areas with movement are set to high quality as in the technology disclosed in Patent Document 1, the entire image will be set to high quality when the vehicle is moving. For this reason, it is necessary to selectively set the image quality of areas that are important to the user, such as objects approaching the vehicle equipped with the image recording device, to high quality.

The present invention has been made in consideration of the above points, and aims to provide an image recording device that can appropriately reduce the capacity of an image file while improving the image quality in necessary areas.

The present invention provides an image recording device including an image acquisition unit that acquires an image of the surroundings of the vehicle, an object recognition unit that recognizes a predetermined object from the image acquired by the image acquisition unit, a judgment reference speed calculation unit that calculates a judgment reference speed using the speed at which the object recognized by the object recognition unit moves relative to the vehicle and the speed of the vehicle, a region of interest extraction unit that extracts, as a region of interest, a region including an object for which the judgment reference speed satisfies a predetermined condition, and an image quality setting unit that sets the image quality of the image of the region of interest extracted by the region of interest extraction unit from the images acquired by the image acquisition unit so that it has higher image quality than images of other regions.

The present invention provides an image recording device that includes an image acquisition unit that acquires an image of the surroundings of the vehicle, an object recognition unit that recognizes a predetermined object from the image acquired by the image acquisition unit and calculates the amount of displacement of the position on the image where the object is recognized, a region of interest extraction unit that extracts an area including the object as a region of interest if the amount of displacement is greater than a predetermined value, and an image quality setting unit that sets the image quality of the image of the region of interest extracted by the region of interest extraction unit from the images acquired by the image acquisition unit so that it has higher image quality than images of other regions.

The present invention provides an image recording device including an image acquisition unit that acquires an image of the surroundings of the vehicle, a motion vector detection unit that detects a motion vector using the image acquired by the image acquisition unit, a region of interest extraction unit that extracts, as a region of interest, a region including a motion vector that satisfies a predetermined condition from among the motion vectors detected by the motion vector detection unit, and an image quality setting unit that sets the image quality of the image of the region of interest extracted by the region of interest extraction unit from the images acquired by the image acquisition unit so that it has a higher image quality than images of other regions.

According to the present disclosure, it is possible to provide an image recording device that can appropriately reduce the size of an image file while improving image quality in necessary areas.

1 is a block diagram showing an example of a configuration of an image recording device according to a first embodiment; 4 is a flowchart for explaining an operation of the image recording device according to the first embodiment. 4 is a diagram for explaining an example of the operation of the image recording device according to the first embodiment; FIG. 4 is a diagram for explaining an example of the operation of the image recording device according to the first embodiment; FIG. 4 is a diagram for explaining an example of the operation of the image recording device according to the first embodiment; FIG. 4 is a diagram for explaining an example of the operation of the image recording device according to the first embodiment; FIG. 4 is a diagram for explaining an example of the operation of the image recording device according to the first embodiment; FIG. 4 is a diagram for explaining an example of the operation of the image recording device according to the first embodiment; FIG. FIG. 11 is a block diagram showing a configuration example of an image recording device according to a second embodiment. FIG. 2 is a block diagram showing an example of a hardware configuration of an image recording device according to the first and second embodiments.

<First embodiment>
Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
Fig. 1 is a block diagram showing a configuration example of an image recording device according to embodiment 1. As shown in Fig. 1, the image recording device 1 according to the embodiment includes an image acquisition unit 11, an object recognition unit 12, a judgment reference speed calculation unit 13, a region of interest extraction unit 14, and an image quality setting unit 15.

The image recording device 1 records the image acquired by the image acquisition unit 11 in the recording unit 18. The image quality of the image recorded at this time is set in the image quality setting unit 15. In this embodiment, the recording unit 18 is a recording medium such as an SD card, a solid state drive (SSD), a hard disk drive (HDD), or a memory. The recording unit 18 may be built into the image recording device 1, or may be configured to be removable from the image recording device 1. The recording unit 18 may also be provided outside the image recording device 1.

The image acquisition unit 11 acquires images of the surroundings of the vehicle. The surroundings of the vehicle may include not only the area in front of the vehicle, but also the areas behind, to the sides, above, and below. The image acquisition unit 11 may also acquire images including the interior of the vehicle. The image acquisition unit 11 may be configured to include a camera, or may be configured to receive an image signal output from a separately provided camera.

The object recognition unit 12 recognizes a predetermined object from the image acquired by the image acquisition unit 11. Specifically, the object recognition unit 12 recognizes objects such as other vehicles and people from the image acquired by the image acquisition unit 11 using image recognition technology. For example, the object recognition unit 12 is provided with an object recognition dictionary that has been trained by supervised machine learning using data on other vehicles and people, and determines whether these objects are included in the image acquired by the image acquisition unit 11. The object recognition unit 12 may recognize a predetermined object from the image acquired by the image acquisition unit 11 using other known image recognition technology. Note that the recognized object may be a guardrail, a building, a roadside tree, a landscape, etc., in addition to objects such as other vehicles and people, and is not limited to these as long as it is an object that is generally recognized as an obstacle to the running of the vehicle. In addition, as described later, the object recognition unit 12 may calculate the amount of displacement of the recognized object on the image. The amount of displacement of the object is a value obtained by acquiring the horizontal and vertical positions (for example, the center position of the recognized object) on the image in which the object is recognized over multiple frames and calculating the maximum change amount in a predetermined period. The amount of displacement is calculated in pixels or as a percentage of the total image screen, and is the amount that a recognized object has moved on the image within a specified period of time.

The judgment reference speed calculation unit 13 calculates the judgment reference speed using the speed at which the object recognized by the object recognition unit 12 moves relative to the vehicle and the speed of the vehicle. For example, the judgment reference speed calculation unit 13 calculates the distance between the object and the vehicle in each frame of the image (not necessarily every frame), and calculates the speed at which the object moves relative to the vehicle using the elapsed time and the amount of change in distance. In addition, a sensor such as LiDAR (Light Detection and Ranging) or millimeter wave radar may be used to measure the distance between the object and the vehicle, and the speed at which the object moves relative to the vehicle may be calculated. In addition, these sensors may be used as auxiliary sensors. Furthermore, the distance between the object and the vehicle may be measured using a stereo camera, the distance between the object and the vehicle may be estimated from the size of the photographed license plate, and the position (distance) of the object as seen from the vehicle may be estimated from the contact position where the object, such as another vehicle or a pedestrian, comes into contact with the ground. Then, the speed at which the object moves relative to the vehicle may be calculated using the estimated object distance and elapsed time.

The speed of the vehicle may be calculated by acquiring the position information of the vehicle using a GNSS (global navigation satellite system) receiver and using the position information of the vehicle and time information. The speed of the vehicle may be acquired by acquiring the running speed of the vehicle via a CAN (Controller Area Network), or the speed at which a stationary object such as a building or roadside tree recognized by the object recognition unit 12 moves relative to the vehicle. The judgment reference speed calculation unit 13 calculates the judgment reference speed using the speed at which the object moves relative to the vehicle and the speed of the vehicle obtained in this manner. In addition, in this embodiment, the vehicle speed is recognized as a vector in which the direction in which the vehicle is moving is positive, and the speed at which the object moves relative to the vehicle is recognized as a vector in which the object approaches the vehicle. In this case, the judgment reference speed calculation unit 13 calculates the judgment reference speed using the speed and vector information of the vehicle and the speed and vector information of the object. An example of calculating the judgment reference speed will be described later.

The region of interest extraction unit 14 extracts, as a region of interest, a region on the image that includes an object whose reference speed calculated by the reference speed calculation unit 13 satisfies a predetermined condition. The predetermined condition is a condition for excluding objects moving away from the vehicle and objects approaching the vehicle at the same speed as the vehicle speed, i.e., stopped objects, and the like, and appropriately extracting objects that require attention for the driving of the vehicle. For example, the region of interest extraction unit 14 may extract, as a region of interest, a region that includes an object whose reference speed is faster than a predetermined speed.

The predetermined condition (speed) may be set to the same speed as the speed of the vehicle, any speed slower than the vehicle, or any speed faster than the vehicle. The predetermined condition may be changed according to the environment in which the image recording device 1 is used. For example, a speed slower than that during the day may be set as the predetermined condition at night. In this way, a wider area of interest can be extracted at night than during the day, and objects that require more attention for the driving of the vehicle can be efficiently extracted. The predetermined condition may be set according to the type of object recognized by the object recognition unit 12, such as other vehicles or people, and for example, when a person is recognized, a slower speed may be set as the predetermined condition when another vehicle is recognized. In this way, when a person is recognized, a wider area of interest can be extracted than when another vehicle is recognized, and objects that require more attention for the driving of the vehicle, such as pedestrians, can be efficiently extracted.

The region of interest extraction unit 14 divides the image acquired by the image acquisition unit 11 into fixed rectangular regions in advance, for example, into 9 regions (3 x 3) or 16 regions (4 x 4). From the divided regions thus obtained, the region of interest extraction unit 14 extracts one or more divided regions that contain an object whose reference speed satisfies a predetermined condition as a region of interest. At this time, the region of interest extraction unit 14 extracts the divided region that contains an object whose reference speed satisfies a predetermined condition as a region of interest for each frame of the image.

In addition, the region of interest extraction unit 14 may extract a rectangular region of any size that includes all objects whose reference speeds satisfy a specified condition, rather than dividing the image in advance. In this case, regions that are not only rectangular but also circular, elliptical, or the shape of the object itself may be extracted as the region of interest. The region of interest extraction unit 14 may also extract a region that includes a part of an object whose reference speed satisfies a specified condition. For example, the region of interest extraction unit 14 may extract a region where the object recognition unit 12 has recognized a specified object.

The region of interest extraction unit 14 may also set the conditions for extracting a region of interest as follows: the reference speed is slower than a predetermined speed, and the object includes an object moving in the same direction as the host vehicle. In this way, it is possible to extract other vehicles or two-wheeled vehicles running parallel to the host vehicle, and not extract other vehicles or two-wheeled vehicles that are stopped, and it is possible to efficiently extract objects that require more attention for the running of the host vehicle. The region of interest extraction unit 14 may further extract, as a region of interest, a region that includes an object whose displacement amount of the object recognized by the object recognition unit 12 is greater than a predetermined value. Specific examples of when the region of interest extraction unit 14 extracts a region of interest will be described later.

The image quality setting unit 15 sets the image quality of the image of the region of interest extracted by the region of interest extraction unit 14, among the images acquired by the image acquisition unit 11, so that the image has a higher image quality than the images of the other regions. For example, the image quality setting unit 15 may set the image of the region of interest to a higher image quality than the default image quality, thereby making the image of the region of interest have a higher image quality than the images of the other regions. The image quality setting unit 15 may also set the image of the region of interest to a lower image quality than the default image quality, thereby making the image of the region of interest have a higher image quality than the images of the other regions. For example, the default image quality is the image quality that is normally applied to images acquired by the image acquisition unit 11, and if a region of interest is not extracted, the entire image is recorded in the default image quality.

For example, when a compression format such as JPEG2000 or M-JPEG is used, an ROI encoding technique that uses a Max-Shift method to control the bit rate of the region of interest (ROI) can be used to set the image of the region of interest to a higher image quality than the images of other regions. Also, the image can be divided into multiple regions, and the image quality of the region of interest and other regions can be made different by setting encoding parameters for each region. The compression format used is not limited to JPEG2000 or M-JPEG, and any compression format can be used as long as it is a compression method that allows the image quality to be set for each region of the image. For example, a compression format such as H.264 or H.265 can be used.

The image quality of each region may be controlled across multiple frames before and after each frame in which the region of interest is extracted. For example, in the case of a B frame (Bi-directional Predicted Frame) in video compression, the image quality of the frames before and after the frame in which the region of interest is extracted, including the frames before, may be controlled.

Next, the operation of the image recording device 1 according to this embodiment (image recording method) will be described with reference to the flowchart shown in FIG. 2. First, the image acquisition unit 11 acquires an image of the surroundings of the vehicle (step S1). Next, the object recognition unit 12 recognizes a predetermined object from the image acquired by the image acquisition unit 11 (step S2). Specifically, the object recognition unit 12 recognizes objects such as other vehicles and people from the image acquired by the image acquisition unit 11 using image recognition technology.

Next, the judgment reference speed calculation unit 13 calculates the judgment reference speed using the speed at which the object recognized by the object recognition unit 12 moves relative to the host vehicle and the speed of the host vehicle (step S3). Then, if there is an object whose judgment reference speed calculated by the judgment reference speed calculation unit 13 satisfies a predetermined condition (step S4: Yes), the region of interest extraction unit 14 extracts an area on the image containing the object whose judgment reference speed satisfies the predetermined condition as a region of interest (step S5). On the other hand, if there is no object whose judgment reference speed calculated by the judgment reference speed calculation unit 13 satisfies the predetermined condition (step S4: No), the process returns to step S1.

Next, the image quality setting unit 15 sets the image quality of the image of the region of interest extracted by the region of interest extraction unit 14 among the images acquired by the image acquisition unit 11 so that it has a higher image quality than the images of other regions (step S6). After that, the image recording device 1 records the image acquired by the image acquisition unit 11 in the recording unit 18 (step S7). At this time, the image is recorded in the recording unit 18 with the image quality set by the image quality setting unit 15. Thereafter, the operations of steps S1 to S7 are repeated.

Next, an example of the operation of the image recording device 1 according to this embodiment will be described in detail with reference to Figures 3 to 8.

First, as shown in FIG. 3, a case will be described in which the other vehicle 22 is traveling in the opposite lane to the host vehicle 21 and the host vehicle 21 and the other vehicle 22 pass each other. In the case shown in FIG. 3, the host vehicle 21 is moving upward on the paper at a speed of 60 km/h. The vector of the host vehicle 21 at this time is positive (+). The other vehicle 22 is moving downward on the paper at a speed of 60 km/h. Since the other vehicle 22 located in front of the host vehicle 21 is moving in a direction approaching the host vehicle 21, the vector of the other vehicle 22 is negative (-). At this time, the speed at which the other vehicle 22 moves relative to the host vehicle 21 calculated by the judgment reference speed calculation unit 13 is 120 km/h. In addition, the speed of the other vehicle 22 is calculated to be 60 km/h, in combination with the speed of the host vehicle 21, which is 60 km/h.

The judgment reference speed calculation unit 13 calculates the judgment reference speed as "+60-(-60)=120km/h" because the moving speed of the host vehicle 21 is 60km/h, the vector is positive, and the moving speed of the other vehicle 22 is 60km/h, the vector is negative. The region of interest extraction unit 14 extracts, as the region of interest, a region including an object whose judgment reference speed thus calculated satisfies a predetermined condition. For example, when the speed of the host vehicle 21 is set to 60km/h as the predetermined condition, the region of interest extraction unit 14 extracts, as the region of interest, a region including the other vehicle 22 because the judgment reference speed of 120km/h is faster than the speed of the host vehicle 21, 60km/h. In other words, when the host vehicle 21 is traveling, the region of interest extraction unit 14 extracts, as the region of interest, a region including an object that exists in the traveling direction of the host vehicle 21 and shows a movement with a negative vector in a direction approaching the host vehicle 21. The image quality setting unit 15 then sets the image quality of the image of the region of interest, including the other vehicle 22, to be higher quality than the images of other regions.

Next, as shown in FIG. 4, a case will be described in which the host vehicle 21 and the other vehicle 22 are traveling on the same lane and the other vehicle 22 is traveling in front of the host vehicle 21. In the case shown in FIG. 4, the host vehicle 21 is moving upward on the paper at a speed of 60 km/h. At this time, the vector of the host vehicle 21 is positive (+). The other vehicle 22 located in front of the host vehicle 21 is also moving upward on the paper at a speed of 60 km/h. Since the other vehicle 22 is not moving in a direction approaching the host vehicle 21, the vector of the other vehicle 22 is positive (+). At this time, the speed at which the other vehicle 22 moves relative to the host vehicle 21 calculated by the judgment reference speed calculation unit 13 is 0 km/h. In addition, the speed of the other vehicle 22 is calculated to be 60 km/h in combination with the speed of the host vehicle 21, which is 60 km/h.

The judgment reference speed calculation unit 13 calculates the judgment reference speed as "+60-(+60)=0 km/h" because the moving speed of the host vehicle 21 is 60 km/h and the vector is positive, and the moving speed of the other vehicle 22 is 60 km/h and the vector is positive. The region of interest extraction unit 14 extracts, as a region of interest, a region including an object for which the judgment reference speed calculated in this manner satisfies a predetermined condition. For example, when the speed of the host vehicle 21 is set as a predetermined condition, the region of interest extraction unit 14 does not extract, as a region of interest, a region including the other vehicle 22, because the judgment reference speed of 0 km/h is slower than the speed of the host vehicle 21, which is 60 km/h. In other words, when the host vehicle 21 is traveling, the region of interest extraction unit 14 does not extract, as a region of interest, a region including an object that exists in the traveling direction of the host vehicle 21 and has a positive vector that is in a direction away from the host vehicle 21. Furthermore, the region of interest extraction unit 14 does not extract, as a region of interest, a region that includes an object that is not moving (has a vector of zero) relative to the vehicle 21. In this case, the image quality setting unit 15 sets the image including the other vehicle 22 to the same image quality as the images of the other regions.

In the example shown in FIG. 4, when the moving speed of the other vehicle 22 is 80 km/h, the judgment reference speed calculation unit 13 calculates that the speed at which the other vehicle 22 moves relative to the host vehicle 21 is 20 km/h, and the speed of the other vehicle 22 is calculated to be 80 km/h by combining this with the speed of the host vehicle 21, which is 60 km/h. Since the vector of the other vehicle 22 is positive, the judgment reference speed is "+60-(+80)=-20 km/h". In this case, since the judgment reference speed of -20 km/h is slower than the speed of the host vehicle 21, which is 60 km/h, the region of interest extraction unit 14 does not extract the region including the other vehicle 22 as the region of interest. In other words, when the other vehicle 22 is traveling in front of the host vehicle 21 and the moving speed of the other vehicle 22 is faster than the speed of the host vehicle 21, the judgment reference speed is a negative value, and therefore the region of interest extraction unit 14 does not extract the region including the other vehicle 22 as the region of interest.

On the other hand, when the moving speed of the other vehicle 22 is 40 km/h, the judgment reference speed calculation unit 13 calculates that the speed at which the other vehicle 22 moves relative to the host vehicle 21 is 20 km/h, and by combining this with the speed of the host vehicle 21 of 60 km/h, the speed of the other vehicle 22 is calculated to be 40 km/h. Since the other vehicle 22 approaches the host vehicle 21, the vector becomes negative. In this case, the judgment reference speed is "+60 - (-40) = 100 km/h". Since the judgment reference speed of 100 km/h is faster than the speed of the host vehicle 21 of 60 km/h, the interest region extraction unit 14 extracts the region including the other vehicle 22 as the interest region. Then, the image quality setting unit 15 sets the image quality of the image of the interest region including the other vehicle 22 so that it has higher image quality than the images of the other regions. That is, when another vehicle 22 is traveling in front of the host vehicle 21 and the moving speed of the other vehicle 22 is slower than the moving speed of the host vehicle 21, the other vehicle 22 approaches the host vehicle 21 and the vector becomes negative. In this case, the reference speed is faster than the speed of the host vehicle 21, so the region of interest extraction unit 14 extracts the region including the other vehicle 22 as the region of interest. In other words, when the host vehicle 21 is traveling, the region of interest extraction unit 14 extracts the region including an object that exists in the traveling direction of the host vehicle 21 and exhibits movement with a negative vector in a direction approaching the host vehicle 21 as the region of interest.

In this embodiment, the vector of an object approaching the vehicle 21 is made negative in this way, so that the reference speed can be intentionally increased. In other words, the calculation of the reference speed in this embodiment differs from the calculation of the so-called relative speed, in that the direction of the vector is changed depending on whether or not the object is approaching the vehicle 21. Therefore, the calculated value itself does not match the relative speed, and is a calculation value unique to this embodiment. By using such a calculated value, the region of interest can be appropriately extracted.

Next, as shown in FIG. 5, a case will be described where the vehicle 21 is traveling on a road with roadside trees (scenery) 25. In the case shown in FIG. 5, the vehicle 21 is moving upward on the paper at a speed of 60 km/h. At this time, the vector of the vehicle 21 is positive (+). Also, the roadside tree (scenery) 25 located in front of the vehicle 21 is stationary, so its moving speed is 0 km/h. The roadside tree (scenery) 25 is approaching the vehicle 21, so its vector is negative. At this time, the speed at which the roadside tree (scenery) 25 is moving relative to the vehicle 21 calculated by the judgment reference speed calculation unit 13 is 60 km/h, and combining this with the vehicle 21 speed of 60 km/h, the speed of the roadside tree (scenery) 25 is calculated to be 0 km/h.

The judgment reference speed calculation unit 13 calculates the judgment reference speed as "+60-(0)=60km/h" because the moving speed of the vehicle 21 is 60km/h, the vector is positive, and the moving speed of the road tree 25 is 0km/h, the vector is negative. The interest area extraction unit 14 does not extract the road tree 25 as an interest area because the judgment reference speed of 60km/h is the same as the speed of the vehicle 21, 60km/h. In this case, the image quality setting unit 15 sets the image including the road tree 25 to the same image quality as the images of the other areas.

Here, the other vehicle 22 shown in FIG. 4 and the roadside tree 25 shown in FIG. 5 are not extracted as the region of interest, but the other vehicle 22 shown in FIG. 4 is moving at the same speed as the vehicle 21. Therefore, the other vehicle 22 shown in FIG. 4 may be considered to be a higher risk to the vehicle 21 than the stationary roadside tree 25 shown in FIG. 5. In this case, in this embodiment, the other vehicle 22 shown in FIG. 4 may be extracted as the region of interest. That is, the region of interest extraction unit 14 may have a form in which, even if the reference speed is slower than a predetermined speed, a region including an object moving in the same direction as the vehicle may be extracted as the region of interest. For example, in this case, the region of interest extraction unit 14 sets the reference speed as a predetermined condition that the speed at which the other vehicle 22 moves relative to the vehicle 21 is 0 km/h, within a predetermined range (for example, ±3 km/h). Since the predetermined condition is satisfied, the region including the other vehicle 22 can be extracted as the region of interest. It is preferable that the predetermined range is variable according to the moving speed of the vehicle 21.

In this case, the image quality setting unit 15 may set the image quality in three stages. For example, the image quality setting unit 15 may set the image quality to low for areas that are not extracted as areas of interest, and high for areas of interest that include objects whose reference speed is faster than a predetermined speed. Furthermore, the image quality setting unit 15 may set the image quality between low and high (medium image quality) for areas of interest that include objects whose reference speed is slower than a predetermined speed and whose moving direction is the same as that of the vehicle.

Next, as shown in FIG. 6, a case will be described where a person 26 approaches the host vehicle 21 parked in a parking lot. In the case shown in FIG. 6, the host vehicle 21 is stopped, so its moving speed is 0 km/h. Also, the person 26 located in front of the host vehicle 21 is moving at a speed of 4 km/h. At this time, the person 26 is moving in a direction approaching the host vehicle 21, so the vector of the person 26 is negative (-). At this time, the speed at which the person 26 is moving relative to the host vehicle 21 calculated by the judgment reference speed calculation unit 13 is 4 km/h, and combining this with the speed of the host vehicle 21, which is 0 km/h, the speed of the person 26 is calculated to be 4 km/h.

The judgment reference speed calculation unit 13 calculates the judgment reference speed as "0-(-4)=4km/h" because the moving speed of the host vehicle 21 is 0km/h, the moving speed of the person 26 is 4km/h, and the vector is negative. When the speed of the host vehicle 21 is set as a predetermined condition, the interest region extraction unit 14 extracts an area including the person 26 as the interest region because the judgment reference speed of 4km/h is faster than the host vehicle 21 speed of 0km/h. In other words, when the host vehicle 21 is stopped, the interest region extraction unit 14 extracts an area including an object that exists around the host vehicle and shows a movement with a negative vector in a direction approaching the host vehicle as the interest region. Then, the image quality setting unit 15 sets the image quality of the image of the interest region including the person 26 so that it has a higher image quality than the images of other regions.

In the example shown in FIG. 6, if the person 26 is moving in a direction away from the vehicle 21, the vector of the person 26 is positive, so the reference speed is "0 - (+4) = -4 km/h." In this case, the reference speed of -4 km/h is smaller than the speed of the vehicle 21, 0 km/h, so the region of interest extraction unit 14 does not extract the region including the other vehicle 22 as the region of interest.

In addition, when the vehicle 21 is stopped as in the example shown in FIG. 6, it is preferable to record objects such as trees and flags swaying in the wind, which may be recognized as moving and approach the vehicle 21, but whose displacement in the image is small, in low image quality. For this reason, the region of interest extraction unit 14 may extract an area including the object as the region of interest when the displacement of the object calculated by the object recognition unit 12 is greater than a predetermined value. This makes it possible to appropriately extract objects with large displacement in the image, such as other vehicles crossing the image from the left end to the right end and people walking on the crosswalk in front of the vehicle 21, as the region of interest, without extracting objects whose displacement remains in a limited area, such as trees and flags swaying in the wind, as the region of interest.

Next, as shown in Figure 7, a case will be described where another vehicle 22 approaches the host vehicle 21 parked in a parking lot. In the case shown in Figure 7, the host vehicle 21 is stopped, so its moving speed is 0 km/h. Also, the other vehicle 22 located in front of the host vehicle 21 is attempting to back into the parking lot and is moving at a speed of 5 km/h. At this time, the other vehicle 22 is moving in a direction approaching the host vehicle 21, so the vector of the other vehicle 22 is negative (-).

The judgment reference speed calculation unit 13 calculates the judgment reference speed as "0-(-5)=5 km/h" because the moving speed of the host vehicle 21 is 0 km/h, the moving speed of the other vehicle 22 is 5 km/h, and the vector is negative. When the speed of the host vehicle 21 is set as a predetermined condition, the interest region extraction unit 14 extracts an area including the other vehicle 22 as the interest region because the judgment reference speed of 5 km/h is faster than the speed of the host vehicle 21 of 0 km/h. In other words, when the host vehicle 21 is stopped, the interest region extraction unit 14 extracts an area including an object that exists around the host vehicle and shows a movement with a negative vector in a direction approaching the host vehicle as the interest region. Then, the image quality setting unit 15 sets the image quality of the image of the interest region including the other vehicle 22 so that it has a higher image quality than the images of the other regions.

Next, as shown in FIG. 8, a case will be described in which the host vehicle 21 and another vehicle 22 traveling behind the host vehicle 21 are traveling in the same lane and the other vehicle 22 is approaching the host vehicle 21. In the case shown in FIG. 8, the host vehicle 21 is moving upward on the paper at a speed of 60 km/h. At this time, the vector of the host vehicle 21 is positive (+). The other vehicle 22 located behind the host vehicle 21 is moving upward on the paper at a speed of 100 km/h. Since the other vehicle 22 is moving in a direction approaching the host vehicle 21, the vector of the other vehicle 22 is negative (-). At this time, the speed at which the other vehicle 22 moves relative to the host vehicle 21 calculated by the judgment reference speed calculation unit 13 is 40 km/h, and by combining this with the speed of the host vehicle 21 of 60 km/h, the speed of the other vehicle 22 is calculated to be 100 km/h.

The judgment reference speed calculation unit 13 calculates the judgment reference speed as "+60-(-100)=160 km/h" because the moving speed of the host vehicle 21 is 60 km/h, the vector is positive, and the moving speed of the other vehicle 22 is 100 km/h, the vector is negative. For example, when the speed of the host vehicle 21 is set as a predetermined condition, the interest area extraction unit 14 extracts an area including the other vehicle 22 as the interest area because the judgment reference speed of 160 km/h is faster than the speed of the host vehicle 21 of 60 km/h. In other words, when the host vehicle 21 is traveling, the interest area extraction unit 14 extracts an area including an object that exists in the opposite direction to the traveling direction of the host vehicle 21 and that shows a movement with a negative vector in a direction approaching the host vehicle 21 as the interest area. Then, the image quality setting unit 15 sets the image quality of the image of the interest area including the other vehicle 22 so that it has a higher image quality than the images of other areas.

In the example shown in FIG. 8, when the moving speed of the other vehicle 22 is 50 km/h, the other vehicle 22 moves away from the host vehicle 21, and the vector of the other vehicle 22 becomes positive. In this case, the judgment reference speed is "+60 - (+50) = +10 km/h". And, since the judgment reference speed +10 km/h is slower than the speed of the host vehicle 21, which is 60 km/h, the region of interest extraction unit 14 does not extract the region including the other vehicle 22 as the region of interest. In other words, when the host vehicle 21 and the other vehicle 22 are traveling in the same direction and the moving speed of the other vehicle 22 is slower than the speed of the host vehicle 21, the judgment reference speed is slower than the speed of the host vehicle 21, and therefore the region of interest extraction unit 14 does not extract the region including the other vehicle 22 as the region of interest.

As described above, in the image recording device 1 according to the present embodiment, a predetermined object is recognized from an image acquired by the image acquisition unit, a judgment reference speed is calculated using the speed of the recognized object and the speed of the vehicle, and an area including an object whose judgment reference speed satisfies a predetermined condition is extracted as a region of interest. Then, the image quality of the extracted region of interest is set to be higher than that of other regions. Therefore, by selectively setting the image quality of the region of interest that is important to the user, that is, the object approaching the vehicle, to high quality and setting the image quality of the region other than the region of interest to relatively low quality, the image quality of the necessary region can be improved while appropriately suppressing the capacity of the image file. This makes it possible to record the image acquired by the image acquisition unit for a long time. In addition, since the capacity of the image file can be reduced, the burden on the user of adding memory for recording images can be reduced. In addition, in the image recording device according to the present embodiment, the image quality of the region of interest can be improved while appropriately suppressing the capacity of the image file by selectively setting the image quality of the region of interest to high quality compared to the image quality of the region other than the region of interest.

<Embodiment 2>
Next, a second embodiment of the present invention will be described.
9 is a block diagram showing a configuration example of an image recording device according to the second embodiment. As shown in FIG. 9, the image recording device 2 according to the second embodiment includes an image acquisition unit 11, a motion vector detection unit 31, a region of interest extraction unit 14, and an image quality setting unit 15. The image recording device 2 according to the second embodiment is different from the image recording device 1 according to the first embodiment in that the image recording device 2 according to the second embodiment includes a motion vector detection unit 31 instead of the object recognition unit 12 and the judgment reference speed calculation unit 13. Other than this, the image recording device 2 according to the first embodiment is the same as the image recording device 1 according to the first embodiment, so the same components are denoted by the same reference numerals and duplicated explanations are omitted as appropriate.

The image acquisition unit 11 acquires images of the surroundings of the vehicle. In this embodiment, the image acquisition unit 11 is described as acquiring images from a camera installed to capture an image in front of the vehicle. The motion vector detection unit 31 detects a motion vector using the image acquired by the image acquisition unit 11. For example, the motion vector detection unit 31 detects a motion vector for each pixel in the image acquired by the image acquisition unit 11. The motion vector detection unit 31 may also divide the image acquired by the image acquisition unit 11 into previously fixed rectangular regions, such as 9 or 16 regions, and detect a motion vector for each divided region thus obtained.

The region of interest extraction unit 14 extracts, as a region of interest, a region including a motion vector that satisfies a predetermined condition among the motion vectors detected by the motion vector detection unit 31. For example, when the vehicle is traveling, stationary objects in the image acquired by the image acquisition unit 11 spread radially from the center of the image toward the outside. This radial movement corresponds to the vehicle's traveling speed and steering angle, and is faster toward the edge of the image. When a motion vector that does not match the radially spreading movement is detected and the amount of this motion vector is equal to or greater than a predetermined threshold, the region of interest extraction unit 14 determines that this motion vector satisfies a predetermined condition and extracts a region including this motion vector as a region of interest. In other words, when the motion vector detected by the motion vector detection unit 31 does not indicate a motion vector similar to that of a stationary object and the amount of this motion vector is equal to or greater than a predetermined threshold, the region of interest extraction unit 14 determines that the predetermined condition is satisfied and extracts the region including this motion vector as a region of interest.

For example, if the motion vector detection unit 31 detects a motion vector for each pixel in an image, the region of interest extraction unit 14 may extract, as the region of interest, a region that includes a motion vector that satisfies a predetermined condition. At this time, the region of interest may be extracted in a predetermined shape and a predetermined size, such as a rectangle or an ellipse. Also, if the motion vector detection unit 31 detects a motion vector for each divided region, the region of interest extraction unit 14 may extract, as the region of interest, a divided region that includes a motion vector that satisfies a predetermined condition. For example, if a motion vector that satisfies a predetermined condition spans multiple divided regions, the multiple divided regions in which the motion vector that satisfies the predetermined condition exists may be extracted as the region of interest.

The image quality setting unit 15 sets the image quality of the image of the region of interest extracted by the region of interest extraction unit 14, among the images acquired by the image acquisition unit 11, so that it has a higher image quality than the images of the other regions. In other words, the image quality setting unit 15 sets the image quality of the image including the pixel or divided region in which a motion vector satisfying a predetermined condition has been detected, so that it has a higher image quality than the images of the other regions.

As described above, in the image recording device 2 according to the present embodiment, a motion vector is detected using an image acquired by the image acquisition unit, and an area including a motion vector that satisfies a predetermined condition among the detected motion vectors is extracted as a region of interest. Then, the image quality of the image of the extracted region of interest is set to be higher than that of the images of other regions. Therefore, by selectively setting the image quality of an area of interest that is important to the user, that is, an object whose motion vector differs from that of other objects to high quality, and setting the image quality of areas other than the region of interest to relatively low quality, it is possible to appropriately suppress the capacity of the image file while improving the image quality of the necessary area. This makes it possible to record the image acquired by the image acquisition unit for a long period of time. In addition, since the capacity of the image file can be reduced, the burden on the user of adding memory for recording images can be reduced.

An example of the hardware configuration of a control device of an image recording device according to embodiments 1 and 2 will be described with reference to FIG. 10. In FIG. 10, the image recording device has a processor 101 and a memory 102. The processor 101 may be, for example, a microprocessor, an MPU (Micro Processing Unit), or a CPU (Central Processing Unit). The processor 101 may include multiple processors. The memory 102 is configured by a combination of volatile memory and non-volatile memory. The memory 102 may include storage located away from the processor 101. In this case, the processor 101 may access the memory 102 via an I/O interface not shown.

In addition, each device in the above-described embodiments may be configured with hardware or software, or both, and may be configured with one piece of hardware or software, or may be configured with multiple pieces of hardware or software. The functions (processing) of each device in the above-described embodiments may be realized by a computer. For example, a program for performing the operations in the embodiments may be stored in memory 102, and each function may be realized by executing the program stored in memory 102 by processor 101.

Such a program can be stored and provided to a computer using various types of non-transitory computer readable media. Non-transitory computer readable media include various types of tangible storage media. Examples of non-transitory computer readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R/Ws, and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (random access memories)). The program may also be provided to a computer by various types of transitory computer readable media. Examples of transitory computer readable media include electrical signals, optical signals, and electromagnetic waves. The transitory computer readable media can provide the program to a computer via wired communication paths such as electric wires and optical fibers, or wireless communication paths.

The present invention is not limited to the above-mentioned embodiment, and can be modified as appropriate without departing from the spirit of the present invention. For example, the operation example of the first embodiment has been described using FIG. 3 to FIG. 8, but the calculation method of the judgment reference speed calculated by the judgment reference speed calculation unit 13 is not limited to the above-mentioned calculation method, and the speed (relative speed) of the other vehicle 22 or person 26 moving relative to the host vehicle 21 may be used as is. In this case, the interest area extraction unit 14 changes the predetermined condition and extracts the interest area depending on whether the vector of the other vehicle 22 or person 26 is moving in a direction approaching the host vehicle 21, whether the relative speed of the other vehicle 22 or person 26 is the same as the speed of the host vehicle 21, whether the other vehicle 22 or person 26 is located in the traveling direction of the host vehicle 21 or in the opposite direction to the traveling direction of the host vehicle 21, etc.

The present invention has been described above in accordance with the above embodiment, but the present invention is not limited to the configuration of the above embodiment, and of course includes various modifications, alterations, and combinations that a person skilled in the art could make within the scope of the invention of the claims of this patent application.

Reference Signs List 1, 2 Image recording device 11 Image acquisition unit 12 Object recognition unit 13 Criterion speed calculation unit 14 Region of interest extraction unit 15 Image quality setting unit 18 Recording unit 21 Vehicle 22 Other vehicle 31 Motion vector detection unit 101 Processor 102 Memory

Claims (5)

an image acquisition unit that acquires an image of the surroundings of the host vehicle;
an object recognition unit that recognizes a predetermined object from the image acquired by the image acquisition unit;
a judgment reference speed calculation unit that calculates a judgment reference speed using a speed at which an object recognized by the object recognition unit moves relative to the host vehicle and a speed of the host vehicle;
a region of interest extraction unit that extracts a region including an object whose reference velocity satisfies a predetermined condition as a region of interest;
an image quality setting unit that sets an image quality of the image of the region of interest extracted by the region of interest extraction unit among the images acquired by the image acquisition unit so that the image has a higher image quality than images of other regions ;
The predetermined condition is set according to the type of the object recognized by the object recognition unit.
Image recording device. The image recording device according to claim 1 , wherein the predetermined condition is set according to an environment in which the image recording device is used . 3. The image recording device according to claim 1, wherein the region of interest extraction unit extracts a region including the object as a region of interest when the direction of movement of the object is the same as that of the host vehicle, even if the reference speed is slower than a predetermined speed. The object recognition unit further calculates a displacement amount of a position on the image where the object is recognized ,
the region of interest extraction unit extracts a region including the object as a region of interest under a further condition that the amount of displacement is greater than a predetermined value .
The image recording device according to any one of claims 1 to 3 . a motion vector detection unit that detects a motion vector using the image acquired by the image acquisition unit,
the region of interest extraction unit extracts the region of interest under a further condition that the region includes a motion vector that satisfies a predetermined condition among the motion vectors detected by the motion vector detection unit .
5. The image recording device according to claim 1 .

Images