JP7198599B2 - Image processing device, image processing method, drive recorder - Google Patents

Description

The present invention relates to an image processing device, an image processing method, and a drive recorder for recording driving or parking conditions of a vehicle.

A drive recorder that records the driving or parking situation of a vehicle is used for purposes such as accident analysis, confirmation of safe driving techniques, and crime prevention.
Some of such drive recorders acquire an omnidirectional image centered on the vehicle by placing an omnidirectional camera in the vehicle interior.
However, conventional omnidirectional drive recorders reduce the amount of data by recording the entire image at a low resolution. As a result, the image quality is degraded, and inconveniences such as the inability to see the license plate of the vehicle in front about ten meters ahead of the own vehicle arise.

On the other hand, high-resolution images from a omnidirectional camera, which can shoot high-resolution images, result in an excessive amount of data, which increases the amount of heat generated during operation and puts a heavy load on the recorder itself. In addition, an expensive large-capacity memory is required to store high-resolution image data.
Also, for example, a recorder is manufactured in which an omnidirectional camera and a high-resolution camera are installed in the main body of the recorder, the omnidirectional image and the high-resolution image are stored separately, and the high-resolution image is combined with the omnidirectional image in post-processing. In that case, the main body of the recorder becomes expensive because two different types of cameras, an omnidirectional camera and a high-resolution camera, are combined.

Furthermore, there is an image pickup system disclosed in Japanese Patent Application Laid-Open No. 2002-200012, which obtains an omnidirectional image with a plurality of optical lenses. In this imaging system, coordinate conversion processing is performed so that an image centered on the point of interest is placed in the central portion of the omnidirectional image that is less likely to be distorted, and an image corresponding to the central portion is cut out and output as a display image. . However, the clipped image is an image that looks natural to the viewer, and since it is a function that simply clips a part of the image, a function that lowers the resolution, etc., there is a problem that the license plate of the vehicle ahead about 10 meters away cannot be seen. there were.

JP 2016-171577 A

The problem to be solved by the present invention is to provide an image processing device, an image processing method, and a drive recorder capable of easily recognizing an important part in an image and reducing the load during operation.

As a first means for solving the above problems, the present invention provides a photographing means installed in a vehicle interior for photographing the surroundings of the vehicle including at least the forward vehicle, and To provide an image processing apparatus comprising: image quality changing means for cutting out an important part and reducing the quality of the photographed image; and storage means for storing the cut-out important part and the photographed image with reduced image quality. It is something to do.
According to the first means, the important part in the reproduced image can be easily visually recognized, and the load such as heat generation during operation can be reduced.

As a second means for solving the above-mentioned problems, the present invention provides a photographing means installed in a vehicle interior for photographing the surroundings of the vehicle, including at least the preceding vehicle, with high resolution;
an image quality changing means for cutting out an important part of the image captured by the image capturing means at a high resolution and reducing the resolution of the captured image to a low resolution;
a storage means for storing the clipped high-resolution important part and the low-resolution captured image;
An object of the present invention is to provide an image processing apparatus characterized by having
According to the second means, the important part in the reproduced image can be easily visually recognized, and the load such as heat generated during operation can be reduced.

As a third means for solving the above problems, the present invention provides, in the above first or second means, wherein the photographing means comprises a plurality of wide-angle lenses for photographing an omnidirectional sphere centered on the vehicle. It is an object of the present invention to provide an image processing apparatus characterized by:
According to the above-mentioned third means, it is possible to take an omnidirectional image centered on the own vehicle with a single camera, and there is no need to use a plurality of different cameras as in the conventional art, thus reducing the overall cost of the device. can be achieved.

As a fourth means for solving the above problems, the present invention provides any one of the first to third means, wherein the important part is a range including at least the license plate of the forward vehicle. An object of the present invention is to provide an image processing apparatus characterized by the following.
According to the fourth means, during playback, the license plate of the preceding vehicle, which is about ten meters away from the own vehicle, can be visually recognized.

As a fifth means for solving the above-mentioned problems, the present invention is the image characterized in that, in the above-mentioned fourth means, the important part is a continuous part of the front, side and rear of the vehicle. An object of the present invention is to provide a processing device.
According to the above-mentioned fifth means, at the time of reproduction, it is possible to easily visually recognize the entire surroundings of the vehicle, including the license plate of the preceding vehicle, from the peripheral (panoramic) image of the own vehicle.

As a sixth means for solving the above-mentioned problems, the present invention provides, in any one of the first to fifth means, wherein the image quality changing means determines the size of the important portion based on the positional coordinates of the image. Another object of the present invention is to provide an image processing apparatus characterized by being able to arbitrarily change the position.
According to the sixth means, it is possible to easily change the location of the important part in the image that changes depending on the object to be photographed.

As a seventh means for solving the above-mentioned problems, the present invention provides, in any one of the first to sixth means, wherein the image quality changing means sets the image quality of the captured image to be stored to be higher than that of the important part. To provide an image processing device characterized by being able to arbitrarily change settings within a drop range.
According to the above seventh means, it is possible to easily select from a range in which the image quality is reduced according to the specs (memory capacity, etc.) of the apparatus.

As an eighth means for solving the above-mentioned problems, the present invention provides, in any one of the first to seventh means, the important portion being framed when reproducing the image processed by the image quality changing means. To provide an image processing apparatus characterized by comprising a reproduction means for displaying a video within a screen, and switching the screen to the important part for reproduction when the important part is selected.
According to the eighth means, an important part with good image quality in the reproduced image is enlarged and displayed, so that, for example, the license plate of the forward vehicle about ten meters ahead of the own vehicle can be easily visually recognized.

As a ninth means for solving the above problems, the present invention provides a drive recorder comprising the image processing device according to any one of the first to eighth above.
According to the ninth means, it is possible to obtain a drive recorder in which the important part in the reproduced image can be easily visually recognized and the load during storage can be reduced.

As a tenth means for solving the above problems, the present invention provides a step of recording the surroundings of the vehicle, including at least the preceding vehicle, with a photographing means installed in the interior of the vehicle;
a step of cutting out an important portion in the photographed image of the photographing means, reducing the image quality of the photographed image to change the image quality; and a step of storing the cut out important portion and the photographed image with the reduced image quality. To provide an image processing method characterized by:
According to the tenth means, the important part in the reproduced image can be easily visually recognized, and the load during operation can be reduced.

According to the present invention configured as described above, the amount of data to be processed is reduced, and the heat generation of the main body of the recorder can be suppressed, so that the load during operation can be reduced. In addition, the size of the apparatus body can be reduced.

1 is a block diagram of an image processing device of the present invention; FIG. 1 is an explanatory diagram 1 of Example 1 of an important part of recorded data; FIG. FIG. 2 is an explanatory diagram 2 of Example 1 of an important part of recorded data; FIG. 2 is an explanatory diagram of important parts of Example 1; FIG. 10 is an explanatory diagram 1 of Example 2 of an important part of recorded data of a modified example; FIG. 12 is an explanatory diagram 2 of Example 2 of the important part of the recorded data of the modified example; FIG. 10 is an explanatory diagram 1 of Example 3 of an important part of recorded data of a modified example; FIG. 12 is an explanatory diagram 2 of Example 3 of the important part of the recorded data of the modified example; FIG. 13 is an explanatory diagram 3 of Example 3 of the important part of the recorded data of the modified example; 1 is an explanatory diagram of a drive recorder that serves as an image processing device of the present invention; FIG. 1 is an explanatory diagram of an action camera that serves as an image processing device of the present invention; FIG. 4 is a flowchart of recording processing by the image processing apparatus of the present invention; FIG. FIG. 4 is a flowchart of reproduction processing by the reproduction means of the present invention;

DETAILED DESCRIPTION OF THE INVENTION Embodiments of an image processing apparatus, an image processing method, and a drive recorder of the present invention will be described in detail below with reference to the accompanying drawings.
FIG. 1 is a block diagram of an image processing apparatus of the present invention. FIG. 2 is an explanatory diagram 1 of Example 1 of the important part of recorded data. FIG. 3 is an explanatory diagram 2 of Example 2 of the important part of the recorded data. FIG. 4 is an explanatory diagram of important parts of the first embodiment. FIG. 5 is an explanatory diagram 1 of Example 2 of the important part of the recorded data of the modified example. FIG. 6 is an explanatory diagram 2 of Example 2 of the important part of the recorded data of the modified example. FIG. 7 is an explanatory diagram 1 of Example 3 of the important part of the recorded data of the modified example. FIG. 8 is an explanatory diagram 2 of Example 3 of the important part of the recorded data of the modified example. FIG. 9 is an explanatory diagram 3 of Example 3 of the important part of the recorded data of the modified example. FIG. 10 is an explanatory diagram of a drive recorder that serves as the image processing device of the present invention. FIG. 11 is an explanatory diagram of an action camera that serves as an image processing apparatus of the present invention. FIG. 12 is a flowchart of recording processing by the image processing apparatus of the present invention. FIG. 13 is a flowchart of reproduction processing by the reproduction means of the present invention.

[Photographing means 20]
The photographing means 20 is detachable inside the vehicle, and photographs the surroundings of the vehicle including the forward vehicle. The photographing means has a plurality of wide-angle (including fisheye lenses) lenses, and in this embodiment, as an example, two wide-angle (fisheye) lenses (front and rear lenses) are provided on the front (front) and rear (rear) sides of the device body. are arranged to acquire an omnidirectional image (front and rear imaging elements). For example, a CMOS (Complementary Metal Oxide Semiconductor) image sensor can be applied to such a photographing means.

[Image quality changing means 30]
For example, when the number of pixels photographed by the photographing means 20 is about 30,000,000 pixels of horizontal 7,808×vertical 3,904 pixels, the amount of data becomes enormous if recorded as a moving image as it is.
On the other hand, when the number of pixels is set to a low resolution of about 4 million pixels (2,880 horizontal x 1,440 vertical), although the amount of data to be recorded can be reduced, due to the reduction in resolution, the number of recorded images such as the license plate of the vehicle in front is reduced. The problem arises that important parts become invisible.

Therefore, the image quality changing means 30 cuts out the central part where the license plate of the preceding vehicle, etc., which is an important part in the recorded image, is reflected, and the number of pixels is changed to about 2 million pixels of horizontal 1,920 × vertical 1,080. High resolution is used, and the number of pixels for areas including important parts (cutout parts) such as the surroundings of the vehicle ahead is set to a low resolution of about 4 million pixels (2,880 horizontal x 1,440 vertical), which is lower than the image quality of the important parts. (See Figures 2, 3 and 4).
As a result, the total number of pixels is about 6,000,000 pixels, which is one-fifth the number of pixels in the case where the entire image is of high quality (about 30,000,000 pixels). And the license plate of the forward vehicle in the high-quality image of the important part is easy to see and can be clearly confirmed. Also, since the amount of data to be processed can be reduced, the amount of heat generated by the entire apparatus can be suppressed.

The image quality in this embodiment is the quality of the image, and a clear image such as the license plate of the vehicle ahead about 10 meters away is called high image quality, and an image in which the license plate cannot be seen is called low image quality. Also, the image quality may be defined numerically by the number of pixels and the frame rate.
In terms of the number of pixels, the important part is a high resolution that allows the license plate of the vehicle in front about 10 meters away from the vehicle to be visible. Therefore, the resolution is lower than that of the high resolution.
Therefore, the photographing means 20 of the present invention uses a camera capable of acquiring high-quality recorded data.

As shown in FIG. 1, the specific configuration of the image quality changing means 30 of the present embodiment consists of a front temporary storage unit that temporarily stores an image captured by the front imaging element, and a rear temporary storage unit that temporarily stores an image captured by the rear imaging element. A temporary storage section, a clipping section of important parts in the recorded image temporarily stored in the front temporary storage section, a front whole image quality changing section that changes the image quality of the front recorded image, and a rearward that changes the image quality of the rear recorded image It has an overall image quality changer.
Such image quality changing means 30 is configured to be able to arbitrarily change the size or position of the important portion based on the positional coordinates of the image. In addition, as shown in FIGS. 5 and 6, the important part to be the cut-out range may be set as a continuous part of the front, side, and rear of the vehicle, in other words, a panorama. In such a range, in addition to the license plate of the vehicle ahead, the situation around the own vehicle (side and rear vehicles, pedestrians on the sidewalk, traffic lights, etc.) can also be confirmed.
The extraction of the important part by the image quality changing means 30 may be automatically set by license plate recognition, face recognition, sign or signal recognition, etc. via artificial intelligence connected to the outside. Multiple parts can be set (see Figures 7-9). As shown in the figure, the license plate of the vehicle in front may be displayed on the front side, and the driver may be displayed on the rear side.
Further, the image quality changing means 30 can arbitrarily change the setting of the photographed image to be stored according to the specs (memory capacity, etc.) of the device within a range in which the image quality of the photographed image is reduced from that of the important portion.

[Storage means 40]
The storage means 40 stores the image data image-processed by the image quality changing means 30 . For this storage means 40, for example, a DRAM (Dynamic Random Access Memory) can be applied. Note that the storage means 40 may adopt a configuration in which an external storage such as an SD memory that can be easily taken out when reproducing recorded images on an external terminal (mobile terminal or the like) is attached.
As shown in FIG. 1, the specific configuration of the storage means 40 consists of an important part storage section for storing high-resolution important parts from the important part clipping section, and a reduced image quality from the front whole image quality changing section (low resolution 2) front and rear overall storage section for recording the front shot image and the rear shot image with reduced image quality (low resolution) from the rear overall image quality changing section.

[Reproducing means 50]
The reproduction means 50 is application software capable of reproducing the recorded images stored in the storage means 40 . The playback means 50 sets a frame surrounding the important part in the recorded image and displays it during playback (see FIGS. 3 and 6). When this frame is selected during playback, the file (image data) of the important part is switched to and this part is enlarged and displayed (see FIG. 4). Here, the reproducing means 50 synchronizes and reproduces the entire image recorded by the photographing means 20 and the recorded data of the important part. Therefore, when an important part is selected at any time during reproduction, it is possible to switch to the important part at that time and reproduce it.
Note that the reproducing means 50 is installed in an external terminal or the device main body which is connected to the device main body via wire or wireless. Also, when playing back a recorded image, it is preferable to transfer data from the device main body using a communication line (3G/LTE/5G) in addition to cable connection (wired).
Also, when a plurality of important parts are set in advance, the user may arbitrarily select and reproduce the selected important parts in an enlarged display during reproduction.

[drive recorder 70]
As shown in FIG. 10, a drive recorder 70, which is an example of an image processing device, has a photographing means 20 which is an omnidirectional camera with a fisheye (wide-angle) lens on the front and rear surfaces of the main body of the recorder. The main body of the recorder can be attached to and detached from the windshield inside the vehicle, and the front lens (not shown) can photograph the front and sides of the vehicle, and the rear lens can photograph the rear and sides of the vehicle. By synthesizing the images of the fisheye lenses on the front and back, it is possible to shoot a 360-degree omnidirectional image.

[Action camera 80]
11A and 11B are explanatory diagrams of an action camera serving as an image processing device, in which (A) shows one main surface (rear surface) and (B) shows the other main surface (front surface). As shown in the drawing, an action camera 80, which is an example of an image processing apparatus, has a photographing means 20 which is an omnidirectional camera having a fisheye (wide-angle) lens on the front and rear surfaces of the camera body. The camera body has a rear wide-angle (rear) lens that captures the photographer himself, and a front wide-angle (front) lens that captures the photographer's surroundings. By synthesizing the images of the fisheye lenses on the front and back, it is possible to shoot a 360-degree omnidirectional image.

Note that the image processing apparatus of the present invention employs a configuration in which the photographing means 20, the image quality changing means 30, and the storage means 40 are arranged in the main body of the apparatus, and the reproduction means 50 and the display section 60 are arranged in the external terminal. The overall size can be reduced. In addition, it is also possible to adopt a configuration in which a reproducing means 50 and a display section 60 are added to the main body of the apparatus.

An image processing method using the image processing apparatus configured as described above will be described below.
The captured image is stored as follows (see FIG. 12).
[Extract important parts]
The image quality changing means 30 performs a process of cutting out an important part (such as the central part where the license plate of the vehicle in front is reflected) in the photographed image previously photographed by the photographing means 20 . The size and position of the clipping range can be arbitrarily set and changed based on the position coordinates of the image data. You can also set it to an image. Also, a plurality of important parts can be set, for example, a place where the license plates of the front and rear vehicles can be seen.
Also, for a captured image including an important portion, settings are made to lower the image quality (for example, lower the resolution, lower the number of frame rates, etc.) than the important portion.

[photograph]
An image is taken by the image taking means 20 . The captured image is temporarily stored in the front and rear temporary storage units.
[Save (cropped part is high resolution / captured image is low resolution)]
The image data image-processed by the image quality changing means 30 is stored in the storage means 40 . For example, the important part cut out by the image quality changing means 30 is saved with high resolution.
As a result, the license plate of the forward vehicle in the high-quality image of the important part can be easily seen and clearly confirmed. In addition, since the amount of data to be processed can be reduced by storing the entire image at a low resolution, the amount of heat generated by the entire apparatus can be suppressed.

A recorded image is reproduced as follows (see FIG. 13).
[play]
For example, an image processing device is connected to an external terminal into which the reproducing means 50 is introduced, and a recorded image of the omnidirectional sphere is displayed (reproduced) on the display unit 60 . At this time, an omnidirectional recorded image whose image quality is lower than that of the important portion is played back, and the important portion surrounded by a frame (such as a dotted line) is displayed in the playback screen.

[Select important parts]
Select a frame indicating an important part at any point during playback of a recorded image of the omnidirectional sphere.
[Play Switch File]
The file switched to the recorded image of the important part synchronized with the omnidirectional recorded image being played is played back. This switching file has better image quality than the omnidirectional recorded image, and the license plate of the vehicle in front can be easily recognized from about 10 meters away from the own vehicle.
According to such an image processing method of the present invention, the important part in the reproduced image can be easily visually recognized, and the load during operation can be reduced.

The preferred embodiments of the present invention have been described above. However, the present invention is not limited to the above-described embodiments, and various modifications are possible without departing from the gist of the present invention.
Moreover, the present invention is not limited to the combinations shown in the embodiments, and can be implemented in various combinations.

INDUSTRIAL APPLICABILITY The present invention can be widely applied, for example, as an in-vehicle drive recorder.

10 Image processing device 20 Photographing means 30 Image quality changing means 40 Storage means 50 Reproducing means 60 Display unit 70 Drive recorder 80 Action camera

Claims (10)

a photographing means installed in the interior of the vehicle for photographing the surroundings of the vehicle including at least the forward vehicle;
an image quality changing means for cutting out an important part in the captured image of the imaging means and reducing the quality of the captured image;
storage means for storing the cut-out important part and the captured image with reduced image quality;
, wherein the important portion is a continuous portion in the front, side, and rear of the vehicle. a photographing means installed in the interior of the vehicle for photographing the surroundings of the vehicle, including at least the forward vehicle, with high resolution;
an image quality changing means for cutting out an important part of the image captured by the image capturing means at a high resolution and reducing the resolution of the captured image to a low resolution;
a storage means for storing the clipped high-resolution important part and the low-resolution captured image;
, wherein the important portion is a continuous portion in the front, side, and rear of the vehicle. 3. The image processing apparatus according to claim 1, wherein said photographing means comprises a plurality of wide-angle lenses for photographing an omnidirectional sphere centered on said vehicle. 4. The image processing apparatus according to any one of claims 1 to 3, wherein said important portion is a range including at least the license plate of said preceding vehicle. 4. The image processing apparatus according to any one of claims 1 to 3, wherein said important part is configured to be automatically set via artificial intelligence connected to the outside. 6. The image processing apparatus according to any one of claims 1 to 5, wherein said image quality changing means can arbitrarily change the size or position of said important portion based on the positional coordinates of the image. Image processing device. 7. The image processing apparatus according to any one of claims 1 to 6, wherein said image quality changing means can arbitrarily change the setting of said photographed image to be stored within a range in which the image quality of said photographed image to be saved is lower than that of said important part. image processing device. 8. The image processing apparatus according to any one of claims 1 to 7, wherein when the image processed by the image quality changing means is reproduced, the important part is displayed in a frame, and when the important part is selected, An image processing apparatus, comprising a reproducing unit for switching a screen to the important part and reproducing the important part. A drive recorder comprising the image processing device according to any one of claims 1 to 8. a step of recording the surroundings of the vehicle, including at least the forward vehicle, with a photographing means installed in the interior of the vehicle;
a step of cutting out an important portion in the image captured by the image capturing means and changing the image quality by reducing the image quality of the captured image;
a step of storing the clipped important part and the captured image with reduced image quality;
and wherein the important portion is a continuous portion in front, sides and rear of the vehicle.

Images