JP7685341B2 - Terminal device, generation method and generation program - Google Patents

Description

The present invention relates to a terminal device, a generation method, and a generation program.

A license plate reader has been proposed that can accurately recognize the registration number written on the license plate from a captured image of a vehicle (Patent Document 1).

JP 2009-15478 A

When attempting to capture a real-life scene with a high contrast ratio (for example, a scene with clear differences in light and dark at night, where some parts are very bright due to street lights or street lamps), it is known that the image is captured with the exposure focused on a specific subject, resulting in some of the light and dark areas being lost, resulting in a phenomenon known as whiteout or blackout.

One example of the problem with the above-mentioned conventional technology is that it is possible to obtain a captured image in which a specific object (such as a license plate) cannot be recognized due to this whiteout or blackout, making it impossible to record a video in which the specific object is clearly visible in an environment where whiteout or blackout occurs.

The present invention has been made in consideration of the above, and aims to provide a terminal device, a generation method, and a generation program that can record video images in which a specific object is clearly shown, even in an environment where whiteout or blackout occurs, for example.

The terminal device described in claim 1 is characterized by having an acquisition means for acquiring a first captured image captured with a first exposure amount and a second captured image captured with a second exposure amount, and a generation means for generating a moving image by synthesizing an image corresponding to a first region showing a predetermined object included in the first captured image and an image corresponding to a second region of the second captured image that is different from the first region.

The generating method described in claim 14 is a generating method executed by a terminal device, and is characterized in that it includes an acquisition means for acquiring a first captured image captured with a first exposure amount and a second captured image captured with a second exposure amount, and a generating means for generating a moving image by synthesizing an image corresponding to a first region showing a predetermined object included in the first captured image and an image corresponding to a second region of the second captured image that is different from the first region.

The generating program described in claim 15 is a generating program for causing a computer to execute an acquisition procedure for acquiring a first captured image captured with a first exposure amount and a second captured image captured with a second exposure amount, and a generating procedure for generating a moving image by synthesizing an image corresponding to a first region showing a predetermined object included in the first captured image and an image corresponding to a second region of the second captured image that is different from the first region.

FIG. 1 is a diagram showing an overall view of a generation process according to the first embodiment. FIG. 2 is a diagram illustrating an example of the configuration of a terminal device according to the first embodiment. FIG. 3 is a diagram illustrating an example of an imaging information storage unit according to the first embodiment. FIG. 4 is a diagram illustrating an example of the synthesis process according to the first embodiment. FIG. 5 is a flowchart illustrating a generation process procedure according to the first embodiment. FIG. 6 is a diagram illustrating an example of a modification according to the first embodiment. FIG. 7 is a diagram illustrating an example of the configuration of a terminal device according to the second embodiment. FIG. 8 is a diagram illustrating an example of a format information storage unit according to the second embodiment. FIG. 9 is a diagram illustrating an example of a generation process according to the second embodiment. FIG. 10 is a diagram illustrating an example of display control by a terminal device. FIG. 11 is a hardware configuration diagram showing an example of a computer that realizes the functions of the terminal device.

Below, a mode for carrying out the present invention (hereinafter, an embodiment) will be described with reference to the drawings. Note that the present invention is not limited to the embodiment described below. Furthermore, in the description of the drawings, the same parts are given the same reference numerals.

(Embodiment 1)
1. Overview of the First Embodiment
An overview of the generation process performed by the terminal device 10 according to the first embodiment (the generation process according to the first embodiment) will be described with reference to Fig. 1. Fig. 1 is a diagram showing an overall view of the generation process according to the first embodiment.

The terminal device 10 is, for example, a stationary navigation device or a drive recorder that is installed in a vehicle. Note that the terminal device 10 is not limited to a navigation device or a drive recorder, and may be a portable terminal such as a smartphone owned by a user who is a passenger of the vehicle.

Here, FIG. 1 shows a scene in which a user places the terminal device 10 in front of the windshield so that the user can capture an image of the area in front of the vehicle, and causes the terminal device 10 to start recording, for example, while driving at night. In such a case, the terminal device 10 captures and records an image of the area in front of the vehicle via the imaging area AR. For this reason, the specified object in FIG. 1 will be described as being the license plate of the vehicle ahead.

In the example of FIG. 1, the terminal device 10 first captures the imaging area AR in a state where the entire imaging area AR is exposed in normal mode (step S11). For example, the terminal device 10 sets an exposure amount that can capture more light as the overall imaging exposure amount used for overall imaging, which is imaging focused on the entire imaging area AR, in response to the fact that the entire image captured through the imaging area AR is dark (in response to the main subject being a dark background). Then, the terminal device 10 captures the imaging area AR with the set overall imaging exposure amount. FIG. 1 shows an example in which the terminal device 10 acquires an overall focus image G12 as an image captured of the imaging area AR with the overall imaging exposure amount. Also, according to the example of FIG. 1, the overall focus image G12 is an image in which the license plate is unclear due to whiteout.

Focusing on the entire imaging area AR means setting the exposure point on the dark background, which is the main subject.

Next, the terminal device 10 performs a process of identifying the position of the number area showing the shape of the license plate in the overall focus image G12 (step S12). For example, when the terminal device 10 detects that a vehicle is captured in the overall focus image G12, the terminal device 10 determines whether or not blown-out highlights have occurred in the vehicle image based on whether or not the brightness of the image (vehicle image) in the vehicle area showing the outer shape of the detected vehicle in the overall focus image G12 exceeds a predetermined value. When the terminal device 10 determines that blown-out highlights have occurred, the terminal device 10 identifies the position of the number area in the vehicle image by controlling the brightness of the overall focus image G12 (e.g., by lowering the brightness). For example, the terminal device 10 lowers the brightness of the overall focus image G12 until the position of the number area can be identified.

Furthermore, when the terminal device 10 is able to identify the position of the number area, it captures the imaging area AR with exposure applied to the number area at the identified position (step S13). For example, the terminal device 10 sets the exposure amount (number image capture exposure amount) to be used for capturing license plate image, which is an image focused on the number area, based on the luminance at the time when the terminal device 10 is able to identify the position of the number area. Then, the terminal device 10 captures the imaging area AR with the set number image capture exposure amount. FIG. 1 shows an example in which the terminal device 10 has acquired a number focus image G13 as an image captured of the imaging area AR with the number image capture exposure amount. Furthermore, the number focus image G13 shown in FIG. 1 is a clear image with reduced blown-out of the number area ARNB, which indicates the license plate of the vehicle ahead.

Next, the terminal device 10 generates a moving image by adding together the whole focus image G12 obtained by capturing the imaging area AR with the whole imaging exposure amount and the number focus image G13 obtained by capturing the imaging area AR with the number imaging exposure amount (step S14). As will be described in detail with reference to FIG. 4, for example, the terminal device 10 generates a frame image by combining a pair of captured images included in each set of captured images (a set of a whole focus image and a number focus image) obtained according to the passage of recording time. Then, the terminal device 10 generates a moving image by combining the frame images obtained for each set. To generate such a moving image, the terminal device 10 can use HDR (High Dynamic-range Rendering) synthesis technology. FIG. 1 shows an example in which the terminal device 10 generates a moving image VD1 as a recorded video based on the whole focus image G12 and the number focus image G13.

Now, as described above, in the example of FIG. 1, when the imaging area AR includes a license plate (an example of a predetermined object), the terminal device 10 according to the first embodiment sets a license plate imaging exposure amount (an example of a first exposure amount) that exposes the license plate, and sets an overall imaging exposure amount (an example of a second exposure amount) that exposes an area of the imaging area AR that is different from the license plate area ARNB (an example of a first area) that indicates the license plate, i.e., the imaging area AR itself (an example of a second area). Then, in the example of FIG. 1, the terminal device 10 acquires a number focus image G13 (an example of a first imaging image) in which the imaging area AR is imaged with the license plate imaging exposure amount, and an overall focus image G12 (an example of a second imaging image) in which the imaging area AR is imaged with the overall imaging exposure amount, and generates a moving image VD1 by synthesizing this number focus image G13 including the license plate area ARNB and this overall focus image G12 corresponding to the imaging area AR.

With such a terminal device 10, for example, in an environment where highlight blowout or shadow crush occurs, it becomes possible to record a video image that shows the license plate clearly without being obscured by highlight blowout or shadow crush.

In the identification process described in FIG. 1, an example was shown in which the terminal device 10 identifies the position of the license plate area in the vehicle image by controlling the brightness when the terminal device 10 is able to detect a vehicle by image analysis of the overall focus image and whiteout occurs in the vehicle image. In other words, an example was shown in which the terminal device 10 narrows down the identification target (search target) from the vehicle to the license plate. However, as long as the terminal device 10 can detect a vehicle in the overall focus image, it is possible to estimate the position of the license plate area from the outline of the detected vehicle without controlling the brightness of the vehicle image. Therefore, the terminal device 10 may set the exposure for capturing the license plate based on the brightness when the terminal device 10 detects a vehicle in the overall focus image.

In addition, if it is known in advance that an exposure level that will not cause whiteout or blackout will be sufficient, the terminal device 10 may set this exposure level as the exposure level for capturing the license plate number.

In the example of FIG. 1, the terminal device 10 determines whether or not whiteout has occurred in the vehicle image based on whether the luminance of the vehicle image exceeds a predetermined value, and if it determines that whiteout has occurred, it identifies the position of the license plate area by controlling the luminance of the vehicle image. However, the terminal device 10 can also determine whether or not blackout has occurred in the vehicle image based on whether the luminance of the vehicle image exceeds a predetermined value, and if it determines that blackout has occurred, it can identify the position of the license plate area in the vehicle image by controlling the luminance of the entire focus image (for example, by increasing the luminance).

2. Configuration of terminal device
Next, the terminal device 10 according to the first embodiment will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of the configuration of the terminal device 10 according to the first embodiment. As shown in Fig. 2, the terminal device 10 includes a communication unit 11, a storage unit 12, a sensor unit 13, an input unit 14, a display unit 15, and a control unit 16.

(Regarding communication unit 11)
The communication unit 11 is realized by, for example, a network interface card (NIC). The communication unit 11 is connected to a network by wire or wirelessly, and transmits and receives information to and from, for example, a predetermined external device. One example of the predetermined external device is a server device that collects information from the terminal device 10 and controls the terminal device 10 to perform imaging based on the collected information.

(Regarding the storage unit 12)
The storage unit 12 is realized by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk, an optical disk, etc. The storage unit 12 has an imaging information storage unit 12a.

(Regarding the imaging information storage unit 12a)
The imaging information storage unit 12a stores various captured images related to generation of a moving image. An example of the imaging information storage unit 12a according to the first embodiment is shown in Fig. 3. In the example of Fig. 3, the imaging information storage unit 12a has items such as "image ID (identifier)", "recording time", "captured image", "focus target", "frame image", and "moving image".

"Image ID" indicates identification information that identifies the captured image of the imaging area AR. "Recording time" indicates the time position at which the "captured image" was captured within the time that has elapsed since recording began. "Captured image" indicates the captured image of the imaging area AR at the "recording time." "Focus target" is information that defines whether the "captured image" is an image obtained by overall imaging, which is imaging that focuses on the entire imaging area AR, or an image obtained by license plate imaging, which is imaging that focuses on the license plate.

A "frame image" is a frame image generated from a set of corresponding "captured images," and is the image that is the source for generating a moving image with a specified frame rate. A "moving image" refers to a moving image generated by combining corresponding "frame images."

(Regarding the sensor unit 13)
The sensor unit 13 captures an image of the surrounding scenery included in an imaging area AR according to a viewing angle, out of the surrounding scenery of the vehicle in which the terminal device 10 is installed, under the control of the control unit 16. For this reason, the sensor unit 13 is a processing unit that can be considered as an imaging unit (camera).

The sensor unit 13 may be composed of multiple sensor units, and the example in FIG. 3 shows an example in which the sensor unit 13 is composed of two sensor units, a first sensor 13a and a second sensor 13b. In addition, when the terminal device 10 has two sensor units in this way, for example, the first sensor 13a (an example of a first sensor) can capture an image of the imaging area AR with an exposure amount for capturing number images (an example of a first exposure amount) under the control of the control unit 16, and the second sensor 13b (an example of a second sensor) can capture an image of the imaging area AR with an exposure amount for capturing an entire image (an example of a second exposure amount) under the control of the control unit 16.

(Regarding input unit 14 and display unit 15)
The input unit 14 includes input devices such as a touch panel, a keyboard, and a mouse, and receives input of various data in response to operations by a user. The input unit 14 then outputs the received input data to the control unit 16. The display unit 15 is configured with a display using liquid crystal or organic EL (Electro Luminescence), and displays various images under the control of the control unit 16.

(Regarding the control unit 16)
2, the control unit 16 is realized by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or the like executing various programs (including the generation program according to the first embodiment) stored in a storage device inside the terminal device 10 using a RAM as a working area. The control unit 16 is also realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

The control unit 16 is also configured with, for example, the following processing units to realize the generation process according to the first embodiment, in which a first captured image captured with a first exposure amount and a second captured image captured with a second exposure amount are acquired, and a moving image is generated by synthesizing an image corresponding to a first region showing a predetermined object included in the first captured image with an image corresponding to a second region in the second captured image that is different from the first region.

As shown in FIG. 2, the control unit 16 has a recording control unit 16a, a specification unit 16b, a setting unit 16c, an acquisition unit 16d, and a generation unit 16e, and realizes or executes the functions and actions of the information processing described below. Note that the internal configuration of the control unit 16 is not limited to the configuration shown in FIG. 2, and may be other configurations as long as they perform the information processing described below. Also, the connection relationship between each processing unit of the control unit 16 is not limited to the connection relationship shown in FIG. 2, and may be other connection relationships.

(Regarding the recording control unit 16a)
The recording control unit 16a performs recording control in response to an operation by a user so as to record the image captured by the sensor unit 13 via the imaging area AR.

(Regarding the identification unit 16b)
The identification unit 16b performs an identification process to identify the position of a predetermined object in a captured image in which the imaging area is captured. As described in FIG. 1, the identification unit 16b detects the predetermined object by controlling the amount of exposure.

According to the example of FIG. 1, the identification unit 16b detects whether or not a vehicle is captured in the overall focus image G12 (second captured image) by, for example, image analysis of the overall focus image G12. If the identification unit 16b detects a vehicle, it determines whether or not blown-out highlights have occurred in the vehicle image based on whether or not the brightness of the vehicle image, which is an image within the vehicle area showing the outline of the detected vehicle in the overall focus image G12, exceeds a predetermined value. If the identification unit 16b determines that blown-out highlights have occurred, it identifies the position of the license plate area in the vehicle image by controlling the brightness of the overall focus image G12 (for example, by lowering the brightness).

Although FIG. 1 shows the identification process targeted at whiteout, the identification unit 16b can also perform similar processing on blackout. That is, the identification unit 16b detects whether or not a vehicle is captured in the overall focus image G12 (second captured image) by image analysis of the overall focus image G12. If the identification unit 16b detects a vehicle, it determines whether or not blackout has occurred in the vehicle image based on whether or not the brightness of the vehicle image showing the outer shape of the detected vehicle in the overall focus image G12 exceeds a predetermined value. If it determines that blackout has occurred, the identification unit 16b controls the brightness of the overall focus image G12 (for example, by increasing the brightness) to identify the position of the license plate area in the vehicle image.

(Regarding the setting unit 16c)
When a predetermined object is included in the imaging area (i.e., when the position of the predetermined object is identified in the captured image in which the imaging area is captured by the identification unit 16b), the setting unit 16c sets a first exposure amount corresponding to the predetermined object. For example, the setting unit 16c sets the first exposure amount based on the luminance when the position of the first area can be identified in the second captured image by controlling the luminance of the second captured image. In other words, the setting unit 16c sets the first exposure amount based on the luminance when the position of the first area is identified in the second captured image by the identification unit 16b.

The setting unit 16c also sets a second exposure amount corresponding to a second region (e.g., the imaging area) that is different from the first region of the imaging area.

In addition, when the terminal device 10 has the sensor unit 13 as one sensor, the setting unit 16c can switch the exposure amount of the sensor unit 13 between the first exposure amount and the second exposure amount at a predetermined timing to capture an image. For example, while the position of a predetermined object is being identified by the identification unit 16b, the setting unit 16c repeats alternating imaging, controlling the sensor unit 13 to capture an image of the imaging area with the first exposure amount, and then controlling the sensor unit 13 to capture an image of the imaging area with the second exposure amount.

(Regarding the Acquisition Unit 16d)
The acquiring unit 16d acquires a first captured image captured with a first exposure amount and a second captured image captured with a second exposure amount. Specifically, the acquiring unit 16d acquires a first captured image captured with the first exposure amount set by the setting unit 16c and a second captured image captured with the second exposure amount set by the setting unit 16c. The acquiring unit 16d also stores the acquired captured images in the imaging information storage unit 12a.

(Regarding the generation unit 16e)
The generating unit 16e generates a moving image by synthesizing an image corresponding to a first region showing a predetermined object included in the first captured image with an image corresponding to a second region in the second captured image different from the first region. Specifically, the generating unit 16e generates a moving image by synthesizing a first captured image including the first region with a second captured image including the second region. For example, when the captured image includes a license plate of a vehicle that is a moving body, and the setting unit 16c sets a first exposure amount corresponding to the license plate, the generating unit 16e synthesizes an image of the license plate captured with the first exposure amount (a license plate focus image) with the second captured image (a whole focus image) to generate a moving image of the vehicle when it is traveling.

When the exposure amount of the sensor unit 13 is switched between the first exposure amount and the second exposure amount at a predetermined timing, the generation unit 16e generates a moving image at a predetermined frame rate by synthesizing the first captured image and the second captured image captured at the different timings. For example, the generation unit 16e generates frame images, which are still images corresponding to one frame, for each pair of one first captured image and one second captured image captured alternately, and generates a moving image by adding together the generated frame images.

Here, an example of a synthesis process for generating a moving image from frame images will be described with reference to FIG. 4. FIG. 4 is a diagram showing an example of the synthesis process according to the first embodiment. FIG. 4 shows an example in which captured images of the imaging area AR at each predetermined time position according to the time lapse since the start of recording are arranged in a chronological order corresponding to the recording time t. The example of FIG. 4 also shows an example in which the terminal device 10 processes internally at 60 fps (a frame rate that is, for example, twice the generated frame rate) when generating a moving image of 30 fps (generated frame rate). Specifically, the terminal device 10 generates a moving image VD1 with a frame rate of 30 fps by treating two captured videos as a set and generating one frame image for each set.

According to the example in FIG. 4, at recording time t=0, which is the time position when recording starts during recording time t, the terminal device 10 captures the image of the imaging area AR with the exposure for capturing an entire image while focusing on the entire imaging area AR, and obtains an entire focus image G10. Also, at the time of recording time t=0, the vehicle ahead is not captured in the entire focus image G10, and the position of the license plate has not been identified.

In addition, at recording time t=T1, which is the time position for the next imaging timing according to the passage of recording time t, the terminal device 10 captures the imaging area AR with the exposure amount for overall imaging while focusing on the entire imaging area AR, and obtains an overall focus image G11. Furthermore, even at the time of recording time t=T1, the vehicle ahead is not captured in the overall focus image G11, and the position of the license plate has not been identified.

In this state, in the example of FIG. 4, the terminal device 10 generates a frame image F#1 by combining the whole focus image G10 and the whole focus image G11.

The terminal device 10 also captures the image of the imaging area AR with the exposure for overall imaging while focusing on the entire imaging area AR at recording time t=T2, which is the time position for the next imaging timing according to the passage of recording time t, and obtains an overall focus image G12. Here, according to the example of FIG. 4, at the time of recording time t=T2, the vehicle ahead is captured in the overall focus image G12, and the position of the license plate is identified.

When the position of the license plate is identified in this manner, at recording time t=T3, which is the time position for the next imaging timing in accordance with the passage of recording time t, the terminal device 10 images the imaging area AR with the exposure amount for number imaging while focusing on the number area ARNB showing the identified number, and obtains a number focus image G13.

In this state, in the example of FIG. 4, the terminal device 10 generates a frame image F#2 by combining the whole focus image G12 and the number focus image G13.

In addition, while the position of the license plate is identified due to the inclusion of a vehicle ahead, the terminal device 10 repeats capturing an overall image of the imaging area AR using the exposure amount for overall imaging and capturing an image of the license plate of the imaging area AR using the exposure amount for license plate imaging.

Specifically, at recording time t=T4, which is the time position for the next imaging timing according to the passage of recording time t, the terminal device 10 captures the imaging area AR with the exposure amount for overall imaging while focusing on the entire imaging area AR, and obtains an overall focus image G14. In addition, since the vehicle ahead is reflected in the overall focus image G14 at recording time t=T4, the position of the license plate is identified.

For this reason, at recording time t=T5, which is the time position for the next imaging timing in response to the passage of recording time t, the terminal device 10 captures the imaging area AR with the exposure amount for capturing the number while focusing on the number area ARNB indicating the identified number, and obtains a number focus image G15.

In this state, in the example of FIG. 4, the terminal device 10 generates a frame image F#3 by combining the whole focus image G14 and the number focus image G15.

At recording time t=T6, which is the time position for the next imaging timing according to the passage of recording time t, the terminal device 10 captures an image of the imaging area AR with the exposure amount for overall imaging while focusing on the entire imaging area AR, and obtains an overall focus image G16. At recording time t=T6, the vehicle ahead is captured in the overall focus image G16, so the position of the license plate is identified.

For this reason, at recording time t=T7, which is the time position for the next imaging timing in response to the passage of recording time t, the terminal device 10 captures the imaging area AR with the exposure amount for capturing the number while focusing on the number area ARNB indicating the identified number, and obtains a number focus image G17.

In this state, in the example of FIG. 4, the terminal device 10 generates a frame image F#4 by combining the whole focus image G16 and the number focus image G17.

Furthermore, at recording time t=T8, which is the time position for the next imaging timing according to the passage of recording time t, the terminal device 10 captures an image of the imaging area AR with the exposure amount for overall imaging while focusing on the entire imaging area AR, and obtains an overall focus image G18. Also, at recording time t=T8, the vehicle ahead is reflected in the overall focus image G18, so the position of the license plate is identified.

For this reason, at recording time t=T9, which is the time position for the next imaging timing in response to the passage of recording time t, the terminal device 10 captures the imaging area AR with the exposure amount for number imaging while focusing on the number area ARNB indicating the identified number, and obtains a number focus image G19.

In this state, in the example of FIG. 4, the terminal device 10 generates a frame image F#5 by combining the whole focus image G18 and the number focus image G19.

Then, as shown in FIG. 4, the terminal device 10 generates a moving image VD1 by combining the frame images F#1, F#2, F#3, F#4, and F#5 that have been generated thus far.

3. Processing Procedure
From here, the procedure of the generation process according to the first embodiment performed by the terminal device 10 will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the procedure of the generation process according to the first embodiment. In Fig. 5, a scene will be described assuming that an image of the view in front of the windshield of a vehicle is recorded and photographed by the terminal device 10 while the vehicle is being driven at night.

In the example of FIG. 5, it is assumed that a user driving a vehicle equipped with a terminal device 10 first operates the terminal device 10 to start recording an image of the view in front of the windshield.

Then, the setting unit 16c sets an exposure amount that can capture more light in accordance with the fact that the entire image captured through the imaging area (an example of the second area) is dark, as an overall imaging exposure amount (an example of the second exposure amount) to be used for overall imaging, which is imaging focused on the entire imaging area (step S101).

Then, the setting unit 16c controls the sensor unit 13 to capture an entire image of the imaging area with the exposure amount for entire imaging set in step S101 (step S102). As a result, the sensor unit 13 captures the imaging area AR with the exposure amount for entire imaging while focusing on the entire imaging area (with the exposure point set on the entire imaging area).

The acquisition unit 16d acquires an entire focus image (an example of a second captured image), which is an image captured in its entirety by the sensor unit 13 in response to the imaging control in step S102 (step S103).

Next, the identification unit 16b performs image analysis on the overall focus image acquired in step S103 to detect whether or not a forward vehicle, which is a vehicle in front of the windshield, is captured in the overall focus image (step S104). If the forward vehicle is not captured in the overall focus image and the identification unit 16b cannot detect the forward vehicle (step S104; No), the identification unit 16b executes the process from step S102.

On the other hand, if the identification unit 16b detects a forward vehicle in the entire focus image (step S104; Yes), the identification unit 16b determines whether or not blown-out highlights have occurred in the vehicle image based on whether or not the brightness of the image corresponding to the vehicle area showing the outline of the detected vehicle, i.e., the vehicle image, exceeds a predetermined value (step S105). If the identification unit 16b determines that blown-out highlights have not occurred in the vehicle image (step S105; No), the identification unit 16b executes the processing from step S102.

On the other hand, if the identification unit 16b determines that whiteout has occurred in the vehicle image (step S105; Yes), it controls the brightness of the overall focus image acquired in step S103 to identify the position of the number area (an example of a first area) indicating the shape of the license plate in the vehicle image included in the overall focus image (step S106).

Next, when the position of the number area is identified by the identification unit 16b, the setting unit 16c sets the exposure amount (exposure amount for number image capture) to be used for capturing the number plate image, which is an image focused on the number area, based on the luminance at the time when the position of the number area is identified (step S107).

The setting unit 16c also controls the sensor unit 13 to capture an image of the license plate in the imaging area with the exposure amount for capturing the license plate number set in step S107 (step S108). As a result, the sensor unit 13 captures an image of the imaging area with the exposure amount for capturing the license plate number, with the license plate number area being focused (the license plate number area exposure point being set).

The acquisition unit 16d acquires a number focus image (an example of a first captured image), which is an image of the license plate captured by the sensor unit 13 in response to the imaging control in step S108 (step S109).

Next, the generation unit 16e determines whether the upper limit of the recording time has been reached or recording has been stopped (step S110). If the generation unit 16e determines that the upper limit of the recording time has not been reached and recording has not been stopped (step S110; No), the generation unit 16e executes the process from step S102. As a result, while the position of the license plate is identified and it is determined that blown-out highlights have occurred at the identified position, overall imaging of the imaging area using the overall imaging exposure amount and imaging of the license plate of the imaging area using the license plate imaging exposure amount are repeated.

On the other hand, if the generating unit 16e determines that the upper limit of the recording time has been reached or that recording has been stopped (step S110; Yes), it generates a frame image by combining pairs of captured images (pairs in chronological order) acquired so far as the recording time has elapsed (step S111) by adding together the paired images for each pair.

The generating unit 16e then generates one moving image by synthesizing each of the frame images generated for each set of paired images according to the time series (step S112). The generating unit 16e also stores the generated moving image in the imaging information storage unit 12a as a recorded video.

In the example of FIG. 5, when the identification unit 16b detects a vehicle ahead in the overall focus image, the identification unit 16b determines whether or not the vehicle image is blown out based on whether or not the luminance of the vehicle image exceeds a predetermined value. Also, when it is determined that the vehicle image is blown out, the generation unit 16e performs image synthesis using the overall focus image and number focus image acquired so far. However, as long as the identification unit 16b can detect a vehicle in the overall focus image, it is not necessary for the identification unit 16b to determine whether or not blown out highlights have occurred. In other words, when a vehicle is detected in the overall focus image, the generation unit 16e may perform image synthesis regardless of the presence or absence of blown out highlights.

In the example of FIG. 5, the identification unit 16b identifies the position of the number area showing the shape of the license plate in the vehicle image, and the setting unit 16c controls the sensor unit 13 to focus on this number area and capture the image, and as a result, the generation unit 16e performs image synthesis using the whole focus image and the number focus image. However, if the identification unit 16b cannot identify the position of the number area showing the shape of the license plate in the vehicle image, the setting unit 16c may control the sensor unit 13 to focus on the vehicle area showing the outer shape of the vehicle detected by the identification unit 16b and capture the image. In such a case, the generation unit 16e can perform image synthesis using the whole focus image and the vehicle focus image captured with the focus on the vehicle area.

4. Modifications
The terminal device 10 according to the first embodiment may be implemented in various different forms other than the above-described first embodiment.

4-1. Extracting object images
In the above-mentioned embodiment 1, the generating unit 16e generates a frame image by combining an entire focus image by whole image capture using the exposure amount for whole image capture and a number focus image by number plate image capture using the exposure amount for number image capture. However, the generating unit 16e may cut out a number image (an example of a corresponding image corresponding to a predetermined object) which is an image of a number plate indicated by the number area ARNB (first area) from the number focus image (first captured image), and generate one frame image by superimposing the cut-out number image on the entire focus image (second captured image). This point will be described with reference to FIG. 6. FIG. 6 is a diagram showing an example of a modified example according to embodiment 1.

In FIG. 6, an example of this modified example will be described using the example in FIG. 1. According to the example in FIG. 6, the generation unit 16e cuts out a number image G13a, which is an image of the number plate indicated by the number area ARNB, from the number focus image G13 obtained by capturing the number plate in step S13 in FIG. 1.

Then, the generating unit 16e generates one frame image by superimposing the number image G13a on the position of the number area ARNB identified in step S12 for the entire focus image G12 obtained by the entire image capture in step S11 in FIG. 1. The generating unit 16e also generates frame images by performing the same process for other recording times, and generates a moving image VD1 by synthesizing the generated frame images.

[4-2. Multiple sensors]
As shown in FIG. 2, the terminal device 10 may have a plurality of sensors. For example, the terminal device 10 may have two sensors, such as a first sensor 13a and a second sensor 13b. In such a case, the first sensor 13a (an example of a first sensor) can capture an image of the imaging area AR with an exposure amount for number image capture (an example of a first exposure amount) under the control of the control unit 16, and the second sensor 13b (an example of a second sensor) can capture an image of the imaging area AR with an exposure amount for whole image capture (an example of a second exposure amount) under the control of the control unit 16. According to this example, the generation unit 16e generates a moving image at a predetermined frame rate by synthesizing a number focus image captured by the first sensor 13a with an exposure amount for number image capture and an entire focus image captured by the second sensor 13b with an exposure amount for whole image capture.

Note that the number plate imaging with the exposure amount for number plate imaging by the first sensor 13a and the overall imaging with the exposure amount for overall imaging by the second sensor 13b may be performed at different times, as in the case where one sensor unit 13 is used, or may be performed at the same time, for example, without being limited to this example.

[4-3. Multiple Objects]
Furthermore, when a plurality of objects of different types are included in the imaging area AR as the predetermined object, the setting unit 16c may set the first exposure amount corresponding to each of the plurality of objects. In this case, the generating unit 16e generates a moving image by synthesizing the images of each of the plurality of objects captured with the first exposure amount with the second captured image.

For example, as shown in FIG. 1, when an overall focus image G12 is acquired, the identification unit 16b not only identifies the position of a number area showing the shape of a license plate in the overall focus image G12, but also identifies the position of a person area showing the shape of a person (which may be other traffic-related objects such as a sign, a traffic signal, a crosswalk, etc.). For example, when the identification unit 16b can detect a person by image analysis of the overall focus image G12, it can identify the position of the person area of the detected person in more detail by brightness control of the overall focus image G12.

In addition, in this state, the setting unit 16c sets a first exposure amount corresponding to the number area based on the luminance at the time when the position of the number area is identified, and also sets a first exposure amount corresponding to the person area based on the luminance at the time when the position of the person area is identified.

The setting unit 16c then controls the sensor unit 13 to capture an image of the imaging area AR using a first exposure amount corresponding to the number area while focusing on the number area, and capture an image of the imaging area AR using a first exposure amount corresponding to the person area while focusing on the person area. As a result, the acquisition unit 16d acquires a number focus image and a person focus image in addition to the whole focus image G12.

Therefore, the generating unit 16e generates one frame image by combining the whole focus image G12, the number focus image, and the person focus image.

The setting unit 16c may set a single first exposure amount that is optimal for both the number area and the person area based on the luminance at the time when the position of the number area is identified and the luminance at the time when the position of the person area is identified.

[4-4. Objects of the same type across multiple targets]
In the above embodiment 1, an example has been shown in which the terminal device 10 identifies the position of the license plate area of one forward vehicle in the captured image in which the imaging area AR is captured, and captures the imaging area AR with exposure applied to the identified license plate area. However, the captured image in which the imaging area AR is captured may include multiple forward vehicles. In such a case, the terminal device 10 may identify the position of the license plate area of each forward vehicle included in the captured image, and capture the imaging area AR with exposure applied to each identified license plate area.

(Embodiment 2)
1. Overview of the Second Embodiment
Next, a second embodiment will be described. In the first embodiment, an example was shown in which the terminal device 10 sets a first exposure amount corresponding to a predetermined object when the predetermined object is included in the imaging area, and sets a second exposure amount corresponding to a second area of the imaging area that is different from the first area. In the first embodiment, an example was shown in which the terminal device 10 generates a moving image by synthesizing a first captured image including a first area captured with the first exposure amount and a second captured image including a second area captured with the second exposure amount.

In response to this, the terminal device 10A according to the second embodiment performs the following processing as a generation processing according to the second embodiment. Specifically, the terminal device 10A sets a first exposure amount corresponding to a first region indicating a predetermined object and a second exposure amount corresponding to a second region based on a format in which the positional relationship between a first region indicating a predetermined object and a second region that is an area of the image capture area other than the first region is predefined. The terminal device 10A then generates a moving image by combining a first captured image including the first region captured with the first exposure amount and a second captured image including the second region captured with the second exposure amount.

In the following description, the same components as those in the first embodiment described above are given the same reference numerals, and detailed descriptions thereof will be omitted or simplified.

2. Configuration of terminal device
A terminal device 10A according to the second embodiment will be described with reference to Fig. 7. Fig. 7 is a diagram showing an example of the configuration of the terminal device 10A according to the second embodiment. As shown in Fig. 7, the terminal device 10A has a communication unit 11, a storage unit 12A, a sensor unit 13, an input unit 14, a display unit 15, and a control unit 16A. The terminal device 10A according to the second embodiment newly has a format information storage unit 12b and an exposure amount setting unit 16f compared to the terminal device 10 according to the first embodiment.

(Regarding the storage unit 12A)
The storage unit 12A is realized by, for example, a semiconductor memory element such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk, etc. The storage unit 12A has an imaging information storage unit 12a and a format information storage unit 12b.

(Regarding the format information storage unit 12b)
The format information storage unit 12b stores information about a format in which a positional relationship between a first area and a second area is defined. Specifically, the format information storage unit 12b stores a plurality of formats in which a positional relationship between a first area and a second area with respect to an imaging area is defined in advance.

Here, FIG. 8 shows an example of the format information storage unit 12b according to the second embodiment. In the example of FIG. 8, the format information storage unit 12b has items such as "Format ID (Identifier)" and "Format Data."

"Format ID" indicates identification information that identifies "format data." "Format data" indicates actual data of a format in which the positional relationship between the first area and the second area is specified.

(Regarding the control unit 16A)
The control unit 16A is realized by a CPU, an MPU, or the like executing various programs (including the generation program according to the second embodiment) stored in a storage device inside the terminal device 10A using a RAM as a working area. The control unit 16A is also realized by an integrated circuit such as an ASIC or an FPGA.

The control unit 16 is also configured with, for example, the following processing units to realize the generation process according to the second embodiment, in which a first captured image captured with a first exposure amount and a second captured image captured with a second exposure amount are acquired, and a moving image is generated by synthesizing an image corresponding to a first region showing a predetermined object included in the first captured image with an image corresponding to a second region in the second captured image that is different from the first region.

As shown in FIG. 7, the control unit 16A has a recording control unit 16a, an identification unit 16b, a setting unit 16c, an acquisition unit 16d, a generation unit 16e, and an exposure setting unit 16f, and realizes or executes the functions and actions of the information processing described below. Note that the internal configuration of the control unit 16A is not limited to the configuration shown in FIG. 7, and may be other configurations as long as they perform the information processing described below. Also, the connection relationship between each processing unit of the control unit 16A is not limited to the connection relationship shown in FIG. 7, and may be other connection relationships.

(Regarding the exposure amount setting unit 16f)
The exposure setting unit 16f sets a first exposure amount and a second exposure amount for the imaging area based on a format in which the positional relationship between a first area indicating a specified object and a second area which is an area of the imaging area other than the first area is predefined.

In the second embodiment, the acquisition unit 16d acquires a first captured image captured with a first exposure amount and a second captured image captured with a second exposure amount in accordance with the exposure amount setting by the exposure amount setting unit 16f. The generation unit 16e then generates a moving image by synthesizing the first captured image as an image corresponding to the first region and the second captured image as an image corresponding to the second region.

The exposure setting unit 16f also applies a format to the captured image in which the imaging area is captured, thereby identifying a first area and a previous two areas in the captured image, and sets a first exposure amount and a second exposure amount based on the luminance of each identified area. For example, the exposure setting unit 16f sets a first exposure amount for the first area based on whether the luminance of the identified first area is higher than a predetermined value, and sets a second exposure amount for the identified second area based on whether the luminance of the identified second area is higher than a predetermined value.

In addition, the exposure setting unit 16f selects a format to be used from among a plurality of formats in response to changes in the captured image in which the imaging area is captured, and sets the first exposure amount and the second exposure amount using the selected format.

[3. About the format]
Here, the format according to the second embodiment will be described. As described above, in the format according to the second embodiment, the positional relationship between the first region indicating a predetermined object and the second region, which is a region of the imaging region other than the first region, is predefined. For example, the main purpose of a drive recorder is to capture other vehicles (e.g., a vehicle ahead) present around the vehicle in which it is installed, and the situation of the surrounding environment of the vehicle (e.g., people, signs, traffic lights, crosswalks, etc.), and it is generally installed in front of the windshield for such purposes. In this case, the image captured through the imaging region is roughly configured to include a roadway ahead, and various surrounding environments are present around the roadway.

Utilizing this configuration, the format according to the second embodiment specifies the positional relationship between a road area (an example of a first area) indicating the shape of the road (an example of a specified object) that is visible ahead and on which the vehicle is traveling, and a surrounding area (an example of a second area) that is the area of the image capture area AR other than the road area. For example, as shown in FIG. 9, in the format according to the second embodiment, the position of the center of the front of the image capture area AR is defined as the position of the road area, and the surrounding area is defined as a position that surrounds this road area. The surrounding area includes the streetscape, people, signs, traffic lights, crosswalks, etc. that are visible ahead.

4. Processing Procedure
Hereinafter, an example of the generation process according to the second embodiment performed by the terminal device 10A will be described with reference to Fig. 9. Fig. 9 is a diagram showing an example of the generation process according to the second embodiment.

Figure 9 shows a scene in which the user places the terminal device 10A in front of the windshield so that the user can capture an image of the area in front of the vehicle, and causes the terminal device 10A to start recording, for example, while driving at night. In such a case, the terminal device 10A captures and records an image of the area in front of the vehicle through the imaging area AR.

For example, when recording is started, the exposure setting unit 16f selects a format to be used from among the formats stored in the format information storage unit 12b in response to a change in the image captured via the imaging area AR (e.g., an image captured of the imaging area AR). The example in FIG. 9 shows an example in which the exposure setting unit 16f selects format FM1 as the format to be used in response to a change in the image captured via the imaging area AR.

According to the format FM1 shown in FIG. 9, the positional relationship between the road area ARRD, which indicates the shape of a straight road when the road is visible ahead with respect to the imaging area AR, and the surrounding area ARSD, which is the area of the imaging area AR other than the road area ARRD, is specified. For example, in the format FM1, the position of the road area ARRD is defined as the center position ahead with respect to the imaging area AR, and the surrounding area ARSD is defined as a position surrounding this road area.

Here, for example, if the exposure setting unit 16f determines from the image captured via the imaging area AR that the vehicle is currently traveling on a straight road, it can select format FM1 as shown in FIG. 9 as the format to be used. On the other hand, if the exposure setting unit 16f determines from the image captured via the imaging area AR that the vehicle is currently traveling on a curved road, it can select another format in which the positional relationship between the road area ARRD, which shows the shape of the curved road when it is visible ahead, and the surrounding area ARSD is specified.

In this way, the shape of the road included in the captured image of the imaging area AR depends on the state of the road on which the vehicle is traveling, such as whether the vehicle is traveling on a straight road or a curved road. Therefore, the exposure setting unit 16f can select the format to be used from among multiple formats based on the state (shape) of the road determined according to changes in the captured image of the imaging area AR.

In the example of FIG. 9 where format FM1 is selected, the exposure setting unit 16f applies format FM1 to the captured image in which the imaging area AR is captured (step S91), thereby identifying the road area ARRD and the surrounding area ARSD in the captured image (step S92).

Next, the exposure setting unit 16f sets the exposure corresponding to each identified region based on the brightness of the image within that region (step S93).

For example, the exposure setting unit 16f sets the road imaging exposure amount (an example of a first exposure amount) to be used for road focus imaging, which is imaging focused on the road area ARRD, based on the brightness of the image in the road area ARRD. As an example, the exposure setting unit 16f sets the road imaging exposure amount to be used for road focus imaging, which is imaging focused on the road area ARRD, to the road area ARRD, based on whether the brightness of the image in the road area ARRD is higher than a predetermined value. For example, when the exposure setting unit 16f can determine that whiteout occurs in the image in the road area ARRD because the brightness of the image in the road area ARRD is higher than a predetermined value, it can set the road imaging exposure amount to an exposure amount lowered to an extent that whiteout does not occur. On the other hand, when the exposure setting unit 16f can determine that blackout occurs in the image in the road area ARRD because the brightness of the image in the road area ARRD is lower than a predetermined value, it can set the road imaging exposure amount to an exposure amount raised to an extent that blackout does not occur.

The exposure setting unit 16f also sets the peripheral imaging exposure (an example of a second exposure) to be used for peripheral focus imaging, which is imaging focused on the peripheral area ARSD, based on the luminance of the image in the peripheral area ARSD. As an example, the exposure setting unit 16f sets the peripheral imaging exposure to be used for peripheral focus imaging, which is imaging focused on the peripheral area ARSD, to the peripheral area ARSD, based on whether the luminance of the image in the peripheral area ARSD is higher than a predetermined value. For example, when the exposure setting unit 16f can determine that whiteout occurs in the image in the peripheral area ARSD because the luminance of the image in the peripheral area ARSD is higher than a predetermined value, it can set the peripheral imaging exposure to an exposure amount lowered to an extent that whiteout does not occur. On the other hand, when the exposure setting unit 16f can determine that blackout occurs in the image in the peripheral area ARSD because the luminance of the image in the peripheral area ARSD is lower than a predetermined value, it can set the peripheral imaging exposure to an exposure amount raised to an extent that blackout does not occur.

Next, the exposure setting unit 16f controls the sensor unit 13 to capture the image of the imaging area AR with the exposure for road imaging set in step S93, and also controls the sensor unit 13 to capture the image of the imaging area AR with the exposure for peripheral imaging set in step S93 (step S94).

In addition, with such imaging control, the sensor unit 13 captures the road area ARRD with the exposure amount for road imaging while focused on the road area ARRD (with the exposure point set on the road area ARRD), and captures the surrounding area ARSD with the exposure amount for surrounding imaging while focused on the surrounding area ARSD (with the exposure point set on the surrounding area ARSD). For example, the sensor unit 13 repeats imaging in which road focus imaging using the exposure amount for road imaging and surrounding focus imaging using the exposure amount for surrounding imaging are performed at different times (alternately).

Furthermore, according to step S94, the acquisition unit 16d acquires a road focus image (an example of a first captured image) which is an image captured by the sensor unit 13 with a road focus, and a peripheral focus image (an example of a second captured image) which is an image captured by the sensor unit 13 with a peripheral focus.

In addition, in this state, it is assumed that the generation unit 16e determines that the upper limit of the recording time has been reached or that recording has been stopped. In this case, the generation unit 16e generates a frame image by combining pairs of captured images (pairs in chronological order) acquired so far as the recording time has passed, by adding together the paired images for each pair (step S95). For example, if the sensor unit 13 alternately performs road focus imaging and peripheral focus imaging, and the acquisition unit 16d acquires paired images consisting of a road focus image and a peripheral focus image, the generation unit 16e generates one focus image by combining the road focus image and the peripheral focus image by adding together the paired images.

In step S94, the sensor unit 13 may simultaneously perform road focus imaging using the exposure amount for road imaging and peripheral focus imaging using the exposure amount for peripheral imaging, in accordance with imaging control by the exposure amount setting unit 16f. In this case, the acquisition unit 16d can acquire one captured image composed of the road focus imaging and peripheral focus imaging obtained by simultaneous imaging. In addition, in this case, the generation unit 16e can use each captured image acquired so far as one frame image according to the passage of recording time.

Finally, the generating unit 16e generates one moving image by synthesizing each generated frame image (step S96). The generating unit 16e also stores the generated moving image in the imaging information storage unit 12a as a recorded video.

As described above in FIG. 9, the terminal device 10A according to the second embodiment sets, for the imaging area AR, an exposure amount for road imaging (first exposure amount) corresponding to the road area ARRD and an exposure amount for peripheral imaging (second exposure amount) corresponding to the peripheral area ARSD based on a format FM1 in which the positional relationship between the road area ARRD (first area) showing the shape of the road (predetermined object) and the peripheral area ARSD (second area) which is the area of the imaging area AR other than the road area is predefined. Then, the terminal device 10A generates a moving image that combines a road focus image (first captured image) captured with the exposure amount for road imaging and a peripheral focus image (second captured image) captured with the exposure amount for peripheral imaging.

With such a terminal device 10A, for example, in an environment where whiteout or blackout occurs, it becomes possible to record a video that is clearly captured without the road being obscured by whiteout or blackout. Furthermore, since vehicles (e.g., a vehicle ahead) that are present around the vehicle on which the terminal device 10A is installed are traveling on this road, the terminal device 10A ultimately makes it possible to record a video that is clearly captured without the license plate being obscured by whiteout or blackout.

Other Embodiments
A user of the terminal device 10 according to the first embodiment and the terminal device 10A according to the second embodiment can view the video stored as a recorded video on a recording medium by connecting a recording medium (e.g., an SD card or a USB memory) corresponding to the imaging information storage unit 12a to his/her own personal computer or smartphone. For example, a user can display a dedicated screen on a personal computer or smartphone by connecting a recording medium to a personal computer or smartphone on which a specific application (e.g., a video viewer for a drive recorder) is installed. On the dedicated screen, the user can perform operations such as playback, fast forward, rewind, pause, and skip on the video. In addition, the dedicated screen may display the position information of the driving vehicle at the time when the currently displayed video was captured and the speed information of the driving vehicle.

The terminal device 10 according to the first embodiment and the terminal device 10A according to the second embodiment may have a function of displaying the above-mentioned dedicated screen on the display unit 15, which is a display. This allows the user to view the dedicated screen directly from the terminal device 10 (10A) without connecting an SD card or USB memory to a personal computer or smartphone. In this case, the terminal device 10 (10A) may have a display control function of enlarging and displaying an image in an area showing a specific object among the moving images displayed on the dedicated screen. For example, a display control unit (not shown) of the terminal device 10 (10A) enlarges and displays (wipes) an image in a number area showing a license plate (i.e., an image of the license plate) among the moving images displayed on the dedicated screen on the dedicated screen. This point will be described with reference to FIG. 10. FIG. 10 is a diagram showing an example of display control by the terminal device 10 (10A).

Figure 10 shows an example in which the terminal device 10 according to the first embodiment displays a dedicated screen D corresponding to a video viewer on the display unit 15. Also, in the example of Figure 10, the terminal device 10 displays a video image VD2 of a vehicle ahead on the dedicated screen D. However, in such a state, the license plate of the vehicle ahead may be small and the user may not be able to recognize the registration number. Therefore, the terminal device 10 enlarges (wipes) the image within the number area ARNB indicating the license plate on the dedicated screen D, out of the captured image currently being displayed as the video image VD2. For example, the terminal device 10 can enlarge and display the image within the number area ARNB in response to a user operation.

Such a terminal device 10 can realize display control according to the needs of the user in a drive recorder (e.g., to capture surrounding vehicles and people), thereby further improving the convenience of video recording.

(Hardware configuration)
The terminal device 10 according to the first embodiment and the terminal device 10A according to the second embodiment described above are realized, for example, by a computer 1000 having a configuration as shown in Fig. 11. The terminal device 10 will be described below as an example. Fig. 11 is a hardware configuration diagram showing an example of a computer that realizes the functions of the terminal device 10. The computer 1000 has a CPU 1100, a RAM 1200, a ROM 1300, a HDD 1400, a communication interface (I/F) 1500, an input/output interface (I/F) 1600, and a media interface (I/F) 1700.

The CPU 1100 operates based on a program stored in the ROM 1300 or the HDD 1400, and controls each component. The ROM 1300 stores a boot program executed by the CPU 1100 when the computer 1000 is started, and programs that depend on the hardware of the computer 1000, etc.

HDD 1400 stores programs executed by CPU 1100 and data used by such programs. Communication interface 1500 receives data from other devices via a specified communication network and sends it to CPU 1100, and transmits data generated by CPU 1100 to other devices via the specified communication network.

The CPU 1100 controls output devices such as a display and a printer, and input devices such as a keyboard and a mouse, via the input/output interface 1600. The CPU 1100 acquires data from the input devices via the input/output interface 1600. The CPU 1100 also outputs generated data to the output devices via the input/output interface 1600.

The media interface 1700 reads a program or data stored in the recording medium 1800 and provides it to the CPU 1100 via the RAM 1200. The CPU 1100 loads the program from the recording medium 1800 onto the RAM 1200 via the media interface 1700 and executes the loaded program. The recording medium 1800 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase change rewritable Disc), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

For example, when the computer 1000 functions as the terminal device 10 according to the first embodiment, the CPU 1100 of the computer 1000 executes programs loaded onto the RAM 1200 to realize the functions of the control unit 16. The CPU 1100 of the computer 1000 reads and executes these programs from the recording medium 1800, but as another example, the CPU 1100 may obtain these programs from another device via a specified communication network.

For example, when the computer 1000 functions as the terminal device 10A according to embodiment 2, the CPU 1100 of the computer 1000 executes a program loaded onto the RAM 1200 to realize the functions of the control unit 16A.

Although not shown in FIG. 10, the computer 1000 may further include a sensor, and when the computer 1000 functions as the terminal device 10 (10A), this sensor captures images under the control of the CPU 1100.

(others)
Furthermore, among the processes described in the above embodiments and modifications, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by a known method. In addition, the information including the processing procedures, specific names, various data and parameters shown in the above documents and drawings can be changed arbitrarily unless otherwise specified. For example, the various information shown in each drawing is not limited to the illustrated information.

In addition, each component of each device shown in the figure is a functional concept, and does not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads and usage conditions. For example, in each of the above embodiments, an example was shown in which the terminal device 10 (10A) is installed in front of the windshield to capture an image of a vehicle ahead, but the terminal device 10 (10A) may be installed in front of the back door glass to capture an image of a vehicle behind.

In addition, in each of the above embodiments, an example has been shown in which the terminal device 10 (10A) operates as a standalone device, but the generation process according to embodiment 1 or the generation process according to embodiment 2 may be realized by linking with an external server device. In such a case, the server device may be configured to have all or any of the processing units shown in Figures 2 and 7.

The above-described embodiments and variations can be combined as appropriate to the extent that they do not cause inconsistencies in the processing content.

Although several embodiments of the present application have been described in detail above with reference to the drawings, these are merely examples, and the present invention can be embodied in other forms that incorporate various modifications and improvements based on the knowledge of those skilled in the art, including the forms described in the disclosure section of the invention.

The above-mentioned "section, module, unit" can be read as "means" or "circuit." For example, a setting section can be read as a setting means or a setting circuit.

REFERENCE SIGNS LIST 10 Terminal device 10A Terminal device 12a Imaging information storage unit 12b Format information storage unit 13 Sensor unit 13a First sensor 13b Second sensor 14 Input unit 15 Display unit 16 Control unit 16A Control unit 16a Recording control unit 16b Identification unit 16c Setting unit 16d Acquisition unit 16e Generation unit 16f Exposure amount setting unit

Claims (5)

an exposure amount setting means for setting a first exposure amount corresponding to a first area and a second exposure amount corresponding to a second area, which is an area other than the road, based on a format in which a first area corresponds to a road on which the vehicle is traveling and a second area is a region other than the road, in a photographing area of an imaging unit that photographs the area ahead of the vehicle;
an acquisition means for acquiring a first captured image in which the photographing area is photographed with the first exposure amount and a second captured image in which the photographing area is photographed with the second exposure amount;
A terminal device characterized by having a generation means for generating a moving image by combining an image corresponding to the first area included in the first captured image with an image corresponding to the second area included in the second captured image. A storage means for storing a plurality of said formats,
The terminal device according to claim 1, characterized in that the exposure setting means selects a format to be used from among the plurality of formats in response to a change in an image captured of the shooting area, and sets the first exposure and the second exposure using the selected format. the storage means stores, as the plurality of formats, a first format in which the first region corresponds to a straight road and a second format in which the first region corresponds to a curved road;
The exposure amount setting means selects the first format as the format to be used when the vehicle is traveling on a straight road, and selects the second format as the format to be used when the vehicle is traveling on a curved road.
3. The terminal device according to claim 2. A generating method executed by a terminal device, comprising:
an exposure amount setting step of setting a first exposure amount corresponding to a first area, which corresponds to a road on which the vehicle is traveling, and a second exposure amount corresponding to a second area, which is an area other than the road, based on a format in which a first area corresponds to a road on which the vehicle is traveling and a second area corresponds to the second area, in a photographing area of an imaging unit that photographs the area ahead of the vehicle;
acquiring a first captured image in which the photographing area is photographed with the first exposure amount and a second captured image in which the photographing area is photographed with the second exposure amount ;
a generating step of generating a moving image by combining an image corresponding to the first region included in the first captured image with an image corresponding to the second region included in the second captured image. an exposure amount setting step of setting a first exposure amount corresponding to a first area, which corresponds to a road on which the vehicle is traveling, and a second exposure amount corresponding to a second area, which is an area other than the road, based on a format in which a first area corresponds to a road on which the vehicle is traveling and a second area corresponds to a second area, in a photographing area of an imaging unit that photographs an area ahead of the vehicle;
an acquisition step of acquiring a first captured image in which the imaging area is captured with the first exposure amount and a second captured image in which the imaging area is captured with the second exposure amount;
a generation step of generating a moving image by synthesizing an image corresponding to the first region included in the first captured image with an image corresponding to the second region included in the second captured image.

Images