JP7659891B2 - Systems, information processing methods, programs, etc. - Google Patents

Description

The present invention relates to an information processing device, an information processing method, a program, etc.

A technology has been disclosed in which, when a user selects the indoor highlight button, brightness correction within a position is performed on video data showing a dark image displayed in the video display area (Patent Document 1).

JP 2019-129530 A

However, the technology in Patent Document 1 only had buttons on the screen to perform image processing on video data, and was unable to perform the image processing required by users in a variety of different situations.

In consideration of the above circumstances, the present invention aims to provide a system that is superior to conventional systems, such as a system that can perform appropriate image processing based on the state or situation of a moving object.

The object of the present invention is not limited to this, and the applicant intends to obtain rights to configurations that aim to obtain effects from parts of the configurations disclosed in this specification and drawings, etc., through divisional applications, amendments, etc. For example, this specification discloses problems in which the phrases "can" and "is possible" are read as "the problem is". The problems are described as being independent, and the applicant intends to obtain rights to the configurations for solving each problem separately through divisional applications, amendments, etc. Even if the problems are implicitly understood from the description in the specification, the applicant intends to include part of the configurations described in this specification in the scope of the patent claims through amendments or divisional applications. The applicant has also disclosed configurations that solve problems that combine these independent problems, and the applicant intends to obtain rights to them.

(1) According to one aspect of the present invention, a system is provided. The system may include a control unit. The control unit may be configured to execute the following steps. In the acquisition step, specific information regarding a state or situation of the moving body may be acquired. In the first correction step, a first correction process may be executed on image data generated by a photographing device possessed by the moving body at a time corresponding to the specific information, based on the specific information. In the display control step, the image data on which the first correction process has been executed may be displayed in a reproducible manner.

In this way, the user can view an image reproduced from image data after the first correction process has been performed on image data generated by the photographing device at a time corresponding to the state or situation of the moving object.
In particular, the correction process for the image data may be appropriate image processing. The image processing may be correction processing for correcting the image data to be easier for the user to view than image data generated by an image capture device possessed by the moving object. In particular, the correction process may be performed constantly. According to this aspect, since appropriate image processing can be performed based on the state or situation of the moving object, it is possible to realize a system capable of presenting image data that is easy for the user to view constantly, regardless of the situation or scene.

For example, it may be provided in the following forms:

(2) In the system, in the first correction step, the correction range, which is the range to be corrected, is variable based on the specific information, and the first correction process is performed on the image data.

According to this aspect, it is not necessary to correct the entire range, so the processing load can be reduced. In addition, because the correction is applied to a preferred range, the user's attention can be directed to the preferred range, regardless of the spatial or temporal range.

(3) In the system, the correction range is a region of interest set in a frame of the image data.

In this manner, correction can be applied to a narrower spatial area where visibility is desired, allowing the user's attention to be directed to the desired range on a frame-by-frame basis.

(4) In the system, the control unit is configured to further execute the following steps: in a first reception step, a first input by a user is received; and in a release step, the first correction process for the image data is released based on the first input.

According to this aspect, when the correction process presented by the system is not necessary, the user can disable it through their own input, thereby improving visibility and presenting an appropriate image according to the situation or scene.

(5) In the system, the control unit is configured to further execute the following steps: in a second reception step, a second input from a user is received; and in a second correction step, a second correction process is performed on the image data based on the second input.

According to this aspect, in addition to the correction process presented by the device, it is possible to improve visibility by adding corrections at the user's discretion, and it is possible to present images that match the user's preferences depending on the situation or scene.

(6) In the system, in the display control step, the image data is displayed in a reproducible manner in which the first correction process and the second correction process are distinguishable.

This aspect allows the user to understand the type of correction process, making it user-friendly.

(7) In the system, the specific information is at least one of the position, orientation, speed, acceleration, time, time zone, weather, driving condition, and pedal work of the moving object.

According to this aspect, at least one of the specific moving object conditions, namely position, speed, acceleration, running state, and pedal work, is taken into consideration, so that more appropriate image processing can be performed based on this information.

(8) In the system, the control unit is configured to further execute an acquisition step, the specific information being the position of the moving body, the acquisition step acquiring road network information, and the first correction step executing a first correction process on the image data based on the position and the road network information.

According to this aspect, appropriate image processing can be performed according to the location and the road conditions at that time. For example, by performing appropriate image processing according to the traffic volume obtained from the road network information, it is possible to present an image that is easier for the user to see.

(9) In the system, the control unit is configured to further execute an acquisition step, the specific information being the position of the moving body, the acquisition step acquiring topographical information, and the first correction step executing a first correction process on the image data based on the position and the topographical information.

According to this aspect, appropriate image processing can be applied depending on the location and the terrain, environment, etc. at that time. For example, by applying different image processing to a downtown area and a forest, it is possible to present an image that is easier for the user to see.

(10) In the system, the specific information is the position and orientation of the moving body, and in the first correction step, a first correction process is performed on the image data based on the position of the sun estimated from the position and orientation.

According to this aspect, appropriate image processing can be performed taking into account the position of the sun. For example, in a situation where the setting sun is shining in, the brightness of the area that is actually to be viewed tends to be reduced due to backlighting, so by performing image processing to correct this, it is possible to present the user with an image that is easy to view.

(11) In the system, the first correction step includes a preparation step and a processing step, the control unit is configured to be able to call a first program stored in advance and a second program different from the first program, the first program executes the preparation step, in which parameters included in the image data are extracted and the values of the extracted parameters are passed to the second program, the second program executes the processing step and the display control step, in which the first correction process is executed on the parameters to change their values, and in which the image data based on the parameter values changed in the processing step are displayed in a reproducible manner.

According to this aspect, a separate program that is more specialized for correction processing can be used, which contributes to reducing calculation costs and improving image processing performance. In particular, by not duplicating the original image data, etc., but rather passing the parameter values to the second program, the second program that is suitable for image processing can read the parameter values and reproduce image data that has been subjected to image processing in accordance with them.

(12) In the above system, the parameters include at least one of hue, brightness, saturation, contrast, gamma, sharpness, playback start time, playback end time, and whether or not to loop.

In this manner, the parameters such as hue, brightness, saturation, contrast, gamma, sharpness, playback start time, playback end time, and whether or not to loop are all typical parameters that image data has, so that a more general-purpose image processing program can be used as the second program, and a system with high usability that is not limited to dedicated software can be realized.

(13) In the system, in the preparation step, operation information regarding the operation of the second program is passed to the second program, and in the display control step, at least a portion of the functions of the second program is restricted based on the operation information, and the image data is displayed in a playable manner.

According to this aspect, the second program can be used solely for correction and playback, and the user does not need to use any extra functions of the second program, which can prevent or eliminate problems such as incorrect operation or the user finding the program difficult to use due to the large number of functions.

(14) In the system, the first program is configured to execute the display control step, and in the display control step, if the second program is not installed, a screen recommending the installation of the second program is displayed.

According to this aspect, even if the second program is not installed, the user can be informed of a recommended second program. As a result, the user can install the second program on his or her computer, etc., and play back image data that has been appropriately subjected to the first correction process.

(15) In the system, the second program is a program stored in an external device, and the control unit is configured to call the second program via a network.

According to this aspect, even if the second program is not installed, the first correction process can be performed via the second program. For example, even if the capacity of the computer being used is limited or a user does not have the authority to install new programs, the first correction process can be performed appropriately.

(16) A system including a control unit, the control unit configured to execute the following steps: in a recording step, an image captured by a camera is recorded as image data; in a display control step, the image captured by the camera is displayed as a live view; and parameters of the recorded image data and the displayed image are set to be different.

In this manner, the image viewed as a live view and the image data that is recorded can be made different, ensuring both visibility and customizability.

(17) In the system, in the recording step, the image is recorded as the image data without correction, and in the display control step, the image is corrected and displayed as the live view.

This aspect makes it possible to improve the visibility of the image viewed as a live view while preventing information loss due to processing of the recorded image data.

(18) In the system, in the recording step, parameters of the displayed image are recorded in chronological order together with the image data as a parameter log.

According to this embodiment, the visibility of the image viewed as a live view can be saved as a record. This information can be used later to reproduce the visibility when playing back the image data.

(19) In the system, the control unit is further configured to execute a third reception step, in which a third input from a user is received, and in the display control step, the image data and the parameter log are read out based on the third input, and the corrected image data is displayed in a reproducible manner.

According to this embodiment, if the user wishes, it is possible to instantly play back images with high visibility, such as those seen in live view, making it possible to realize a system with high usability.

(20) In the system, the control unit is configured to be able to call up a pre-stored third program and a fourth program different from the third program, and in the display control step, when the image data is played back in the fourth program, information is displayed that encourages playback in the third program.

According to this embodiment, even if an unintended program is played, it is possible to guide the user to play a preferred program, and appropriate information can be provided to the user.

(21) In the system, the information prompting playback is displayed together with the playback screen of the image data.

In this manner, it is possible to provide the user with appropriate information that the current appearance is undesirable, but that there is no problem with the image data itself.

(22) An information processing method comprising the steps of the system.

By using such a method, the functionality of the above-mentioned system can be realized and provided in the form of a method.

(23) A program that causes a computer to function as a control unit of the system.

According to this embodiment, the functions of the above-mentioned system can be realized and provided in the form of a program.
Of course, this is not the case.

The inventions shown in (1) to (23) above can be combined in any way. For example, the invention shown in (1) may be combined with at least a part of the configuration of at least one of the inventions from (2) onwards. In particular, the invention shown in (1) may be combined with at least a part of the configuration of at least one of the inventions from (2) onwards. Any configuration may be extracted from the inventions shown in (1) to (23) and the extracted configurations may be combined. The applicant of this application intends to acquire rights to the inventions including these configurations. Furthermore, even if there is a description such as "in the case of" or "when", this is not intended to describe the configuration as being limited to that case or time. These are examples of better configurations, and the applicant intends to acquire rights to configurations other than these cases or times. Furthermore, the parts described in order are not limited to this order. Configurations in which some parts have been deleted or the order has been changed are also disclosed, and the applicant intends to acquire rights to them.

The present invention makes it possible to perform appropriate image processing based on the state or situation of a moving object, thereby realizing a system that can consistently present image data that is easy for the user to view, regardless of the situation or scene.

Note that the effects of the invention of this application are not limited to these, and effects achieved from the configuration parts disclosed in this specification and drawings, etc. are also disclosed, and it is our intention to obtain rights to the configuration that achieves said effects through divisional applications, amendments, etc. For example, in this specification, statements such as "can do" and "is possible" are statements that clearly indicate the effect that is achieved, and there are also parts that show the effect even without statements such as "can do" and "is possible." Also, there are effects that can be understood from the configuration even without such statements.

FIG. 1 is a diagram illustrating an example of a system configuration of an information processing system according to the first embodiment. FIG. 2 is a diagram illustrating the configuration of the vehicle according to the first embodiment. FIG. 3 is a diagram showing an example of the external configuration of the imaging device. FIG. 4 is a diagram showing the configuration of the bracket. FIG. 5 is a block diagram showing the electrical configuration of the photographing device. FIG. 6 is a diagram illustrating an example of a hardware configuration of the server according to the first embodiment. FIG. 7 is a diagram illustrating an example of a hardware configuration of the information display terminal according to the first embodiment. FIG. 8 is a diagram illustrating an example of a functional configuration of the server according to the first embodiment. FIG. 9 is a diagram illustrating an example of a functional configuration of the information display terminal according to the first embodiment. FIG. 10 is an activity diagram showing the flow of information processing by the server according to the first embodiment. FIG. 11 is an activity diagram showing the flow of information processing in the information display terminal according to the first embodiment. FIG. 12 is a diagram illustrating an example of a screen before image processing, which is displayed by the display control unit according to the first embodiment. FIG. 13 is a diagram illustrating an example of a screen after image processing, which is displayed by the display control unit according to the first embodiment. FIG. 14 is a diagram illustrating an example of a screen after image processing, which is displayed by the display control unit of the first modified example of the first embodiment. FIG. 15 is a diagram illustrating an example of a screen before image processing, which is displayed by the display control unit in the second modification of the first embodiment. FIG. 16 is a diagram illustrating an example of a screen before image processing, which is displayed by the display control unit in the third modification of the first embodiment. FIG. 17 is a diagram illustrating an example of a screen before image processing, which is displayed by the display control unit of the fourth modified example of the first embodiment. FIG. 18 is a diagram illustrating an example of a screen before image processing, which is displayed by the display control unit of the fifth modified example of the first embodiment. FIG. 19 is a diagram illustrating an example of a screen before image processing, which is displayed by the display control unit of the sixth modification of the first embodiment. FIG. 20 is a diagram illustrating an example of a screen before image processing, which is displayed by the display control unit of the seventh modification of the first embodiment. FIG. 21 is a diagram illustrating an example of a screen before image processing, which is displayed by the display control unit in the eighth modification of the first embodiment. FIG. 22 is a diagram illustrating an example of a screen before image processing, which is displayed by the display control unit in the ninth modification of the first embodiment. FIG. 23 is a diagram illustrating an example of a screen after image processing, which is displayed by the display control unit of the tenth modification of the first embodiment. FIG. 24 is a diagram illustrating an example of a screen after image processing, which is displayed by the display control unit of the eleventh modification of the first embodiment. FIG. 25 is a diagram illustrating an example of a screen after image processing, which is displayed by the display control unit in the twelfth modification of the first embodiment. FIG. 26 is a diagram illustrating an example of a screen after image processing, which is displayed by the display control unit in the thirteenth modification of the first embodiment. FIG. 27 is a diagram illustrating an example of a screen after image processing, which is displayed by the display control unit of the fourteenth modification of the first embodiment. FIG. 28 is a diagram illustrating an example of a screen after image processing, which is displayed by the display control unit of the fifteenth modification of the first embodiment. FIG. 29 is a diagram illustrating an example of a screen displayed by the display control unit according to the sixteenth modification of the first embodiment. FIG. 30 is a diagram illustrating an example of a screen after image processing, which is displayed by the display control unit of the seventeenth modification of the first embodiment. FIG. 31 is a diagram showing an example of a screen displayed by the display control unit according to the eighteenth modification of the first embodiment. FIG. 32 is a diagram illustrating an example of a screen before image processing, which is displayed by the display control unit of the nineteenth modification of the first embodiment. FIG. 33 is a diagram showing an example of a screen displayed by the display control unit according to the twentieth modification of the first embodiment. FIG. 34 is a diagram showing an example of a screen displayed by the display control unit according to the twenty-first modification of the first embodiment. FIG. 35 is a diagram showing an example of a screen displayed by the display control unit according to the twenty-second modification of the first embodiment. FIG. 36 is a diagram showing an example of a screen displayed by the display control unit according to the twenty-third modification of the first embodiment. FIG. 37 is a schematic diagram showing a system configuration of an information processing system according to a modified example of the second embodiment. FIG. 38 is a schematic diagram illustrating a functional configuration of an information processing terminal according to a modification of the second embodiment. FIG. 39 is a functional block diagram realized by the memory unit 22 and the control unit 23. FIG. 40 is an activity diagram showing the flow of information processing according to the fourth embodiment. FIG. 41 is a schematic diagram showing an example of a spatial correction range of the first correction process. FIG. 42 is a schematic diagram showing an example of a temporal correction range of the first correction process. FIG. 43 is a schematic diagram showing additional information 43 added to the playback screen 4 when the first correction process and the second correction process are executed. FIG. 44 is an activity diagram showing the flow of information processing according to the third modification of the fourth embodiment. FIG. 45 is a functional block diagram realized by the control unit 33. FIG. 46 is an activity diagram showing the flow of information processing according to the fifth embodiment. FIG. 47 is a schematic diagram showing the image data IM displayed as a live view and the image data IM recorded so as to be playable later. FIG. 48 is a schematic diagram showing a state in which additional information 43 encouraging playback of the recommended third program P3 is superimposed on the playback screen 4.

Below, a description will be given of a mode for carrying out the present invention with reference to the drawings. Note that the embodiment shown below is one embodiment of the present invention, and the contents of the present invention should not be interpreted as being limited based on the following description.

The program for realizing the software appearing in this embodiment may be provided as a non-transitory computer-readable recording medium, or may be provided so that it can be downloaded from an external server, or may be provided so that the program is started on an external computer and its functions are realized on a client terminal (so-called cloud computing).

In this embodiment, a "unit" may also include, for example, a combination of hardware resources implemented by a circuit in the broad sense and software information processing that can be specifically realized by these hardware resources. In addition, this embodiment handles various types of information, which may be represented, for example, by physical values of signal values representing voltage and current, high and low signal values as a binary bit collection consisting of 0 or 1, or quantum superposition (so-called quantum bits), and communication and calculations may be performed on a circuit in the broad sense.

In the broad sense, a circuit is a circuit realized by at least appropriately combining a circuit, circuitry, a processor, and memory. In other words, it includes application specific integrated circuits (ASICs), programmable logic devices (e.g., simple programmable logic devices (SPLDs), complex programmable logic devices (CPLDs), and field programmable gate arrays (FPGAs)), etc.

[Embodiment 1]
[System configuration of information processing system]
An example of the information processing system 3000 will be described below. FIG. 1 is a diagram showing an example of the system configuration of the information processing system 3000 according to the first embodiment. As shown in FIG. 1, the information processing system 3000 includes a vehicle 3400, a server 3700, a plurality of information display terminals 3800A, 3800B, and 3800C, and a network 3900. The vehicle 3400 is configured to be able to communicate with the server 3700 via the network 3900. More specifically, the vehicle 3400 transmits information held by the vehicle 3400 to the server 3700 via the network 3900. In addition, the plurality of information display terminals 3800A, 3800B, and 3800C are configured to be able to communicate with the server 3700 via the network 3900. More specifically, the plurality of information display terminals 3800A, 3800B, and 3800C acquire information in the server 3700 via the network 3900. It should be noted that the information display terminals 3800A, 3800B, and 3800C may be any device that functions as a UI (User Interface), and specific examples include a personal computer, a smartphone, and a tablet.

[Overall configuration of vehicle 3400]
FIG. 2 is a diagram for explaining the configuration of the vehicle 3400 of the first embodiment. FIG. 2 shows a schematic diagram of the vehicle 3400 viewed from the side. The vehicle 3400 has an imaging device 3100 as a first imaging device and an imaging device 3200 as a second imaging device. The vehicle 3400 is, for example, an internal combustion engine vehicle having an engine as a driving source, a hybrid vehicle having an engine and a running motor as a driving source, an electric vehicle having a running motor as a driving source, etc. The vehicle 3400 is, for example, a four-wheeled automobile, but it may be any vehicle in which the imaging device 3100 and the imaging device 3200 can be arranged. The vehicle may be, for example, a large transport vehicle with four or more wheels such as an automobile, a bus, or a truck, a two-wheeled vehicle such as a motorcycle or a bicycle, or other vehicles. The vehicle may be, for example, a transportation vehicle such as a train, a monorail, or a linear motor car.

The imaging device 3100 is a front camera arranged on the front side of the vehicle 3400. The imaging device 3100 is attached, for example, to a predetermined position on the front side of the passenger compartment of the vehicle 3400, and captures images of the front of the vehicle 3400 through the windshield. The imaging device 3100 is a drive recorder. More specifically, the imaging device 3100 has a function of capturing images, a function of recording image data showing the captured images, and a function of recording image data obtained from the imaging device 200.

The imaging device 3200 is a rear camera disposed at the rear of the vehicle 3400. For example, the imaging device 3200 is attached to a predetermined position at the rear of the passenger compartment of the vehicle 3400, and captures images of the rear of the vehicle through the rear window. The imaging device 3200 has a function of capturing images and a function of outputting image data showing the captured images to the imaging device 3100.

The image capturing device 3100 and the image capturing device 3200 are connected via a cable 3300. The cable 3300 is a wired communication path connecting the image capturing device 3100 and the image capturing device 3200. The cable 3300 has, for example, a power line that supplies operating power from the image capturing device 3100 to the image capturing device 3200, and a signal line that transmits various signals between the image capturing device 3100 and the image capturing device 3200. The image capturing device 3200 operates by receiving power from the image capturing device 3100 via the cable 3300. Note that the image capturing device 3100 and the image capturing device 3200 may be connected by a wireless communication path of Wi-Fi (registered trademark), Bluetooth (registered trademark), or other standard, instead of a wired communication path. The image capturing device 3100 may also be used without being connected to the image capturing device 200 by communication.

[External configuration of the imaging device 3100]
3 is a diagram showing an example of the external configuration of the imaging device 3100. FIG. 3A is a diagram showing the imaging device 3100 as viewed from the upper right diagonal direction on the front side. FIG. 3B is a diagram showing the imaging device 3100 as viewed from the upper right diagonal direction on the rear side. The imaging device 3100 has a housing 3101. The housing 3101 is a rectangular parallelepiped that is longer in the left-right direction than in the up-down direction and has a relatively small thickness. The housing 3101 has a top surface 31011 that faces upward when attached to the vehicle 3400, a first side surface 31012, a second side surface 31013, a third side surface 31014 located on the opposite side to the second side surface 31013, and a fourth side surface 31015 located on the opposite side to the first side surface 31012.

The upper surface 31011 is provided with a joint rail 3102 and a camera jack 3191. The joint rail 3102 is detachable from a bracket (e.g., a bracket 3600 described later) for mounting the image capture device 3100 to a predetermined mounting position of the vehicle 3400. The mounting position may be, for example, the windshield of the vehicle 3400 (e.g., near the upper end of the windshield), or the rearview mirror or the ceiling of the vehicle 3400. When the image capture device 3100 is mounted on the vehicle 3400, the first side 31012 faces the front side of the vehicle 3400. At this time, the second side 31013 faces the right side when viewed from the rear side of the vehicle 3400. The third side 31014 faces the left side when viewed from the rear side of the vehicle 3400. The fourth side 31015 faces the rear side of the vehicle 3400.

The camera jack 3191 is a terminal to which one end of the cable 3300 is connected. The camera jack 3191 is compatible with the USB Type C standard, for example, and may be a terminal through which the image capture device 3100 communicates with the image capture device 3200 according to the Ethernet standard.

The first side surface 31012 is provided with a photographing lens 3151, a sound output hole 3103, and a microphone hole 3104. The photographing lens 3151 is a lens for collecting light that is included in the photographing unit (the photographing unit 3015 described later) of the photographing device 3100. The sound output hole 3103 is provided above the photographing lens 3151 and is a hole that transmits sound output from the sound output unit (the sound output unit 3014 described later) of the photographing device 3100 from the inside of the housing 3101 to the outside. The microphone hole 3104 is provided below the photographing lens 3151 and is a hole that transmits sound from the outside to the inside of the housing 3101. The sound that transmits into the inside of the housing 3101 is input to a microphone (the microphone 3121 described later) of the photographing device 3100.

An event record button 3122 is provided on the second side 31013. The event record button 3122 is an operating means for instructing the start or end of recording (video recording) of images captured by the image capture unit 3015. When the event record button 3122 is operated by a user while event recording is not being performed, the image capture device 3100 starts event recording. Event recording will be described in detail later. The event record button 3122 is provided on the second side 31013 facing the driver's seat so that it is easy for the driver of a right-hand drive vehicle 3400 to operate.

The third side 31014 is provided with a terminal 3192 and a storage medium insertion port 3181. The terminal 3192 is a terminal for receiving power from an external device. The terminal 3192 is, for example, a DC jack. A connector on one end of a power cord (for example, a cigarette lighter plug cord) is connected to the terminal 3192. The connector on the other end of the power cord is connected to a power supply terminal (for example, a cigarette lighter socket) provided on the vehicle 3400, for example.

The terminal 3192 may be connected to an OBDII adapter that can be connected to the OBDII ("II" is the Roman numeral for "2") connector of the vehicle 3400. The OBDII connector is also called a fault diagnosis connector, and is a terminal that is connected to the vehicle's ECU (Engine Control Unit) and outputs various vehicle information at predetermined intervals (e.g., every 0.5 seconds). By connecting the terminal 3192 to the OBDII connector using the OBDII adapter, the imaging device 3100 can receive a supply of power for operation and acquire vehicle information.

The vehicle information is information related to the state of the vehicle 3400. The vehicle information may be, for example, at least one of the following: the speed of the vehicle 3400 (vehicle speed), engine RPM, engine load factor, throttle degree, ignition timing, percentage of remaining fuel, intake manifold pressure, intake air flow rate (MAF), injection open time, engine coolant temperature (coolant temperature), temperature of air taken into the engine (intake temperature), air temperature outside the vehicle (outside air temperature), amount of remaining fuel in the fuel tank (remaining fuel amount), fuel flow rate, instantaneous fuel consumption, accelerator opening, turn signal information (operation of left and right turn signals (on/off)), brake opening, steering wheel rotation angle, gear position, and door open/closed state information.

The storage medium insertion port 3181 is an insertion port for inserting the storage medium 3500 as an external storage means into the inside of the image capture device 3100. The storage medium 3500 is a storage medium in which images captured by the image capture device 3100 or the image capture device 3200 are recorded, such as an SD card. The SD card may have any shape, such as an SD memory card, a mini SD card, or a micro SD card. The storage medium 3500 may further store a viewer (e.g., a dedicated viewer) program for playing back the stored images on an information display terminal such as a personal computer.

The fourth side surface 31015 is provided with an operation unit 3123, a display surface 3131, and a light-emitting unit 3021. The operation unit 3123 has a first button 31231, a second button 31232, a third button 31233, and a fourth button 31234. The first button 31231, the second button 31232, the third button 31233, and the fourth button 31234 are arranged vertically along one side of the right side of the display surface 3131. The functions that can be assigned to each of these buttons include, for example, the following functions.

The first button 31231 functions as a button for switching images when pressed for a long time, and functions as a button for instructing formatting of the storage medium 3500 when pressed for a short time. The image displayed on the display surface 3131 is, for example, one or both of the image currently being captured by the image capture device 3100 and the image currently being captured by the image capture device 3200. Formatting the storage medium 3500 is to initialize the storage medium 3500, and is understood to be, for example, at least one of erasing data such as images stored in the storage medium 3500, writing setting information indicating the contents of the operation setting to the storage medium 3500 so that the image capture device 3100 can use the storage medium 3500 (for example, so that images can be recorded and read), and putting the storage medium 3500 into a specific file state.

The second button 31232 is a button for displaying a selection screen for selecting an image to be played by the image capture device 3100. The third button 31233 is a button for displaying a menu related to settings of the image capture device 3100 and the image capture device 3200. The fourth button 31234 is a button for instructing the start and stop of image recording. For example, when the fourth button 31234 is pressed briefly during recording using the continuous recording function described below, the recording is paused. When the fourth button 31234 is pressed briefly during the pause, image recording using the continuous recording function is resumed. When the fourth button 31234 is pressed for a long period of time, the frame rate at which images are recorded can be changed.

The display surface 3131 is an area in which an image is displayed by a display unit (display unit 3013 described later) of the imaging device 3100. The display surface 3131 is, for example, a rectangular or square area. A touch sensor 3124 is provided over the display surface 3131 to detect a touch operation by the user.

The light-emitting unit 3021 is located above the first button 31231 and emits light in a specified color.

The image capturing device 3200 may also have a function as a drive recorder, and may have a configuration similar to that of the image capturing device 3100, for example. In addition, instead of or in addition to the image capturing device 3200, one or more image capturing devices capturing images in other directions may be used as image capturing devices connected to the image capturing device 3100 via communication. Examples of other directions include the right rear diagonal direction, the left rear diagonal direction, and the vehicle width direction (sideways) of the vehicle 3400.

[Configuration of bracket 3600]
Fig. 4 is a diagram showing the configuration of the bracket 3600. Fig. 4(A) is a diagram showing the bracket 3600 as viewed from the upper right diagonal direction on the front side. Fig. 4(B) is a diagram showing the bracket 3600 as viewed from the side. Fig. 4(C) is a diagram showing the bracket 3600 as viewed from the lower right diagonal direction on the front side. The bracket 3600 is an example of a mounting member for mounting the image capturing device 3100 to the vehicle 3400.

The bracket 3600 is an attachment member that uses a ball joint mechanism. In the bracket 3600, the flat base portion 3610 has an attachment surface 3611 that is attached to the windshield of the vehicle 3400. The base portion 3610 is inclined at a predetermined angle with respect to the support of the ball stud 3620. An adhesive member such as double-sided tape is attached to the attachment surface 3611, and the bracket is attached to the windshield of the vehicle 3400 via the adhesive member.

The ball stud 3620 is a part that stands up from the surface of the base portion 3610 opposite to the mounting surface 3611. The ball portion 3621 of the ball stud 3620 is attached to the socket portion 3630. The nut 3640 is attached detachably around the socket portion 3630. A screw groove is formed on the outer periphery of the socket portion 3630. The nut 3640 that fits into the screw groove is attached to the outer periphery of the socket portion 3630. When the ball portion 3621 of the ball stud 3620 is attached in the socket portion 3630 and before the nut 3640 is tightened, the socket portion 3630 can rotate in a desired direction along the circumferential surface of the ball portion 3621 to change the attitude and position of the base portion 3650. When the nut 3640 is tightened, the attitude and position of the base portion 3650 are fixed.

The base portion 3650 is formed as a pair with the socket portion 3630, and is a portion for mounting to the imaging device 3100. The base portion 3650 has a pair of guide rails 3651 on the underside. The pair of guide rails 3651 are configured to be slidable along the joint rails 3102 of the imaging device 3100. A claw-shaped tip portion 3652 is provided at the tip portion of the base portion 3650. The tip portion 3652 is hooked onto the joint rails 3102 near the end portion on the front side of the imaging device 3100, whereby the bracket 3600 is attached to the imaging device 3100.

[Electrical configuration of the imaging device 3100]
5 is a block diagram showing the electrical configuration of the imaging device 3100. The control unit 3011 controls each unit of the imaging device 3100. The control unit 3011 is, for example, a computer including a processor 3111 and a memory 3112. The processor 3111 has, for example, a central processing unit (CPU), a micro processing unit (MPU), a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), and a field programmable gate array (FPGA). The memory 3112 is a main storage device having, for example, a RAM (Random Access Memory) and a ROM (Read Only Memory). The processor 3111 temporarily stores a program read from the ROM of the memory 3112 in the RAM. The RAM of the memory 3112 provides a working area for the processor 3111. The processor 3111 performs various controls by temporarily storing data generated during the execution of the program in the RAM while performing arithmetic processing. The control unit 3011 further includes a clock unit 3113 that keeps time. The clock unit 3113 is, for example, a real-time clock. The clock unit 3113 may be mounted on the motherboard of the processor 3111 or may be externally attached to the processor 3111.

The input unit 3012 accepts information input from the user. The input unit 3012 has, for example, the microphone 3121, the event record button 3122, the operation unit 3123, and the touch sensor 3124 described above. The microphone 3121 converts sound incident via the microphone hole 3104 or the like into an electrical signal. The microphone 3121 is, for example, a condenser microphone. The touch sensor 3124 detects the position touched by the user on the display surface 3131. The touch sensor 3124 is, for example, of a capacitive type.

The display unit 3013 displays an image on the display surface 3131. The display unit 3013 is, for example, a liquid crystal display (LCD).

The audio output unit 3014 outputs audio. Examples of this audio include notification sounds, background music, and audio messages. The audio output unit 3014 includes, for example, an audio processing circuit and a speaker.

The image capturing unit 3015 captures images and generates image data obtained by capturing images. The image capturing unit 3015 has, for example, an image capturing lens 3151 and an image capturing element that captures the light collected by the image capturing lens 3151. The image capturing element is, for example, a CMOS (Complementary MOS) or a CCD (Charge Coupled Device). The image capturing unit 3015 generates a color (multi-color) image consisting of, for example, red (R), green (G), and blue (B) color components.

The communication unit 3016 communicates with an external device. The communication unit 3016 has a communication circuit for wirelessly communicating with an external device, for example, by Wi-Fi (registered trademark), Bluetooth (registered trademark), or other wireless LAN (Local Area Network) communication or short-range wireless communication. The communication unit 3016 may have a communication circuit for performing communication in compliance with standards for mobile communication systems such as LTE (Long Term Evolution), 4G, and 5G.

The sensor unit 3017 has various sensors. The sensor unit 3017 has, for example, at least one of an acceleration sensor, a gyro sensor, an air pressure sensor, and an illuminance sensor. The acceleration sensor is, for example, a three-axis acceleration sensor that measures the acceleration of the vehicle in the forward/backward, left/right, and up/down directions. The gyro sensor is a sensor that measures the inclination of the image capture device 3100. The acceleration sensor and the gyro sensor may be used to estimate the position of the vehicle 3400 by autonomous navigation, for example, when signals from GNSS satellites cannot be received. The air pressure sensor measures air pressure. The air pressure sensor is used, for example, to detect differences in elevation and determine whether the road is a highway or an ordinary road. The illuminance sensor is a sensor that detects illuminance indicating the brightness inside the vehicle cabin, which is the periphery of the image capture device 3100. The illuminance sensor is used, for example, to adjust the brightness of the display of the display unit 3013.

The reader/writer 3018 functions as a medium holding unit that holds the storage medium 3500 inserted into the image capture device 3100 through the storage medium insertion port 3181. The reader/writer 3018 writes data to the storage medium 3500 and reads data from the storage medium 3500. The reader/writer 3018 may hold only one storage medium 3500, but may also be configured to be capable of holding two or more storage media 3500 simultaneously.

The terminal unit 3019 has a terminal for electrically connecting to an external device. The terminal unit 3019 has the camera jack 3191 and the terminal 3192 described above. An external battery may be used as a device connected to the terminal unit 3019 so that the image capturing device 3100 and the image capturing device 3200 can operate even without power supply from the vehicle 3400 side. The device connected to the terminal unit 3019 may be, for example, a device having a function of supporting safe driving of the user. Such devices include, for example, a device having a function of capturing an image of the driver (e.g., a face) and detecting and notifying the driver's state, such as distracted driving and drowsy driving, and a device having a function of detecting and notifying obstacles around the vehicle 3400 (for example, a device used for vehicle detection for forward vehicle collision warning systems (FCWS)). The device connected to the terminal portion 3019 may also be an in-vehicle device such as a radar detector, a laser detector, a car navigation device, or a display device.

The position information acquisition unit 3020 acquires position information indicating the position (more specifically, the current position) of the image capture device 3100. The position of the image capture device 3100 can be regarded as the position of the vehicle 3400 in which the image capture device 3100 is placed, the position of the image capture device 3200 placed in the vehicle 3400, and the positions of the driver and other people riding in the vehicle 3400. The position information acquisition unit 3020 acquires position information (latitude information and longitude information) of the image capture device 3100 based on a signal from a GPS (Global Positioning System), which is one of the GNSS (Global Navigation Satellite Systems). The location information acquisition unit 3020 may also use Michibiki as a Quasi-Zenith Satellite System (QZSS).

The light emitting unit 3021 emits light in a predetermined color. The light emitting unit 3021 has, for example, a light emitting diode.

The power supply control unit 3022 controls the supply of power to each part of the image capture device 3100 and the image capture device 3200. The power supply control unit 3022 has, for example, a power switch and a power supply control circuit. The power supply control unit 3022 supplies power supplied from the vehicle 3400 via the terminal unit 3019 to each part of the image capture device 3100 and the image capture device 3200. The power supply control unit 3022 may further have a secondary battery, a button battery, or an electric double layer capacitor (also called a supercapacitor) as a power storage means.

The imaging device 3100 may further include an auxiliary storage device, such as a flash memory (e.g., eMMC, SSD), as an internal storage means. As the auxiliary storage device, various types of storage media, such as optical storage media, magnetic storage media, and semiconductor storage media, may be used.

[Image recording function of the imaging device 3100]
The image capturing device 3100 has one or more of the following functions as an image recording function. The image recording function is a function for recording an image captured by the image capturing device 3100 as image data in a predetermined file format. The image data may be image data in a moving picture format, for example, an MPEG (Moving Picture Experts Group) format (e.g., MPEG2, MPEG4), AVI, MOV, WMV, etc. The image data includes information on the shooting date and time, information on the shooting location, etc., as Exif (Exchangeable image file format) information.

[Continuous recording function]
The continuous recording function (also called the continuous video recording function) is a function that continuously (i.e., constantly) records images captured by one or both of the image capturing devices 3100 and 3200 while the image capturing device 3100 is in operation. When the control unit 11 is executing the continuous recording function, it stores video data captured from the start to the stop of the drive source of the vehicle 3400. The control unit 3011 detects the start of the drive source, for example, by the power supply (e.g., accessory power supply) of the vehicle 3400 being turned on. The control unit 3011 detects the stop of the drive source, for example, by the power supply of the vehicle 3400 being turned off.

[Event recording function]
The event recording function is a function for recording images captured by one or both of the image capturing devices 3100 and 3200 in response to the occurrence of a specific event. The event is an event for which an image captured by the image capturing device 3100 or the image capturing device 3200 should be recorded, for example, when the user performs an operation such as a sudden steering or sudden braking while the vehicle 3400 is traveling, or when the vehicle 3400 collides with another object. The control unit 3011 determines that an event has occurred, for example, based on the acceleration measured by the acceleration sensor of the sensor unit 3017. More specifically, the control unit 3011 determines that an event has occurred when the measured acceleration is equal to or greater than a predetermined threshold value or shows a predetermined change over time. The conditions for determining the occurrence of an event are not limited to the above. The control unit 3011 may determine that an event has occurred based on the vehicle information, for example, when the state of the vehicle 3400, such as the vehicle speed or steering state, satisfies a predetermined condition. The control unit 3011 may analyze an image captured by the image capturing unit 3015 or the image capturing device 3200, and determine that an event has occurred when dangerous driving (e.g., tailgating, approaching, or abnormal approach) of the vehicle 3400 or another vehicle is detected. The control unit 3011 also determines that an event has occurred when the event record button 3122 is operated.

When the control unit 3011 determines that an event has occurred, it records images taken during a predetermined period (hereinafter referred to as the "event recording period") before and after the occurrence of the event in the storage medium 3500. The control unit 3011 may temporarily record images taken by the photographing unit 3015 and the photographing device 3200 in the memory 3112 (e.g., RAM), and when it determines that an event has occurred, record images during the event recording period read from the memory 3112 in the storage medium 3500. The control unit 3011 may, for example, record images for a total of 40 seconds, including 20 seconds before the occurrence of the event and 20 seconds after the occurrence of the event, as one file. The event recording period is an example, and may vary depending on the type of event, or may be changeable by the user. The control unit 3011 may record images consisting of multiple files for each occurrence of an event in the storage medium 3500. The control unit 3011 may further record values measured by the sensor unit 3017 during the event recording period (e.g., acceleration in each of the three axial directions) and position information acquired by the position information acquisition unit 3020 in association with the image in the storage medium 3500.

[Parking monitoring function]
The parking monitoring function is a function for recording images captured by one or both of the image capturing devices 3100 and 3200 while the vehicle 3400 is parked. The parking monitoring function is a function for monitoring the inside of the vehicle 3400 while parked or the outside around the vehicle 3400. When the power supply of the vehicle 3400 is off, the control unit 3011 receives power from an external battery and records images in the storage medium 3500. Regarding whether or not the vehicle 3400 is parked, the control unit 3011 determines whether or not the vehicle 3400 is parked based on one or more of the following: the power supply of the vehicle 3400 is off; the power source of the vehicle 3400 is off; the power supply from the external battery has started; the vehicle speed is 0 km/h or a predetermined speed or less; and the location information acquired by the location information acquisition unit 3020 is predetermined location information (for example, location information of a home, workplace, or parking lot).

The parking monitoring function may have a time lapse mode and a motion detection mode. More specifically, when the time lapse mode is selected by the user, the control unit 3011 records images at a frame rate lower than other image recording functions such as the continuous recording function and the event recording function. For example, the frame rate of the other image recording functions is 20 to 30 frames/second, while the frame rate of the time lapse mode is 1 frame/second. The motion detection mode is a mode in which images are recorded in response to the detection of a moving object. More specifically, when the control unit 3011 detects a moving object from a change in the images captured by the image capturing device 3100 and the image capturing device 3200 when the motion detection mode is selected by the user, the control unit 3011 records images captured during a predetermined period before and after the detection in the storage medium 3500. The frame rate may be the same as that of the continuous recording function and the event recording function.

The imaging devices 3100 and 3200 may be imaging devices that capture spherical images such as a full sphere or a hemisphere. The imaging device 3100 may be an imaging device that does not have a display unit 3013. The housing of the imaging device 3100 does not have to be rectangular and may be cylindrical, for example.

[Hardware configuration of server 3700]
The hardware configuration of the server 3700 will be described below with reference to FIG.

FIG. 6 is a diagram showing an example of the hardware configuration of the server 3700 according to the first embodiment. In the first embodiment, the server 3700 serves as an image processing device. The server 3700 includes, as its hardware configuration, a control unit 3710, a storage unit 3720, and a communication unit 3730. The control unit 3710, the storage unit 3720, and the communication unit 3730 are electrically connected to each other via a communication bus.

The control unit 3710 is composed of a microcomputer including a CPU and a semiconductor memory, and controls the overall operation of the server 3700. In particular, the control unit 3710 realizes various functions related to the server 3700 by reading and executing a predetermined program stored in the storage unit 3720. That is, the control unit 3710 executes processing based on a predetermined program stored in the storage unit 3720, thereby realizing the functional configuration shown in FIG. 8 described later and the processing of the activity diagram shown in FIG. 10 described later. Note that the control unit 3710 is not limited to being single. The server 3700 may be configured to include multiple control units 3710.

The storage unit 3720 is configured with any one of a hard disk drive (HDD), a random access memory (RAM), a read only memory (ROM), and a solid state drive (SSD), or any combination of these. The storage unit 3720 stores image information sent from the vehicle 3400, information other than images sent from the vehicle 3400, various data related to various programs including a part of the image processing program according to the first embodiment, data used when the control unit 3710 executes processing based on a program, and the like.

The communication unit 3730 is a so-called network interface that is communicatively connected to the vehicle 3400 and each of the information display terminals 3800A, 3800B, and 3800C via a network 3900 such as the Internet and a LAN. In the first embodiment, the communication unit 3730 of the server 3700 that performs image processing is an image processing device that can communicate with the communication unit 3830 of the information display terminal 3800. The main information that the server 3700 receives via the communication unit 3730 includes image information captured by the imaging device 3100 of the vehicle 3400, image information captured by the imaging device 3200, information on the status of the vehicle 3400 (the speed of the vehicle 3400 (vehicle speed), acceleration, engine RPM, engine load factor, throttle opening, ignition timing, percentage of remaining fuel, intake manifold pressure, intake air volume (MAF), injection open time, engine coolant temperature (coolant temperature), temperature of air taken into the engine (intake temperature), air temperature outside the vehicle (outside air temperature), amount of remaining fuel in the fuel tank (remaining fuel amount), fuel flow rate, instantaneous fuel consumption, accelerator opening, turn signal information (operation of left and right turn signals (on/off)), brake opening, steering wheel rotation steering angle, gear position, and information on the door open/closed status, location information, etc.), date and time information, air pressure, illuminance, image processing instructions, image download request instructions, request instructions for information other than images, etc. The main information transmitted by the server 3700 via the communication unit 3730 includes image information before image processing is performed, image information after image processing is performed, information on the image processing method, information on the state of the vehicle 3400 (vehicle speed, acceleration, engine RPM, engine load factor, throttle opening, ignition timing, percentage of remaining fuel, intake manifold pressure, intake air volume (MAF), injection open time, engine coolant temperature (coolant temperature), temperature of air taken into the engine (intake temperature), air temperature outside the vehicle (outside air temperature), amount of remaining fuel in the fuel tank (remaining fuel volume), fuel flow rate, instantaneous fuel consumption, accelerator opening, turn signal information (operation of left and right turn signals (on/off)), brake opening, steering wheel rotation angle, gear position, door open/closed status information, position information, etc.), date and time information, air pressure, illuminance, etc.

[Hardware configuration of information display terminal 3800]
Next, the hardware configuration of the information display terminal 3800 will be described with reference to FIG.

Figure 7 is a diagram showing an example of the hardware configuration of the information display terminal 3800 according to the first embodiment. The information display terminal 3800 includes, as its hardware configuration, a control unit 3810, a storage unit 3820, a communication unit 3830, an input unit 3840, and an output unit 3850. The control unit 3810, the storage unit 3820, the communication unit 3830, the input unit 3840, and the output unit 3850 are electrically connected to each other via a communication bus.

The control unit 3810 is composed of a microcomputer including a CPU and a semiconductor memory, and controls the overall operation of the information display terminal 3800. In particular, the control unit 3810 realizes various functions related to the information display terminal 3800 by reading and executing a predetermined program stored in the storage unit 3820. That is, the control unit 3810 executes processing based on a predetermined program stored in the storage unit 3820, thereby realizing the functional configuration shown in FIG. 9 described later and the processing of the activity diagram shown in FIG. 11 described later. Note that the control unit 3810 is not limited to being single. The information display terminal 3800 may be configured to include multiple control units 3810.

The storage unit 3820 is configured with any one of a HDD, SSD, RAM, and ROM, or any combination of these. The storage unit 3820 stores a program for exchanging data related to image processing with the server 3700, data used when the control unit 3810 executes processing based on the program, and the like.

The communication unit 3830 is a so-called network interface that is communicatively connected to the network 3900 to transmit and receive various data between the information display terminal 3800 and the server 3700. That is, the main information that the information display terminal 3800 receives via the communication unit 3830 includes image information before image processing is performed, image information after image processing is performed, information related to the image processing method, information related to the state of the vehicle 3400 (such as the speed (vehicle speed) of the vehicle 3400, acceleration, engine RPM, engine load factor, throttle opening, ignition timing, percentage of remaining fuel, intake manifold pressure, intake air volume (MAF), injection open time, engine coolant temperature (coolant temperature), temperature of air taken into the engine (intake temperature), air temperature outside the vehicle (outside air temperature), amount of remaining fuel in the fuel tank (remaining fuel amount), fuel flow rate, instantaneous fuel consumption, accelerator opening, turn signal information (operation of left and right turn signals (on/off)), brake opening, steering wheel rotation steering angle, gear position, and information on the door open/closed state), date and time information, location information, air pressure, and illuminance. The main information transmitted by the information display terminal 3800 via the communication unit 3830 includes instructions for image processing, instructions requesting image downloads, instructions requesting information other than images, etc.

The input unit 3840 accepts operations on the information display terminal 3800. Examples of the input unit 3840 include a mouse, a keyboard, a touch panel, a microphone, and buttons.

The output unit 3850 outputs the results of operations performed on the input unit 3840. Examples of the output unit 3850 include a display, a speaker, a light, and a button.

[Functional configuration of server 3700]
The functional configuration of the control unit 3710 of the server 3700 will be described below with reference to FIG.

FIG. 8 is a diagram showing an example of the functional configuration of the server 3700 according to the first embodiment. As shown in FIG. 8, the server 3700 includes, as its functional configuration, an image processing unit 3711, a communication control unit 3712, and a memory control unit 3713.

The image processing unit 3711 performs image processing on the image in the storage unit 3720 or the image transmitted from the information display terminal 3800 based on a request from the information display terminal 3800. This allows the server 3700, which can communicate with the information display terminal 3800, to perform image processing. In the first embodiment, the image processing unit 3711 performs various image processing based on a request from the information display terminal 3800. However, the server 3700 may have an image processing unit for each image processing. In addition, instead of the software of the server 3700 performing image processing, dedicated hardware (e.g., GPU, etc.) of the server 3700 may perform image processing. When dedicated hardware performs image processing, one piece of hardware of the server 3700 may be configured to perform multiple image processing, or multiple pieces of hardware of the server 3700 corresponding to each of the multiple image processing may be configured to perform image processing.

The communication control unit 3712 controls the exchange of information with the vehicle 3400 and the information display terminal 3800 via the network 3900.

The memory control unit 3713 stores information in the memory unit 3720 and retrieves stored information.

[Functional configuration of information display terminal 3800]
The functional configuration of the control unit 3810 of the information display terminal 3800 will be described below with reference to FIG.

FIG. 9 is a diagram showing an example of the functional configuration of the information display terminal 3800 according to the first embodiment. As shown in FIG. 9, the information display terminal 3800 includes, as its functional configuration, a display control unit 3811, a communication control unit 3812, and a storage control unit 3813.

The display control unit 3811 controls the information to be output to the output unit 3850 of the information display terminal 3800. More specifically, the display control unit 3811 controls the display of a screen including information related to the image.

The communication control unit 3812 controls the exchange of information with the server 3700 via the network 3900. More specifically, the communication control unit 3812 acquires images captured by the imaging device 3100, images captured by the imaging device 3200, etc. from the server 3700.

The memory control unit 3813 stores information in the memory unit 3820 and retrieves stored information.

[Outline of information processing]
The flow of information processing in the information processing system 3000 according to the first embodiment will be described with reference to FIGS.

[Information Processing of Server 3700]
First, a description will be given of the flow of information processing by the server 3700. Fig. 10 is an activity diagram showing the flow of information processing by the server 3700 according to embodiment 1. Below, an overview of the flow of information processing will be given with reference to this activity diagram.

At A4001, the communication control unit 3712 waits to receive information from the vehicle 3400. When the communication control unit 3712 determines that information has been received from the vehicle 3400, it advances the process to A4002. In the following description, the communication control unit 3712 receives, as information from the vehicle 3400, identification information for identifying the vehicle 3400, image information captured by the imaging device 3100, image information captured by the imaging device 3200, vehicle information of the vehicle 3400, and other information (for example, information on the time when the image was captured, etc.).

In A4002, the memory control unit 3713 stores the received image information, vehicle information, and other information in the memory unit 3720. More specifically, the memory control unit 3713 stores the image information, vehicle information, and other information in a specified area of the memory unit 3720 in association with the identification information that identifies the received vehicle 3400.

In A4003, the image processing unit 3711 analyzes the image information, vehicle information, and other information. In the first embodiment, the image processing unit 3711 analyzes the acquired image information to determine, for example, that the vehicle 3400 is traveling in fog from 07:22:10 on April 29, 2021 to 07:24:18 on April 29, 2021. The image processing unit 3711 associates the analysis result for each time information with identification information that identifies the vehicle 3400, and stores it in a predetermined area of the storage unit 3720 together with the image information, vehicle information, and other information.
In the present embodiment, the server 3700 is assumed to analyze the image information. However, the information display terminal 3800 that receives the image information may analyze the image information.

At A4004, the communication control unit 3712 transmits image information, vehicle information, analysis results, and other information to the information display terminal 3800 based on instructions from the information display terminal 3800. More specifically, the communication control unit 3712 transmits image information, vehicle information, and other information from the specified date and time onward regarding the vehicle identified by the identification information instructed by the information display terminal 3800 to the information display terminal 3800.

At A4005, the communication control unit 3712 waits to receive an image processing request from the information display terminal 3800. When the communication control unit 3712 determines that an image processing request has been received from the information display terminal 3800, it advances the process to A4006. For example, the communication control unit 3712 receives a request to perform fog correction for the entire range of the image as an image processing request from the information display terminal 3800.

In A4006, the image processing unit 3711 executes image processing on the image based on the image processing request. For example, the image processing unit 3711 performs fog correction on the entire image based on the image processing request. Fog correction is image processing that adjusts the brightness of the image, etc., to improve poor visibility caused by fog.

At A4007, the communication control unit 3712 transmits the results of image processing in the image processing unit 3711 to the information display terminal 3800.

[Information processing of information display terminal 3800]
Next, a description will be given of the flow of information processing by the information display terminal 3800. Fig. 11 is an activity diagram showing the flow of information processing by the information display terminal 3800 according to embodiment 1. Below, the flow of information processing will be outlined with reference to this activity diagram.

In A4011, the display control unit 3811 displays a launch button for the external video player on the viewer. When the launch button on this viewer is selected, the communication control unit 3812 launches the external video player in the server 3700. At this time, the communication control unit 3812 transmits to the server 3700, as "command line arguments", the file name of the video, video effect settings, parameters for disabling at least a portion of the operation unit of the external video player, and the like. At this time, the communication control unit 3812 may also transmit the playback start time, playback end time, whether or not to loop, and the like. Thereafter, the communication control unit 3812 controls the external video player by sending a control command. At this time, the communication control unit 3812 may transmit an instruction to disable at least a portion of the operation unit of the external video player.

At A4012, the communication control unit 3812 waits to receive information from the server 3700. If the communication control unit 3812 determines that information has been received from the server 3700, it advances the process to A4013. The communication control unit 3812 will be described as receiving image information, vehicle information, analysis information analyzed by the image processing unit 3711, and other information as information from the server 3700. More specifically, in the first embodiment, the communication control unit 3812 will be described as receiving image information, acceleration information, vehicle speed information, position information, analysis information, date and time information, etc. from the server 3700.

In A4013, the display control unit 3811 displays the received image information, vehicle information, analysis information, and other information. More specifically, in embodiment 1, the display control unit 3811 displays image information, acceleration information, vehicle speed information, position information, analysis information, date and time information, etc.

In A4014, the display control unit 3811 determines whether or not a range specification for image processing has been accepted. If the display control unit 3811 determines that a range specification for image processing has been accepted, the process proceeds to A4015. If the display control unit 3811 determines that a range specification for image processing has not been accepted, the process proceeds to A4016.
In the following description, it is assumed that when no range designation for image processing is received, the entire range of the image is subject to image processing. That is, although omitted in Fig. 11 for simplification, when no range designation for image processing is received, "the entire image" is held as the range information. In the first embodiment, it is assumed that "the entire image" is held as the range information.

In A4015, the communication control unit 3812 holds the range information of the image processing selected by the operator of the information display terminal 3800 via a screen or the like.

In A4016, the display control unit 3811 waits for receipt of an instruction to execute image processing selected from among multiple image processing from the user via the screen. In the first embodiment, the display control unit 3811 will be described as having received an instruction to execute fog correction from among multiple image processing from the user via the screen.

When a user instructs the server 3700 to perform image processing, in A4017 the communication control unit 3812 transmits an instruction to the server 3700 to perform image processing. In the first embodiment, the communication control unit 3812 transmits an instruction to the server 3700 to perform fog correction on the image.

In A4018, the communication control unit 3812 waits to receive the image resulting from processing by the server 3700. When the communication control unit 3812 receives the image, it advances the process to A4019. In the first embodiment, the communication control unit 3812 receives the image after fog correction processed by the server 3700.

In A4019, the display control unit 3811 displays the information acquired from the server 3700 on the screen. In the first embodiment, the display control unit 3811 displays the image after fog correction acquired from the server 3700 on the screen.

In this embodiment, the external video player in the server 3700 is controlled from the PC Browser (PC application) which is the viewer on the information display terminal 3800. Examples of the external video player include free software such as VLC, cloud image processing services, and image sharpening services. When installing the viewer of this embodiment, the display control unit 3811 asks whether to also install the external video player. When there is an instruction to install the external video player, the storage control unit 3813 installs the external video player. When installing the external video player or immediately after starting it, the display control unit 3811 displays a guide to the button for starting the external video player and a note when using the external video player. The display control unit 3811 may display, as a note when using the external player, "When you operate the button of the external video player, it may be different from the display state of the viewer" and "If the display state of the viewer is different, it is recommended to close the external video player and start the external video player again from the viewer."

When a cloud-based system is used, the storage control unit 3813 may be configured to have a function of storing an ID or the like in the storage unit 3820 for automatic login to the cloud image sharpening service. The communication control unit 3812 may also be configured to have a function of automatically sending information, a function of uploading, and a function of determining whether conversion is complete and either automatically downloading when conversion is complete or notifying that downloading is possible. Furthermore, the display control unit 3811 may have a function of playing the downloaded image in synchronization with the image before conversion.

Furthermore, the display control unit 3811 controls to display that when playing with a non-recommended viewer, the image can be played beautifully with a dedicated viewer. When a dark image is recorded, it is corrected and displayed beautifully when played with a dedicated viewer, but when played with a standard player for an OS such as Windows (registered trademark), the played image may be dark. Therefore, for example, when playing with a non-recommended viewer, the display control unit 3811 controls to provide black band areas above and below the image and display "This image will be played beautifully with a dedicated viewer" in this black band area. At this time, the display control unit 3811 may display a code (e.g., QR code (registered trademark)) that references a URL where the viewer can be installed together with the character string. As another form, the display control unit 3811 may display the character string "This image will be played beautifully with a dedicated viewer" at the beginning of the image when playing with a non-recommended viewer, or may display a code (e.g., QR code (registered trademark)) that references a URL where the viewer can be installed.

[Details of information processing]
Next, the above-mentioned information processing will be described in detail.

Figure 12 is a diagram showing an example of a screen before image processing, displayed by the display control unit 3811 of embodiment 1. The display control unit 3811 controls the display of information in the server 3700 on the viewer 4100. The viewer 4100 is configured to be able to call various functions and display information. More specifically, the viewer 4100 includes a file button 4101, a display button 4102, a play button 4103, a tool button 4104, and an information button 4105, and these buttons enable calling and referencing of various functions. The viewer 4100 also includes a video display area 4110, an acceleration graph display area 4120, a map image display area 4130, a date and time display area 4150, a vehicle status display area 4160, and a speed graph display area 4170, and these areas allow reference to the image being played and information related to the image. Furthermore, the viewer 4100 includes a video display area 4110, a fog correction button 4140, an HDR button 4141, a sharpening button 4142, an other processing button 4143, a partial processing button 4144, a stream display button 4145, an operation button 4146, and a playlist display area 4180, and in response to the operation of these buttons, the display control unit 3811 requests the server 3700 to process images and play images. In other words, the viewer 4100 is configured to be able to accept multiple operation instructions from the user. At this time, each of the multiple operation instructions instructs the display of a different processing result related to the image.

The file button 4101 is configured to be able to execute various file manipulation processes. More specifically, when the file button 4101 is selected, the display control unit 3811 displays file-related processes such as "view file," "save over," and "save as" in a pull-down format on the screen, and when one of the file-related processes is selected, a request is made to the control unit 3810, etc. to execute the selected process.

The display button 4102 allows the execution of various processes related to the display of the image display area 4110. More specifically, when the display button 4102 is selected, the display control unit 3811 displays display-related processes such as "Display side by side," "Display in a row," and "Minimize" in a pull-down format, and when one of the display-related processes is selected, the selected process is executed.

The play button 4103 allows the execution of various processes related to the playback of video. More specifically, when the play button 4103 is selected, the display control unit 3811 displays in a pull-down format processes related to playback, such as "Play next playlist," "Play previous playlist," "Play from next image processing," and "Play from previous image processing," and when one of the display-related processes is selected, the selected process is executed.

The tool button 4104 can call up various functions. More specifically, when the tool button 4104 is selected, the display control unit 3811 displays a pull-down menu of options such as "Macro" and "User Settings," and when one of the processes is selected, a request is made to the control unit 3810, etc. to execute the selected process.

The information button 4105 allows the user to refer to various information. More specifically, when the tool button 4104 is selected, the display control unit 3811 displays buttons related to information such as "Version Information" in a pull-down format, and by selecting one of the buttons, the selected information is displayed so that the user can refer to it.

The video display area 4110 is an area where images captured by the imaging device 3100 and the imaging device 3200 and images processed by the image processing unit 3711 are displayed. The display format of the video display area 4110 can be changed as desired by a user instruction. More specifically, the video display area 4110 may display multiple images after image processing using different methods side by side based on a user instruction, or may display an image after image processing and an image before image processing side by side. Furthermore, the video display area 4110 may display any information, such as an acceleration graph display area 4120.

The acceleration graph display area 4120 is an area where the acceleration of the vehicle that captured the image is displayed in three dimensions. In the acceleration graph display area 4120, the X axis is formed in the depth direction, the Y axis is formed in the horizontal direction, and the Z axis is formed in the vertical direction. Here, the X axis is an axis showing the acceleration in the front and rear of the vehicle, the Y axis is an axis showing the acceleration in the lateral direction of the vehicle, and the Z axis is an axis showing the acceleration in the vertical direction of the vehicle. Acceleration in the front of the vehicle (positive acceleration) is graphed upward on the X axis, and acceleration in the rear of the vehicle (negative acceleration) is graphed downward on the X axis. Acceleration in the right direction toward the vehicle is graphed upward on the Y axis, and acceleration in the left direction toward the vehicle is graphed downward on the Y axis. Acceleration in the upward direction of the vehicle is graphed leftward on the Z axis, and acceleration in the downward direction of the vehicle is graphed rightward on the Z axis.

The map image display area 4130 is an area where an image of a map in the vicinity of the shooting location of the image displayed in the image display area 4110 is displayed. Data representing the vehicle's position is transmitted from the information display terminal 3800 to a map server (not shown) present on the network 3900, and data representing a map in the vicinity of the vehicle's position is transmitted from this map server to the information display terminal 3800, whereby an image of the map is displayed in the map image display area 4130. At this time, a current location icon 4131 is displayed on the map image display area 4130.

The viewer 4100 includes a fog correction button 4140. When the fog correction button 4140 is selected, the image processing unit 3711 performs fog correction on the image displayed in the video display area 4110. More specifically, when the fog correction button 4140 is selected, the communication control unit 3812 of the information display terminal 3800 instructs the server 3700 to perform fog removal processing on the image. The server 3700, which has received the instruction, performs fog removal processing on the image using the image processing unit 3711. The communication control unit 3712 of the server 3700 transmits the image that has been subjected to fog removal processing to the information display terminal 3800. The display control unit 3811 controls the video display area 4110 to display the image that has been subjected to fog removal processing. In other words, video correction can be performed by selecting a button in the PC Browser (PC application). In addition, at this time, if the fog correction button 4140 is operated again while fog correction is being performed, the image processing unit 3711 can perform control to cancel fog correction.

The display control unit 3811 can also control the display to adjust the degree of mist compensation according to the number of times the mist compensation button 4140 is selected or the selection time. More specifically, the display control unit 3811 controls the display to increase the degree of mist compensation according to the number of times the mist compensation button 4140 is pressed, or controls the display to increase the degree of mist compensation according to the time the mist compensation button 4140 is pressed. In other words, the display control unit 3811 receives an operation instruction for the mist compensation button 4140 selected by the user, receives an instruction to display the processing results related to the image corresponding to the operation instruction, and controls to display the image display area 4110 including the processing results related to the image. The mist compensation button 4140 is an example of an object. The operation of selecting the mist compensation button 4140 is an example of an object operation.

The viewer 4100 includes an HDR (High Dynamic Range) button 4141. When the HDR button 4141 is selected, the image processing unit 3711 performs HDR processing on the image displayed in the video display area 4110. Here, HDR processing is a process of expressing an image without blown-out highlights or crushed shadows by synthesizing multiple photos with different brightnesses. More specifically, when the HDR button 4141 is selected, the communication control unit 3812 of the information display terminal 3800 instructs the server 3700 to perform HDR processing on the image. The server 3700 that has received the instruction uses the image processing unit 3711 to synthesize multiple photos with different brightnesses and perform HDR processing. The communication control unit 3712 of the server 3700 transmits the HDR-processed image to the information display terminal 3800. The display control unit 3811 controls the display to display a video display area 4110 that includes an image that has been subjected to HDR processing.

The display control unit 3811 also receives an instruction to display the processing results for the image corresponding to the button selected by the user, and controls to display a screen including the processing results for the image. More specifically, the image processing unit 3711 receives an instruction to perform fog correction by selecting the fog correction button 4140, receives an instruction to perform HDR processing by selecting the HDR button 4141, and controls to display the processing results in the image display area 4110. Here, the buttons related to image processing, such as the fog correction button 4140 and the HDR button 4141, are different for different processing results related to the image. The fog correction button 4140 and the HDR button 4141 are examples of objects. This allows any image processing corresponding to the button to be executed by the user's selection. Also, when multiple buttons are displayed, the display control unit 3811 can control to not display them simultaneously. More specifically, the display control unit 3811 can control to not display the fog correction button 4140 and the HDR button 4141 simultaneously.

Furthermore, at this time, when the fog correction button 4140 and the HDR button 4141 are selected via the screen, the display control unit 3811 may receive an instruction to display the first processing result of the image corresponding to the fog correction button 4140 and an instruction to display the second processing result of the image corresponding to the HDR button 4141, and control to display a screen including the first processing result of the image and the second processing result of the image. More specifically, when the fog correction button 4140 and the HDR button 4141 are selected via the viewer 4100, the display control unit 3811 receives an instruction to display the processing result of the fog correction of the image corresponding to the fog correction button 4140 and an instruction to display the processing result of the HDR processing of the image corresponding to the HDR button 4141, and controls to display a screen including the processing result of the fog correction of the image and the processing result of the HDR processing of the image. This allows multiple arbitrary image processes to be performed simultaneously. The fog correction button 4140 is an example of a first object. The HDR button 4141 is an example of a second object.

The viewer 4100 includes a sharpening button 4142. When the sharpening button 4142 is selected, the image processing unit 3711 performs sharpening processing on the image displayed in the image display area 4110. More specifically, when the sharpening button 4142 is selected, the communication control unit 3812 of the information display terminal 3800 instructs the server 3700 to perform sharpening processing. The server 3700, which has received the instruction, sharpens the image using the image processing unit 3711. The communication control unit 3712 of the server 3700 transmits the image that has been sharpened to the information display terminal 3800. The display control unit 3811 controls the display of the image display area 4110 including the image that has been sharpened.

The viewer 4100 includes an other processing button 4143. When the other processing button 4143 is selected, the display control unit 3811 controls the viewer 4100 to display various image processing options. Here, examples of the image processing options include an image processing adjustment area 6290, which will be described later. When this image processing option is selected, the communication control unit 3812 of the information display terminal 3800 instructs the server 3700 to perform image processing. The server 3700 that receives the instruction processes the image using the image processing unit 3711. The communication control unit 3712 of the server 3700 transmits the image that has been subjected to image processing to the information display terminal 3800. The display control unit 3811 controls the display of the video display area 4110 including the image that has been subjected to image processing.

The viewer 4100 includes a partial processing button 4144. When the partial processing button 4144 is selected, the display control unit 3811 controls the display so that a portion of the video display area 4110 can be selected. When this image processing option is selected, the communication control unit 3812 of the information display terminal 3800 instructs the server 3700 to perform image processing on a portion of the image. The server 3700 that receives the instruction performs image processing on a portion of the image by the image processing unit 3711. The communication control unit 3712 of the server 3700 transmits the image that has been subjected to image processing to the information display terminal 3800. The display control unit 3811 controls to display the video display area 4110 including the image that has been subjected to image processing on a portion of the image. Here, the display control unit 3811 may control to display the image before image processing and the image after image processing on the video display area 4110 including the image that has been subjected to image processing on a portion of the image. Note that the method for performing partial processing may be another mode. For example, the display control unit 3811 accepts the specification of a correction target range for the video display area 4110. The display control unit 3811 further provides a range registration checkbox. The display control unit 3811 may set the specified range as the correction target range while the correction button is pressed with the range registration checkbox checked. At this time, the display control unit 3811 may provide a button to temporarily remove the correction while the range remains in a corrected state.

The viewer 4100 includes a stream display button 4145. When the stream display button 4145 is selected, the display control unit 3811 performs control to display multiple images side-by-side in the video display area 4110.

Each button displayed on the viewer 4100 may be in other forms. For example, each button displayed on the viewer 4100 may be a radio button, a check box, a toggle button, or the like, instead of a push button. Also, for example, each button has been described as a soft button displayed on the viewer 4100, but it may be provided in a location other than the viewer 4100, or may be a physical button such as a physical switch.

The date and time display area 4150 is an area in which the recording date and time of the video displayed in the video display area 4110 is displayed. The date and time display area 4150 may also be configured to display the current date and time.

The vehicle status display area 4160 is an area where information such as the vehicle speed at the time the video was recorded is displayed. The vehicle status display area 4160 displays the status of the vehicle to which the imaging device 3100, 3200 that recorded the video displayed in the video display area 4110 is attached.

The speed graph display area 4170 is an area where the vehicle speed is displayed in a graph. In addition to the speed, the speed graph display area 4170 may also display acceleration in a graph.

The playlist display area 4180 is an area where information about stored video data is displayed. When information about video data in the playlist display area 4180 is selected, the display control unit 3811 accesses the video data in the server 3700 and displays it in the video display area 4110. At this time, the video data may be stored in the server 3700, the information display terminal 3800, or an external storage device.

13 is a diagram showing an example of a screen after image processing displayed by the display control unit 3811 of the first embodiment. The display control unit 3811 controls the display of information processed in the server 3700 on the viewer 4200. The image display area 4210 of the viewer 4200 in the first embodiment is an image that has been subjected to fog correction by selecting the fog correction button 4140 for the image display area 4110 in FIG. 12. That is, the image display area 4210 includes an image after image processing. At this time, the fog correction button 4240 is displayed in a different color from the other buttons so that it can be seen that it has been selected. Furthermore, at this time, the storage control unit 3813 stores the image captured by the imaging device 3100 in the storage area. As a result, the information display terminal 3800 can store the image before image processing in the storage unit 3820 while displaying the image after image processing in the image display area 4210.

In the first embodiment, an example has been described in which the storage control unit 3813 controls to store the image before image processing, and the display control unit 3811 controls to display the image after image processing. However, the image before image processing may be reproduced and displayed using the image after image processing. More specifically, the storage control unit 3713 controls to store the image after image processing and the image processing performed on the image in the storage unit 3720. Next, the image processing unit 3711 controls to reproduce the image before image processing based on the information of the image processing performed on the image after image processing. Then, the display control unit 3811 controls to display the reproduced image before image processing. At this time, the display control unit 3811 may also display which data is the image displayed in the video display area 4210. More specifically, the display control unit 3811 may display whether the displayed data is the original data, the data after image processing, or the data reproduced from the data after image processing to the data before image processing.

[Modification 1]
A first modification of the first embodiment will be described.
14 is a diagram showing an example of a screen after image processing displayed by the display control unit 3811 of the first modified example of the first embodiment. The image display area 4310 of the viewer 4300 in the first modified example includes a fog correction area 4311 in which fog correction has been performed on a part of the image display area 4310, and a sharpening area 4313 in which sharpening processing has been performed on an area different from the fog correction area 4311. In this way, the image processing related to the fog correction button 4340 of the first modified example and the image processing related to the sharpening button 4342 are configured so that the areas to be subjected to image processing in the image are different. The fog correction button 4340 is an example of a first object. The sharpening button 4342 is an example of a second object. The information display terminal 3800 of the first modified example can execute any image processing on different areas.
For example, the user selects a predetermined area (an area corresponding to the defog area 4311) in the image display area 4110, and selects the defog button 4340. The user also selects a predetermined area (an area corresponding to the sharpening area 4313) in the image display area 4110, and selects the sharpening button 4342. By making such a selection, a screen such as that shown in FIG.
Moreover, the display control unit 3811 can display what kind of processing has been performed in which area by displaying the processing content in each processing area. That is, the display control unit 3811 can display a processing content display section 4312 in the mist correction area 4311 so as to show that the mist correction area 4311 has performed mist correction. Moreover, the display control unit 3811 can display a processing content display section 4314 in the sharpening area 4313 so as to show that the sharpening processing has been performed. Furthermore, the display control unit 3811 can display that the mist correction button 4340, the sharpening button 4342, and the partial processing button 4344 are being performed by changing the colors of these buttons.
In the example of the first modification, defogging and sharpening are used as examples of image processing for different regions, but the present invention is not limited to this and may be other image processing. Also, the same image processing (e.g., defogging) may be performed to different degrees on different regions.

[Modification 2]
A second modification of the first embodiment will be described.
15 is a diagram showing an example of a screen before image processing, displayed by the display control unit 3811 of the second modified example of the first embodiment. The objects displayed in the viewer 4400 in the second modified example are a sunny correction button 4440, a cloudy correction button 4441, a rain correction button 4442, and a fog correction button 4443. The sunny correction button 4440, the cloudy correction button 4441, the rain correction button 4442, and the fog correction button 4443 are examples of objects related to image processing related to weather. In the second modified example, different image corrections are applied to sunny, rainy, and foggy conditions using weather information in the viewer.
The display control unit 3811 accepts an instruction for image quality correction such as gamma correction, color correction (color emphasis, color balance), and edge emphasis corresponding to the weather, by selecting each button. For example, the display control unit 3811 requests the image processing unit 3711 of the server 3700 to perform image processing such as gamma correction, color correction (color emphasis, color balance), and edge emphasis on the image so as to erase the rain by selecting the rain correction button 4442. In addition, the display control unit 3811 requests the image processing unit 3711 of the server 3700 to perform image processing such as gamma correction, color correction (color emphasis, color balance), and edge emphasis on the image so as to make the fog transparent and make the other side of the fog visible, by selecting the fog correction button 4443. In addition, the display control unit 3811 may display, as the names of these buttons, button names for performing image correction according to the purpose of the image, such as an operation button for erasing a snowstorm, a button for making the fog transparent so that the other side of the fog can be seen, and a button for erasing rain. Furthermore, the display control unit 3811 may provide new objects related to "clear", "cloudy", "rain", and "fog" in the viewer 4400, or provide objects related to "clear", "slightly cloudy", "smoke", "dust storm", "blizzard", "drizzle", "sleet", "snow", "grail", "lightning", and the like instead of "clear", "cloudy", "smoke", "sandstorm", "snowstorm", "drizzle", "drizzle", "sleet", "snow", "grail", "lightning", and the like. Instead of providing objects for each weather, an almighty automatic correction may be performed in which these weather conditions are automatically detected and correction is performed automatically. When the almighty automatic correction is configured to be possible, the display control unit 3811 displays a button for performing/not performing image correction on the screen. If the display control unit 3811 determines that there is a limit to the almighty automatic correction, it creates some kind of state. For example, the display control unit 3811 displays another correction means. With these, it is possible to perform appropriate image processing in accordance with the weather.

[Modification 3]
A third modification of the first embodiment will now be described.
16 is a diagram showing an example of a screen before image processing, which is displayed by the display control unit 3811 of Modification 3 of the embodiment 1. In the video display area 4510 of the viewer 4500 in Modification 3, an image captured by the imaging device 3100 of the vehicle 3400 traveling on a road surface in foggy weather is displayed.
At this time, the display control unit 3811 controls to highlight and display the fog correction button 4540 as an appropriate correction method. That is, the display control unit 3811 changes the button displayed on the screen based on the state of the image. This makes it possible to perform appropriate image processing based on the information in the image. As one aspect of changing the button, the display control unit 3811 can display a new button. As another aspect of changing the button, the outline of the button can be highlighted. Furthermore, as yet another aspect of changing the button, the display control unit 3811 can display the button by shining it. For example, when the vehicle 3400 runs in a city or a forest, the brightness changes depending on the location and/or direction of the run, so the display control unit 3811 may control to display a new brightness correction button, highlight the outline of the brightness correction button, or display the brightness correction button by shining it accordingly.

[Modification 4]
A fourth modification of the first embodiment will be described.
17 is a diagram showing an example of a screen before image processing, displayed by the display control unit 3811 of Modification 4 of the embodiment 1. An image captured by the imaging device 3100 of the vehicle 3400 immediately before entering a tunnel is displayed in the video display area 4610 of the viewer 4600 in Modification 4. At this time, the current location icon 4631 in the map image display area 4630 indicates that the vehicle 3400 is immediately before entering a tunnel.
The communication control unit 3812 of the information display terminal 3800 acquires road network information related to the position information of the vehicle 3400 from a map server (not shown) via the network 3900. Then, the display control unit 3811 controls to display a tunnel correction button 4640 suitable for image processing in a tunnel. That is, the display control unit 3811 controls to display a button suggesting image processing for an image based on the road network information on the screen. Here, by selecting the tunnel correction button 4640, the image processing unit 3711 automatically performs an appropriate type of correction for the tunnel position. Specifically, by selecting the tunnel correction button 4640, the image processing unit 3711 performs image processing to adjust the brightness to correct the darkness of the tunnel, reduce the influence of lighting provided in the tunnel, and reduce the influence of the lighting device of the vehicle ahead. Therefore, it is possible to perform appropriate image processing based on the road network information. Here, road network information is information relating to road conditions, and more specifically, this information includes, in addition to tunnel location information, speed limit information, road width information, 12-hour weekday traffic volume information, travel speed information (for a certain section, the distance of the section divided by the time required for travel, the required time including stops due to traffic lights, congestion, etc.), number of lanes information, etc.

[Modification 5]
A fifth modification of the first embodiment will be described.
18 is a diagram showing an example of a screen before image processing, displayed by the display control unit 3811 of the fifth modified example of the first embodiment. The sensor unit 3017 of the fifth modified example measures acceleration information of the vehicle 3400. That is, the vehicle information of the fifth modified example includes the acceleration information measured by the sensor unit 3017. The image display area 4710 of the viewer 4700 in the fifth modified example displays an image of the vehicle 3400 that is accelerating rapidly, captured by the image capture device 3100. At this time, the acceleration graph display area 4720 is a graph showing a large upward inclination of the X-axis, and the speed graph display area 4770 is a graph showing a rapid increase in speed. In addition, the viewer 4700 may be provided with an acceleration display unit 4721 such as "1G or more detected" at this time.
The display control unit 3811 controls to display an acceleration correction button 4740 suitable for image processing when the vehicle 3400 accelerates. That is, the display control unit 3811 controls to display on the acceleration correction button screen based on information from the sensor unit 3017 that detects the acceleration of the vehicle 3400 having the image capture device 3100. Here, when the acceleration correction button 4740 is selected, image processing is performed to correct screen shaking and/or poor visibility caused by sudden acceleration. This makes it possible to perform appropriate image processing based on information from the acceleration sensor of the moving object. The image capture device 3100 is an example of an image capture unit. The vehicle 3400 is an example of a moving object. The sensor unit 3017 is an example of an acceleration sensor. The acceleration correction button is an example of an object that suggests image processing for an image.
Furthermore, the display control unit 3811 accepts one or more acceleration correction settings by the user. Specifically, the display control unit 3811 accepts one or more acceleration correction settings by the user inputting a numerical value for performing the acceleration correction, dragging the acceleration graph display area 4720 or the speed graph display area 4770, etc. At this time, the display control unit 3811 displays bands of different colors in the range for each type of correction in the acceleration graph display area 4720 or the speed graph display area 4770, displays the dragged range, etc.

[Modification 6]
A sixth modification of the first embodiment will now be described.
19 is a diagram showing an example of a screen before image processing, displayed by the display control unit 3811 of Modification 6 of the first embodiment. An image captured by the imaging device 3100 of the vehicle 3400 is displayed in the image display area 4810 of the viewer 4800 in Modification 6. At this time, a current location icon 4831 in a map image display area 4830 where the vehicle 3400 is traveling indicates the location on the map where the vehicle 3400 is traveling.
The display control unit 3811 can set a processing area 4832 on the map image display area 4830 based on a user operation or the like, and performs control so as to display a fog compensation button 4840 when the current location icon 4831 enters this area. In other words, the display control unit 3811 performs control so as to display on the screen a button or the like that suggests image processing for an image based on the position information of the vehicle 3400 having the image capturing device 3100. This makes it possible to perform appropriate image processing based on the position information of the vehicle 3400.
As described above, the processing area 4832 may be set based on a user operation or the like, or may be set automatically. For example, in a case where an image processing button for brightening an image is set to be displayed when the vehicle 3400 enters a tunnel, the display control unit 3811 performs control so that an image processing button for brightening an image is displayed on the screen when the display control unit 3811 determines that the vehicle 3400 has entered a tunnel based on the position information of the vehicle 3400.

[Modification 7]
A seventh modification of the first embodiment will now be described.
20 is a diagram showing an example of a screen before image processing, displayed by the display control unit 3811 of Modification 7 of the first embodiment. An image captured by the imaging device 3100 of the vehicle 3400 of a road where it is expected to be cloudy with rain, is displayed in an image display area 4910 of the viewer 4900 in Modification 7. At this time, the viewer 4900 can be configured to display weather information 4911 for that road. In this embodiment, the viewer 4900 displays weather information 4911 relating to cloudy with rain.
The communication control unit 3812 acquires weather information of the route taken by the vehicle 3400 from a weather information providing server (not shown) based on the position information of the vehicle 3400 and the date and time information included in the image captured by the image capture device 3100. The display control unit 3811 controls to display a button suggesting image processing based on the position information of the vehicle 3400 and the weather information corresponding to the position information. That is, the display control unit 3811 controls to display an object suggesting image processing for the image on the screen based on the position information of the vehicle 3400 having the image capture device 3100 and the weather information of the date and time when the image was captured. In the seventh modification, image correction according to the time period recorded by the viewer can be applied. For example, the image processing unit 3711 can apply different image corrections to morning, night, and evening based on an instruction from the communication control unit 3812 and the weather information 4911.
In the example of the seventh modification, the display control unit 3811 controls the viewer 4900 to display a rain correction button 4941 based on the position information of the vehicle 3400 and the weather information. When the rain correction button 4941 is selected, image processing is executed to correct darkness due to rain clouds and poor visibility due to rain. This makes it possible to perform appropriate image processing based on the position information of the vehicle 3400 and the weather information of the date and time at that position. Here, the weather information includes information such as cloudy and rain, as well as sunny, clear, slightly cloudy, haze, dust storm, blizzard, drizzle, sleet, fog, snow, hail, and thunder.

[Modification 8]
An eighth modification of the first embodiment will now be described.
21 is a diagram showing an example of a screen before image processing, displayed by the display control unit 3811 of Modification 8 of the first embodiment. In the image display area 5010 of the viewer 5000 in Modification 8, an image captured by the imaging device 3100 of the vehicle 3400 approaching an intersection is displayed. At this time, the current location icon 5031 in the map image display area 5030 indicates that the vehicle 3400 is approaching the intersection. The speed graph display area 5070 shows that the vehicle 3400 is decelerating as it approaches the intersection.
The display control unit 3811 controls to display the brake correction button 5040 in response to the deceleration of the vehicle 3400. That is, the display control unit 3811 controls to display an object suggesting image processing for an image on the screen based on the operation information of the vehicle 3400 having the image capture device 3100. Here, by selecting the brake correction button 5040, blurring of the field of view due to deceleration can be reduced, and the influence of the brake lamp on the image display area 5010 can be reduced. This makes it possible to perform appropriate image processing based on the operation information of the vehicle 3400. Here, the operation information of the vehicle 3400 is information related to the operation among the vehicle information. In addition to the brake, the vehicle information includes information that can be obtained by the operation of the turn signal, back, and other OBD (On-board diagnostics).

[Modification 9]
A ninth modification of the first embodiment will be described.
Fig. 22 is a diagram showing an example of a screen before image processing, displayed by the display control unit 3811 of the ninth modification of the first embodiment. An image captured by the imaging device 3100 of the vehicle 3400 is displayed in the image display area 5110 of the viewer 5100 in the ninth modification. At this time, information related to the date and time is displayed in the date and time display area 5150. In the example of Fig. 22, it is shown that the time is night.
The display control unit 3811 controls to display the night correction button 5140 based on date and time information, which is attribute information of the image. That is, the display control unit 3811 controls to display a button suggesting image processing for the image based on the date and time when the image was captured on the screen. Here, by selecting the night correction button 5140, the darkness of the night is corrected to be bright so that what is around can be displayed. This makes it possible to perform appropriate image processing based on the date and time when the image was captured.
Here, the night correction button is an example of a button that suggests image processing for an image based on the date and time, and may be, for example, a button for correcting brightness due to a sunset to clarify the field of view.
The automatic correction may be performed by combining the corrections according to the above-described modified examples 3 to 9. The combination of the corrections according to the modified examples 3 to 9 may be set as an OR condition or an AND condition.

[Modification 10]
A tenth modification of the first embodiment will be described.
FIG. 23 is a diagram showing an example of a screen after image processing displayed by the display control unit 3811 of the modification 10 of the first embodiment. In the image display area 5210 of the viewer 5200 in the modification 10, an image captured by the image capture device 3100 of the vehicle 3400 traveling on a road surface in foggy weather is displayed. In addition, the result of fog correction is displayed in the fog correction area 5211, which is a part of the image display area 5210. That is, the fog correction area 5221 displays the result of image processing performed on a part of the image captured by the image capture device 3100. When an instruction is given to perform image processing on the fog correction area 5221, the image processing unit 3711 performs image processing on a part of the image captured by the image capture device 3100. In addition, the fog correction area 5211 is an area set according to a user operation or the like before performing image processing. That is, the part of the image is an area set among the entire area of the image captured by the image capture device 3100.
More specifically, the display control unit 3811 sets the range of image processing by selecting the partial processing button 5245 and specifying the range on the screen. Next, when the fog correction button 5240 is selected, the display control unit 3811 determines that the set range is the fog correction area 5211 where fog correction is performed. Then, the communication control unit 3812 of the information display terminal 3800 requests the server 3700 to perform fog correction for the fog correction area 5211 via the network 3900. Upon receiving the request, the image processing unit 3711 performs fog correction for the fog correction area 5211. The communication control unit 3812 acquires the result of fog correction for the fog correction area 5211. The display control unit 3811 displays the result of image processing for the fog correction area 5211. This allows image processing to be performed on a part of the captured image. This also allows the area where image processing is performed to be set.

[Modification 11]
An eleventh modification of the first embodiment will now be described.
24 is a diagram showing an example of a screen after image processing displayed by the display control unit 3811 of the modified example 11 of the first embodiment. In the image display area 5310 of the viewer 5300 in the modified example 11, an image captured by the image capture device 3100 of the vehicle 3400 in the middle of the night is displayed as an example. In addition, the night correction area 5311, which is a part of the image display area 5310, displays the result of night correction. Furthermore, the date and time display area 5350 indicates that the time is night. When the date and time display area 5350 is night, the night correction area 5311 displays the result of night correction. That is, the image processing on the image is image processing on a part of the image captured by the image capture device 3100, and the part of the image is an image captured during a set time period among the images captured by the image capture device 3100.
The display control unit 3811 sets the range of image processing based on the operation of the partial treatment button 5345 and the date and time information of the image. Next, the display control unit 3811 determines that the range is the night correction area 5311 where correction is performed at night based on the operation of the night correction button 5340. Furthermore, the communication control unit 3812 instructs the server 3700 via the network 3900 to perform image processing on the night correction area 5311. Next, the image processing unit 3711 executes image processing on the night correction area 5311. Furthermore, the communication control unit 3812 acquires the result of the image processing on the night correction area 5311. Finally, the display control unit 3811 displays the result of the image processing on the night correction area 5311. This makes it possible to perform appropriate image processing on a partial area of the image taken during the set time period.
In the eleventh variant example, the partial treatment button 5345 and the night correction button 5340 have been used as examples for explanation, but the display control unit 3811 may be provided with a means for inputting a time period into the viewer 5300, and may instruct the viewer 5300 to execute the set image processing on images taken during the time period input via this means.

[Modification 12]
A twelfth modification of the first embodiment will now be described.
FIG. 25 is a diagram showing an example of a screen after image processing, displayed by the display control unit 3811 of the modified example 12 of the first embodiment. The position information acquisition unit 3020 of the modified example 12 acquires the position information of the vehicle 3400. That is, the other information of the modified example 12 includes the position information acquired by the position information acquisition unit 3020. The image display area 5410 of the viewer 5400 in the modified example 12 displays an image captured by the imaging device 3100 of the vehicle 3400. In addition, the fog correction area 5411, which is a part of the image display area 5410, displays the result of fog correction. Furthermore, the current location icon 5431 in the map image display area 5430 indicates the location on the map where the vehicle 3400 is traveling. In addition, the map image display area 5430 displays a processing area 5432. When the current location icon 5431 is in this processing area 5432, the fog correction area 5411 displays the result of fog correction. In other words, image processing of an image is image processing of a portion of an image captured by the imaging device 3100, and the portion of the image is an image captured within a range in which the position information of the vehicle 3400 having the imaging device 3100 is set.
First, the display control unit 3811 sets the range of image processing based on the operation of the partial processing button 5445 and the position information. Next, based on the operation of the fog correction button 5440, it is determined that the range is the fog correction area 5411 where fog correction is performed at a predetermined position. Furthermore, the communication control unit 3812 instructs the server 3700 via the network 3900 to perform image processing on the fog correction area 5411. Next, the image processing unit 3711 executes image processing on the fog correction area 5411. Furthermore, the communication control unit 3812 acquires the result of the image processing on the fog correction area 5411. Finally, the display control unit 3811 displays the result of the image processing on the fog correction area 5411. This makes it possible to appropriately perform image processing on a partial area of an image captured by the vehicle 3400 moving at the set position.
In the twelfth modification example, the partial treatment button 5445 and the fog correction button 5440 have been described as examples. However, the display control unit 3811 may be provided with a means for inputting latitude information and longitude information to the viewer 5300, and may instruct the viewer 5300 to execute the set image processing on an image captured at the latitude and longitude input via this means.
In addition, the input of location information is not limited to latitude and longitude information, and the display control unit 3811 may instruct the display control unit 3811 to execute the set image processing on an image captured at a location within a range selected by the user on the map displayed on the viewer 5300.
In addition, the display control unit 3811 may display historical route information of the user's vehicle on the viewer 5300, and instruct the viewer 5300 to perform the set image processing on images captured at positions within a range selected by the user on the displayed historical route.

[Modification 13]
A thirteenth modification of the first embodiment will be described.
FIG. 26 is a diagram showing an example of a screen after image processing, displayed by the display control unit 3811 of the modified example 13 of the first embodiment. In the image display area 5510 of the viewer 5500 in the modified example 13, an image captured by the imaging device 3100 of the vehicle 3400 on a road where it is expected to be cloudy and then rain is displayed. In addition, in the fog correction area 5511, which is a part of the image display area 5510, the result of fog correction is displayed. Furthermore, the viewer 5500 can be made to display weather information 5512 on the road. Here, the viewer 5500 displays weather information 5512 related to cloudy and then rain. When the weather at the date and time of the image being played back is cloudy, the fog correction area 5511 displays the result of fog correction. That is, the image processing on the image is image processing on a part of the image captured by the imaging device 3100, and the part of the image is an image showing the weather for which weather information at the date and time when the image was captured at the position of the vehicle 3400 having the imaging device 3100 is set.
First, the communication control unit 3812 of the information display terminal 3800 acquires weather information of the route taken by the vehicle 3400 from a weather information providing server (not shown) based on the position information of the vehicle 3400 and the date and time information included in the image captured by the image capturing device 3100. Next, the display control unit 3811 sets the range of image processing based on the operation of the partial processing button 5545, the time information, the weather information, and the like. Furthermore, based on the operation of the fog correction button 5540, it is determined that the range is the fog correction area 5511 in which fog correction is performed at a predetermined position. Furthermore, the communication control unit 3812 instructs the server 3700 via the network 3900 to perform image processing on the fog correction area 5511. Next, the image processing unit 3711 executes image processing on the fog correction area 5511. Furthermore, the communication control unit 3812 acquires the result of the image processing on the fog correction area 5511. Finally, the display control unit 3811 displays the result of the image processing on the fog correction area 5511. This makes it possible to perform appropriate image processing on a partial area of the captured image based on the weather at the position of the vehicle 3400. Here, the weather information includes not only cloudy and rainy weather, but also sunny, fine, slightly cloudy, haze, dust storm, blizzard, drizzle, sleet, fog, snow, hail, and thunder.

[Modification 14]
A fourteenth modification of the first embodiment will now be described.
FIG. 27 is a diagram showing an example of a screen after image processing, displayed by the display control unit 3811 of the modification 14 of the first embodiment. The sensor unit 3017 of the modification 14 measures acceleration information of the vehicle 3400. That is, the vehicle information of the modification 14 includes the acceleration information measured by the sensor unit 3017. The image display area 5610 of the viewer 5600 in the modification 14 displays an image captured by the imaging device 3100 of the vehicle 3400 that is accelerating rapidly. At this time, the acceleration graph display area 5620 is a graph showing a large upward inclination of the X-axis, and the speed graph display area 5670 is a graph showing a rapid increase in speed. In addition, the viewer 5600 may be provided with an acceleration display unit 5621 such as "1G or more detected". In addition, the acceleration correction area 5611, which is a part of the image display area 5610, displays the result of acceleration correction. Furthermore, when the acceleration sensor of the vehicle 3400 detects 1 G or more, the result of the acceleration correction is displayed in the acceleration correction area 5611. That is, the image processing for the image is image processing for a part of the image captured by the image capturing device 3100, and the part of the image is an image in which the sensor information of the vehicle 3400 having the image capturing device 3100 is within a set threshold value.
First, the communication control unit 3812 of the information display terminal 3800 acquires image information and acceleration information of the vehicle 3400 from the server 3700 via the network 3900. Next, the display control unit 3811 sets the range of image processing based on the operation of the partial action button 5645, the acceleration information of the vehicle 3400, and the like. Furthermore, based on the operation of the acceleration correction button 5640, it is determined that the range is the acceleration correction area 5611 in which acceleration correction is performed at a predetermined position. Next, the communication control unit 3812 instructs the server 3700 via the network 3900 to perform image processing on the acceleration correction area 5611. Furthermore, the image processing unit 3711 executes image processing on the acceleration correction area 5611. Next, the communication control unit 3812 acquires the result of image processing on the acceleration correction area 5611. Finally, the display control unit 3811 displays the result of image processing on the acceleration correction area 5611. This makes it possible to appropriately perform image processing on a partial area of the image captured based on the sensor information of the vehicle 3400. Here, the sensor information includes, in addition to the acceleration of the vehicle 3400, information such as the speed of the vehicle 3400 (vehicle speed), engine speed, engine load factor, throttle opening, ignition timing, percentage of remaining fuel, intake manifold pressure, intake air volume (MAF), injection open time, engine coolant temperature (coolant temperature), temperature of the air taken into the engine (intake temperature), air temperature outside the vehicle (outside air temperature), amount of remaining fuel in the fuel tank (remaining fuel amount), fuel flow rate, instantaneous fuel consumption, accelerator opening, turn signal information (operation of left and right turn signals (on/off)), brake opening, steering wheel rotation steering angle, gear position, and door open/close status information.
The automatic correction may be performed by combining the corrections according to the above-described modified examples 10 to 14. The combination of the corrections according to the modified examples 10 to 14 may be set as an OR condition or an AND condition.

[Modification 15]
A fifteenth modification of the first embodiment will be described.
28 is a diagram showing an example of a screen after image processing, which is displayed by the display control unit 3811 of the fifteenth modification of the first embodiment. In the video display area 5710 of the viewer 5700 in the fifteenth modification, a video captured by the imaging device 3100 of the vehicle 3400 is displayed.
The display control unit 3811 controls to set an automatic fog correction area 5711 as a range for correction based on a preset setting, and to set a manual fog correction area 5713 as a range for correction based on a user operation. The display control unit 3811 also controls to display a processing content display section 5712 such as "automatic processing" for the automatic fog correction area 5711, and a processing content display section 5714 such as "manual processing" for the manual fog correction area 5713. That is, the screen includes information on whether the range for image processing was selected automatically or manually. This makes it possible to distinguish between a manually set range and an automatically set range for the range of image processing that has been executed.

[Modification 16]
A sixteenth modification of the first embodiment will be described.
29 is a diagram showing an example of a screen displayed by the display control unit 3811 of the modified example 16 of the first embodiment. In the video display area 5810 of the viewer 5800 in the modified example 16, an original image display unit 5811 before image processing, a processing 1 result display unit 5813 after image processing, a processing 2 result display unit 5815 after image processing, and a processing 3 result display unit 5817 after image processing are displayed as different streams. Each display unit displays an image captured by the imaging device 3100 of the vehicle 3400. That is, the screen includes an image captured by the imaging device 3100 and one or more images after image processing.
The display control unit 3811 controls to display a plurality of images by selecting a stream display button 5845. Here, the display control unit 3811 may also display a plurality of images in a chronological order. Furthermore, here, the display control unit 3811 controls to display an original display unit 5812 such as "original" for the original image display unit 5811, to display a processing 1 content display unit 5814 such as "processing 1" for the processing 1 result display unit 5813, to display a processing 2 content display unit 5816 such as "processing 2" for the processing 2 result display unit 5815, and to display a processing 3 content display unit 5818 such as "processing 3" for the processing 3 result display unit 5817. This allows the images before and after image processing to be displayed for comparison.
Moreover, the display control unit 3811 displays the original image display unit 5811, the processing 1 result display unit 5813, the processing 2 result display unit 5815, and the processing 3 result display unit 5817 as a plurality of streams. Furthermore, the display control unit 3811 displays the original image display unit 5811, the processing 1 result display unit 5813, the processing 2 result display unit 5815, and the processing 3 result display unit 5817 as a target for writing or a target for playing, etc., so that they can be selected. At this time, the display control unit 3811 can control the selected display unit to play or to output as an image file. In the sixteenth modification, the display control unit 3811 opens a plurality of candidate video windows, the processing 1 result display unit 5813, the processing 2 result display unit 5815, and the processing 3 result display unit 5817, and plays them in time synchronization with the original image display unit 5811, which is the original video window. The display control unit 3811 accepts the selection of the best one from among them by the user's operation of a selection button, etc. The storage control unit 3813 stores the selection result and the relationship with the information of the original video in the storage unit 3820. Furthermore, the storage control unit 3813 trains a learning model to learn the user's image processing tendency based on the stored selection result and the relationship with the information of the original video, and stores the result in the storage unit 3820. The display control unit 3811 adjusts the image processing based on the result of the machine learning. That is, the display control unit 3811 adjusts the image processing parameters based on the learning information regarding the selection of the image after one or more image processes. This allows the display control unit 3811 to learn the selected image processing result and perform subsequent image processing more appropriately. The display control unit 3811 may also perform image processing on a part of the image.

[Modification 17]
A seventeenth modification of the first embodiment will be described.
30 is a diagram showing an example of a screen after image processing, which is displayed by the display control unit 3811 of the seventeenth modification of the first embodiment. In the video display area 5910 of the viewer 5900 in the seventeenth modification, a video captured by the imaging device 3100 of the vehicle 3400 is displayed.
When the defog button 5940 is selected, the display control unit 3811 controls the image processing result of defogging to be displayed in the image display area 5910. The display control unit 3811 also controls the image display area 5910 to display a processing content display unit 5911 such as "Defog correction in progress." In other words, the screen includes information on what kind of image processing has been performed on the image captured by the imaging device 3100. This makes it possible to check what kind of image processing has been performed.

[Modification 18]
An eighteenth modification of the first embodiment will now be described.
31 is a diagram showing an example of a screen displayed by the display control unit 3811 of Modification 18 of the embodiment 1. In a playlist display area 6080 of a viewer 6000 in Modification 18, recording information in the storage unit 3720 or the storage unit 3820 is displayed.
The storage control unit 3813 stores information on what kind of image processing was performed on the image captured by the imaging device 3100 as log information in the storage unit 3820. At this time, the storage control unit 3813 can control the log information to be recorded separately into static information that does not change during shooting and dynamic information that changes depending on the image state during shooting. More specifically, the storage control unit 3813 controls the static information to be recorded for one file, and the dynamic information to be recorded at least at the change point, preferably across multiple frames such as all frames. In addition, in the eighteenth modification, the server 3700 may be provided with a function to record according to the user's selection of whether to record with or without fog removal processing, and a function to record information that can determine which was selected in the viewer. The storage control unit 3713 of the server 3700 can perform the same processing on the storage unit 3720. The display control unit 3811 can control the log information to be read and processed at different timings. The storage control unit 3813 may be configured to record drive recorder parameters related to image quality, such as parameters of the drive recorder's image sensor and ISP (described later), in association with the drive recorder's video data captured with those parameters. In this case, the display control unit 3811 may perform playback using those parameters when playing back on the viewer or the drive recorder. The display control unit 3811 may have a function of switching between playing back without using those parameters when playing back using those parameters.
At this time, the display control unit 3811 can display information on what kind of image processing has been performed on the image captured by the image capturing device 3100. That is, the display control unit 3811 controls to display what kind of image processing has been performed and at what timing on the image captured by the image capturing device 3100, as in the playlist 6081. This makes it possible to check what kind of image processing has been performed.
Furthermore, when playing the playlist 6081, the display control unit 3811 may display a selection as to whether to play the image before image processing or to play the image after image processing based on the log information. Furthermore, at this time, the display control unit 3811 may be configured to convert the image after image processing to the image before image processing based on the log information and output it to the screen. At this time, the image processing unit 3711 performs image processing so as to return the image to the original image based on the log information related to the image processing in the storage unit 3720 and the image after image processing. Next, the communication control unit 3812 acquires the image that has been returned to the original image. Furthermore, the display control unit 3811 displays the image that has been returned to the original image. This allows the image after image processing to be returned to the original image before image processing.

[Modification 19]
A nineteenth modification of the first embodiment will be described.
32 is a diagram showing an example of a screen before image processing, which is displayed by the display control unit 3811 of Modification 19 of the embodiment 1. In the image display area 6110 of the viewer 6100 in Modification 19, an image captured by the imaging device 3100 of the vehicle 3400 traveling in fog is displayed.
The image processing unit 3711 of the server 3700 analyzes the image and detects fog. Next, the communication control unit 3712 transmits the fog detection result to the information display terminal 3800. Thereafter, the display control unit 3811 controls the detection information display unit 6111 to display "Fog detected" based on the information that fog has been detected. Also, at this time, the display control unit 3811 controls the fog correction button 6140 to be displayed based on the information that fog has been detected. In other words, the display control unit 3811 changes the buttons included on the screen according to the object detected from the image. This makes it possible to change the method of image processing based on the object in the image.

[Modification 20]
A twentieth modification of the first embodiment will now be described.
33 is a diagram showing an example of a screen displayed by the display control unit 3811 of the modification 20 of the first embodiment. The viewer 6200 in the modification 20 includes an image processing adjustment area 6290. The display control unit 3811 accepts adjustment of image quality parameters by a user's operation of the image processing adjustment area 6290. The image processing adjustment area 6290 includes a hue adjustment object 6291, a brightness adjustment object 6292, a contrast adjustment object 6293, a saturation adjustment object 6294, a gamma value adjustment object 6295, a sigma value adjustment object 6296, a radius adjustment object 6297, and a fluctuation amount adjustment object 6298. When the hue adjustment object 6291 is selected, image processing is performed to adjust the hue of the image. When the brightness adjustment object 6292 is selected, image processing is performed to adjust the brightness of the image. When the contrast adjustment object 6293 is selected, image processing is performed to adjust the contrast of the image. Moreover, by selecting a saturation adjustment object 6294, image processing is performed to adjust the saturation of the image. Furthermore, by selecting a gamma value adjustment object 6295, image processing is performed to adjust the gamma value of the image. Furthermore, by selecting a sigma value adjustment object 6296, image processing is performed to adjust the sigma value of the image. Furthermore, when processing is performed on a part of the image, by selecting a radius adjustment object 6297, image processing is performed to adjust the boundary between the area where the image processing is performed and the area where the image processing is not performed. By selecting a variation adjustment object 6298, image processing is performed to adjust the number of images per unit time. That is, the screen includes an adjustment object that adjusts the degree of image processing. In the modification 20, each adjustment object is displayed in the form of a slider, but in addition to the slider, a radio button, a check box, a pull-down menu, or the like may be used.
When the display control unit 3811 receives an operation of the operation unit provided on these adjustment objects, it adjusts the degree of image processing according to the received operation. For example, in the modification 20, when the hue adjustment object 6291 is moved to the left end, the display control unit 3811 requests the image processing unit 3711 to perform image processing to set the hue to 0. When the hue adjustment object 6291 is moved to the center, the display control unit 3811 requests the image processing unit 3711 to perform image processing to set the hue to 50. When the hue adjustment object 6291 is moved to the right end, the display control unit 3811 requests the image processing unit 3711 to perform image processing to set the hue to 100. The same applies to the brightness adjustment object 6292, the contrast adjustment object 6293, the saturation adjustment object 6294, the gamma value adjustment object 6295, the sigma value adjustment object 6296, the radius adjustment object 6297, and the fluctuation amount adjustment object 6298. That is, the display control unit 3811 adjusts the degree of image processing in response to the operation of the adjustment object, thereby making it possible to adjust the degree of image processing such as brightness and darkness.
The adjustment object can be configured to adjust the image while recording or playing it. At this time, the storage control unit 3813 controls to store the timing of the adjustment object operation in association with the timing of recording the image or the timing of playing the image. This makes it possible to know at what timing the degree of image processing was adjusted.
Furthermore, the storage control unit 3813 stores the adjustment result and the relationship with the information of the original video in the storage unit 3820. Furthermore, the storage control unit 3813 trains a learning model to learn the user's image processing tendency based on the stored adjustment result and the relationship with the information of the original video, and stores the result in the storage unit 3820. The display control unit 3811 adjusts the image processing based on the result of this machine learning. That is, the display control unit 3811 adjusts the degree of image processing based on learning information of the degree of adjustment of past image processing. This makes it possible to learn the adjustment result of the degree of image processing and perform subsequent image processing more appropriately. Furthermore, the display control unit 3811 may perform image processing on a part of the image.
Furthermore, the display control unit 3811 may be configured to be able to instruct processing in the viewer to refrain from further correction processing for image quality parameters that have been corrected by the drive recorder. On the other hand, the display control unit 3811 may be configured to instruct processing in the viewer to add more correction to image quality parameters that have not been corrected or cannot be corrected by the drive recorder.

[Modification 21]
A twenty-first modification of the first embodiment will now be described.
34 is a diagram showing an example of a screen displayed by the display control unit 3811 of Modification 21 of the first embodiment. The viewer 6300 in Modification 21 includes an image processing adjustment area 6390. The image processing adjustment area 6390 includes an image fixing instruction section 6391, a sharpness fixing instruction section 6392, a banding noise removal fixing instruction section 6393, and a film graining fixing instruction section 6394. In other words, the screen includes a fixing instruction section that prohibits adjustment by an adjustment object. The fixing instruction section is an example of a fixing instruction object.
When a fixing instruction is given to the fixing instruction section that fixes the adjustment object, the display control section 3811 prohibits the change of the operation of the adjustment object. In other words, the display control section 3811 can fix the adjustment of the degree of image processing by operating these fixing instruction sections. For example, in the modification 21, by checking the check box of the image fixing instruction section 6391, the display control section 3811 fixes the adjustment of the degree of "hue", "brightness", "contrast", "saturation", and "gamma" in the image adjustment area. At this time, the display control section 3811 displays the image quality adjustment result of the fixed adjustment object, that is, the display control section 3811 prohibits the adjustment by the adjustment object in response to the operation of the fixing instruction object. This makes it possible to fix the degree of image processing. At this time, the display control section 3811 displays the adjustment of "hue", "brightness", "contrast", "saturation", and "gamma" as if the adjustment is fixed. For example, when a check box for operating the fixed instruction part is checked, the display control unit 3811 displays the color of the operation part of the adjustment object in a lighter color so that it cannot be operated, or hides the operation part of the adjustment object so that it cannot be operated.

[Modification 22]
A twenty-second modification of the first embodiment will now be described.
35 is a diagram showing an example of a screen displayed by the display control unit 3811 of Modification 22 of the first embodiment. In the viewer 6400 of Modification 22, a file button 6401 is expanded to display options related to image output, such as "Browse file", "Save over", "Save as", "Write processing results", "Write with location information deleted", and "Close". In other words, the screen includes an image output object.
When the file button 6401 is selected, the display control unit 3811 controls to display a menu related to file processing such as "view file", "overwrite save", "save as", "write processing result", "write with location information deleted", and "close". When any file processing is selected, the processing is performed. Here, when "write with location information deleted", the control unit 3810 deletes the location information linked to the image information and outputs the image information. In other words, the control unit 3810 deletes the location information of the image and outputs the image in response to an operation of the image output object. This allows an operation to delete the location information linked to the image. Here, when "write with location information deleted" is selected, the control unit 3810 may write out the video for posting with the location information deleted. Here, the video for posting is a video converted into a format for video posting. At this time, the control unit 3810 may automatically set a title, thumbnail, etc. for the video for posting. Furthermore, when "Write Processing Result" is selected, the control unit 3810 outputs the image information without deleting the location information associated with the image information. Here, when "Write Processing Result" is selected, the control unit 3810 may write out the video for posting without deleting the location information. Furthermore, when outputting the image information, the control unit 3810 may be configured to output only the image before image processing, may output the image after image processing, or may be configured to output the image before image processing and the image after image processing at the same time. Furthermore, when using the write function for posting, the control unit 3810 may be configured to have a function of storing the original video and the processed video in one file and uploading it. At this time, the video to be written out may be a stream in which both videos are arranged side by side, or may be a video stream in which both videos are arranged back and forth in time and connected. Furthermore, at this time, the control unit 3810 may write out the video by embedding a display indicating that each video is the "original video" and the "processed video". Furthermore, the control unit 3810 may be configured to have a function of selecting the spatial range into which the video is written out by using a rectangle or the like when writing out the video. At this time, it is preferable that image quality correction is performed only within the selected range, and then the image is re-encoded and written out.

[Modification 23]
A twenty-third modification of the first embodiment will now be described.
36 is a diagram showing an example of a screen displayed by the display control unit 3811 of Modification 23 of the first embodiment. An image captured by the imaging device 3100 of the vehicle 3400 traveling behind another vehicle 6511 is displayed in an image display area 6510 of a viewer 6500 in Modification 23. The image display area 6510 includes license plate information 6512 that can identify the other vehicle 6511 by performing HDR processing based on the operation of an HDR button 6541.
The display control unit 3811 controls to display a personal information processing button 6540 when the image display area 6510 contains information that can identify an individual, such as license plate information 6512. The display control unit 3811 controls to display the license plate information 6512 in a masked state based on the operation of the personal information processing button 6540. That is, when the image processed image contains information that can identify an individual, the display control unit 3811 requests execution of a masking process on the information that can identify an individual. This makes it possible to perform appropriate image processing so that information that can identify an individual is not included. Here, the information that can identify an individual includes a name, a date of birth, a face, a license plate, information that can be linked to a specific individual, and the like.

[Modification 24]
In the above-described embodiment, for example, as shown in FIG. 12, an example in which buttons corresponding to each of a plurality of image processes are displayed on the screen of the information display terminal 3800 has been described. However, the information display terminal 3800 may display fewer buttons than the number of the plurality of image processes. The display control unit 3811 may request image processing from the image processing unit 3711, etc., in response to the operation corresponding to the set button. More specifically, the display control unit 3811 may display, for example, one button on the screen, and when the button is clicked, request the image processing unit 3711, etc. to perform a first image process, when the button is double-clicked, request the image processing unit 3711, etc. to perform a second image process, and when the button is triple-clicked, request the image processing unit 3711, etc. to perform a third image process.
According to the twenty-fourth modification, it is possible to provide a screen that enables the user to instruct the image processing that he or she desires with a small number of buttons.

[Embodiment 2]
In the first embodiment, the server 3700 processes images and the images are displayed on the information display terminal 3800 .
In the modified example, a case where an information processing terminal 6800 processes an image and displays the image will be described with reference to Figs. 37 and 38.

[System configuration of information processing system 6600]
37 is a schematic diagram showing a system configuration of an information processing system 6600 according to a modified example of the second embodiment. As shown in FIG. 37, the information processing system 6600 includes a vehicle 3400, a plurality of information processing terminals 6800D, 6800E, and 6800F, and a network 3900. The vehicle 3400 is configured to be able to communicate with the plurality of information processing terminals 6800D, 6800E, and 6800F via the network 3900. More specifically, the vehicle 3400 transmits information held by the vehicle 3400 to the plurality of information processing terminals 6800D, 6800E, and 6800F via the network 3900. The plurality of information processing terminals 6800D, 6800E, and 6800F are configured to be able to communicate with the vehicle 3400 via the network 3900. More specifically, the plurality of information processing terminals 6800D, 6800E, 6800F acquire information of the plurality of information processing terminals 6800D, 6800E, 6800F via the network 3900.

[Functional configuration of the information processing terminal 6800]
Fig. 38 is a schematic diagram showing a functional configuration of an information processing terminal 6800 according to a modified example of embodiment 2. The hardware configuration of the information processing terminal 6800 is similar to the hardware configuration of the information display terminal 3800 of embodiment 1. That is, the control unit of the information processing terminal 6800 executes processing based on a program stored in the storage unit of the information processing terminal 6800, thereby realizing the functional configuration and the like of the information processing terminal 6800 shown in Fig. 38.
As shown in FIG. 38, the information processing terminal 6800 includes, as its functional components, a display control unit 6811, a communication control unit 6812, a memory control unit 6813, and an image processing unit 6814.

The display control unit 6811 controls the information to be output to an output unit (not shown) of the information processing terminal 6800. More specifically, the display control unit 6811 controls the display of a screen including information related to an image.

The communication control unit 6812 controls the exchange of information with the server 3700 via the network 3900.

The memory control unit 6813 stores information in the memory unit of the information processing terminal 6800 and retrieves stored information.

The image processing unit 6814 performs image processing on the image in the storage unit of the information processing terminal 6800 based on an operation instruction via the screen. This allows the information processing terminal 6800 to perform image processing. In the second embodiment, the image processing unit 6814 performs various image processing based on a request via the screen of the information display terminal 3800. However, the information processing terminal 6800 may have an image processing unit for each image processing. In addition, instead of the software of the information processing terminal 6800 performing image processing, dedicated hardware (e.g., GPU, etc.) of the information processing terminal 6800 may perform image processing. When dedicated hardware performs image processing, one piece of hardware of the information processing terminal 6800 may be configured to perform multiple image processing, or multiple pieces of hardware of the information processing terminal 6800 corresponding to each of the multiple image processing may be configured to perform image processing.

According to the second embodiment, image processing can be performed on the information processing terminal side.

[Embodiment 3]
In the third embodiment, an example in which image processing is performed within the imaging device 3100 of the vehicle 3400 will be described.

The control unit 3011 records the image captured by the imaging unit 3015 in the storage medium 3500. The control unit 3011 performs image processing on the recorded image using the processor 3111. At this time, the control unit 3011 records in the storage medium 3500 what type of image processing was performed and at what timing. At this time, the control unit 3011 can transmit the information recorded in the storage medium 3500 via the network 3900.

The control unit 3011 controls the image processing to be written out by performing a predetermined operation during playback in the image capture device 3100. The control unit 3011 issues a predetermined notification while the image processed in the image capture device 3100 is being written out. More specifically, while the image processed in the image capture device 3100 is being written out, the control unit 3011 controls the display unit 3013 to display a message saying "Do not turn off the engine" and controls the audio output unit 3014 to output a message saying "Do not turn off the engine". These notifications can be issued continuously (repeatedly).

In the third embodiment, the control unit 3011 may also have an image processing function capable of removing fog. In this case, the control unit 3011 removes fog from the target image in response to a user's operation. Next, the control unit 3011 records the image after the fog removal process in response to a user's operation. Furthermore, the control unit 3011 may be configured to reproduce and play the image before the fog removal process. This reproduction may be performed by the control unit 3011 recording the image before the fog removal process in association with the data after the fog removal process, or by recording information indicating the degree of fog removal by the fog removal process in association with the image after the fog removal process and using this information. For example, if there is a large difference between the state seen by the driver and the state after the image correction, the image recorded will be different from the state originally seen by the driver, which may be disadvantageous to the driver if he or she causes an accident, but it is possible to grasp the degree of fog removal by the fog removal process.

The control unit 3011 may also have a function of recording according to the user's selection of whether to record with or without fog removal processing. Furthermore, the control unit 3011 may have a function of displaying this information on the information display terminal. The control unit 3011 may also have a function of recording information that allows a determination on the viewer of the information display terminal as to whether recording with or without fog removal processing was selected. Note that the information may be transferred from the drive recorder to the information display terminal via the network 3900, or may be transferred using an information recording medium such as an SD card.

According to the third embodiment, image processing can be performed on the vehicle side.

[Embodiment 4]
Next, a fourth embodiment will be described. The fourth embodiment may be implemented by the information processing system 3000 shown in Fig. 1. Fig. 39 is a functional block diagram realized by the storage unit 22 and the control unit 23. The control unit 23 shown in Fig. 39 may be, for example, a control unit 3810 provided in the information display terminals 3800A, 3800B, 3800C, etc. The storage unit 22 shown in Fig. 39 may be, for example, a storage unit 3820 provided in the information display terminals 3800A, 3800B, 3800C, etc.

The storage unit 22 stores various information defined by the above description. This can be implemented, for example, as a storage device such as a solid state drive (SSD) that stores various programs related to the information display terminal 3800 executed by the control unit 23, or as a memory such as a random access memory (RAM) that stores temporarily required information (arguments, arrays, etc.) related to the program calculations. It may also be a combination of these. In particular, the storage unit 32 may store, for example, a first program P1 and a second program P2 that serve as a viewer for image data IM as various programs related to the information display terminal 3800 executed by the control unit 33.

The control unit 23 may function as an acquisition unit 231, a reception unit 232, a correction unit 233, a release unit 234, and a display control unit 235 by executing various programs stored in the storage unit 22. In other words, information processing by the software stored in the storage unit 22 may be specifically realized by the control unit 23, which is an example of hardware, and may be executed as each functional unit included in the control unit 23. Each functional unit related to the control unit 23 will be described below.

The acquisition unit 231 may be configured to acquire various types of information. For example, the acquisition unit 231 may acquire specific information IF1 related to the state or situation of the moving object as an acquisition step. This will be described in further detail later.

The reception unit 232 may be configured to receive various types of information. For example, the reception unit 232 may receive a first input by the user as a first reception step. The reception unit 232 may receive a second input by the user as a second reception step. This will be described in further detail later.

The correction unit 233 may be configured to execute a correction process. For example, as a first correction step, the correction unit 233 may execute a first correction process on the image data IM generated by the imaging device 3100 of the moving object at a time corresponding to the specific information IF1 based on the specific information IF1. The first correction process may include an automatic correction process by a program such as artificial intelligence. As a second correction step, the correction unit 233 may execute a second correction process on the image data IM based on a second input. The second correction process may include any manual correction process based on, for example, a user input. Furthermore, the correction unit 233 may execute a preparation step and a processing step. This will be described in more detail later.

The release unit 234 may be configured to release the correction process performed by the correction unit 233. For example, as a release step, the release unit 234 may release the first correction process performed on the image data IM based on the first input. This will be described in further detail later.

The display control unit 235 is configured to control various visual information to be displayed on a display unit capable of displaying the same. Here, the visual information may be information itself that is generated in a manner that is visible to the user, such as a screen, an image, an icon, text, etc., or may be rendering information for displaying a screen, an image, an icon, text, etc. For example, the display control unit 235 may display the image data IM on which the first correction process has been performed in a reproducible manner as a display control step. This will be described in further detail later.

[Outline of information processing]
Next, an information processing method of the information processing system 3000 according to the fourth embodiment will be described. The information processing method includes steps of the information display terminal 3800, which is an example of an information processing device. Fig. 40 is an activity diagram showing the flow of information processing according to the fourth embodiment. Below, this information processing will be outlined along this activity diagram.

First, when the moving object is moving, the image capturing device 3100 may record the moving state as image data IM (activity A101). The image data IM may be, for example, video format data having a predetermined frame rate. The frame rate of the image capturing device 3100 is not particularly limited, but is, for example, 10 fps or more, preferably 30 fps or more, more preferably 60 fps or more, and specifically, for example, 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190, 200, 210, 220, 230, 240, 250 fps, and may be within a range between any two of the numerical values exemplified here. The generated image data IM may be displayed on the display unit 3013 of the image capturing device 3100 as a live view. The recording of such image data IM may be performed continuously until the traveling of the moving object is completed.

Next, the information display terminal 3800 connected to the image capture device 3100 via a network may receive a series of image data IM generated by continuous recording in activity A101 (activity A102). That is, the reception unit 332 may receive the image data IM from the image capture device 3100. Of course, instead of a network, the reception unit 332 may receive the image data IM via a storage medium such as an SD card (registered trademark). In the following explanation, for simplicity, an example via a network is given, but this is not necessarily limited to this.

Next, the acquisition unit 231 may acquire specific information IF1 related to the state or situation of the moving object as an acquisition step (activity A103). Specifically, the control unit 23 may acquire the specific information IF1 by reading out a dedicated program stored in the storage unit 22, and store the specific information IF1 in the storage unit 22. The specific information IF1 may be included in the image data IM received by the reception unit 332, or may be received as separate information from the shooting device 3100 or an external device. Preferably, the specific information IF1 may be at least one of the position, direction, speed, acceleration, time, time zone, weather, running state, and pedal work of the moving object. For example, an appropriate type of correction may be automatically performed for the tunnel position, or different image corrections may be applied for sunny, rainy, and foggy conditions. Also, different image corrections may be applied in the morning, at night, and in the evening, or an appropriate correction method may be selected and corrected according to the situation, weather, etc., such as fog, snow, and rain. Optimal parameters for each may be prepared in advance. More specifically, the buttons may be implemented as buttons that perform image correction according to the purpose of the image, such as an operation button to erase a snowstorm, a button to make fog transparent so that the other side of the fog can be seen, a button to erase rain, etc. Furthermore, as an all-purpose automatic correction, these may be appropriately combined.

According to this aspect, more appropriate image processing can be performed by taking into account at least one of the specific moving object conditions, namely position, speed, acceleration, running state, and pedal work, thereby further improving usability.

Next, the correction unit 233 may execute a first correction process on the image data IM generated by the imaging device 3100 of the moving object at the time corresponding to the specific information IF1 based on the specific information IF1 as a first correction step (activity A104). Specifically, the control unit 23 may determine the correction range for the image data IM based on the specific information IF1 by reading out a dedicated program stored in the storage unit 22. That is, preferably, the correction unit 233 may execute a first correction process on the image data IM as a first correction step, by varying the correction range, which is the range to be corrected, based on the specific information IF1. According to such an aspect, since it is not necessary to correct the entire range, the processing load can be suppressed. In addition, since the correction is applied to a preferred range, a function with high usability can be realized. Conversely, the correction unit 233 may execute a first correction process on the image data IM by varying the range that should not be corrected based on the specific information IF1. Furthermore, the first correction process may be performed appropriately with both the range to be corrected and the range not to be corrected being variable.

Then, as a display control step, the display control unit 235 may display the image data IM on which the first correction process has been performed in a reproducible manner (activity A105). It is preferable that such a display be reproducible as video data by displaying frames of the image data IM in a frame-by-frame manner.

During the display control by the display control unit 235, the reception unit 232 may receive a first input from the user as a first reception step. The control unit 33 may be configured to be able to switch between the presence and absence of correction processing on the image data IM based on the first input (activity A106). For example, when correction processing is being performed on the image data IM, the release unit 234 may release the first correction processing on the image data IM based on the first input as a release step. For example, when correction processing on the image data IM is released, the correction unit 233 may resume the first correction processing on the image data IM based on the first input. According to this aspect, when the correction processing presented by the device is unnecessary, it is possible to increase visibility by releasing it at the user's will, and usability can be further improved. For example, this can be achieved by providing a button to temporarily release the correction while leaving the correction applied to the range.

In summary, the information display terminal 3800, which is an example of an information processing device according to this embodiment, includes a control unit 23. The control unit 23 includes the following units. As a first correction step, the correction unit 233 executes a first correction process on image data IM generated by the imaging device 3100 possessed by the moving object at a time corresponding to the specific information IF1, based on the specific information IF1. As a display control step, the display control unit 235 displays the image data IM on which the first correction process has been executed in a reproducible manner. According to this aspect, appropriate image processing can be applied based on the state or situation of the moving object, thereby improving usability.

[Details of information processing]
Next, the details of the information processing outlined above will be described. Fig. 41 is a schematic diagram showing an example of a spatial correction range of the first correction processing. Fig. 42 is a schematic diagram showing an example of a temporal correction range of the first correction processing. That is, the first correction processing may be applied with a limited location range or time range.

In the embodiment of FIG. 41, image data IM is displayed by the display control unit 235, and this is shown as a playback screen 4. The correction range is a region of interest R set in the frame of the image data IM. The playback screen 4 has a correction region 41 and a non-correction region 42. The region of interest R is shown as the correction region 41. Specifically, it is preferable that the correction region 41 be determined according to various information included in the specific information IF1.

For example, the size of the correction area 41 may be changed according to the speed and acceleration of the moving object. For example, when the moving object turns left, a correction process may be executed so that the left side of the moving object is visualized, out of concern for preventing a hit-and-run accident. The shape and size of the correction area 41 are entirely arbitrary, and may be, for example, rectangular, circular, or elliptical. The shape may be configured to be changeable based on user input. According to such an embodiment, correction can be applied to a narrow area where visibility is originally desired to be improved, thereby improving usability.

In addition, for example, appropriate correction processing may be performed so that the position of the sun can be understood from the time and time period, and the user can easily see it even in backlit conditions. In the embodiment of FIG. 42, a correction time 51 is shown in a seek bar 5 that is located below the playback screen 4 and indicates the playback time of the image data IM. The seek bar 5 may correspond to a concept such as a timeline in video editing software. For example, the correction time 51 is a time period during which the setting sun shines in, and the shooting device 3100 shoots in backlight due to the traveling direction of the moving object. In other words, the specific information IF1 is the position and orientation of the moving object. As a first correction step, the correction unit 233 may perform a first correction processing on the image data IM based on the position of the sun estimated from the position and orientation. According to such an embodiment, appropriate image processing can be performed taking into account the position of the sun, thereby further improving usability.

The various correction processes described above may also be implemented as manual correction by the user, which is an example of a second correction process. In other words, the reception unit 232 may receive a second input by the user as a second reception step, and the correction unit 233 may execute a second correction process on the image data IM based on the second input as a second correction step. According to this aspect, in addition to the correction process presented by the device, it is possible to improve visibility by adding correction at the user's discretion, thereby further improving usability.

In addition, in relation to the first correction process and the second correction process, the display control unit 235 may display the image data IM in a reproducible manner in which the first correction process and the second correction process are distinguishable. In other words, the manually added range and the automatically added range may be displayed in a distinguishable manner. FIG. 43 is a schematic diagram showing additional information 43 added to the playback screen 4 when the first correction process and the second correction process are executed. As shown in FIG. 43, additional information 43 such as "Automatic correction in progress" or "Manual correction in progress" may be added to the upper right of the playback screen 4. Here, the additional information 43 is a text annotation, but an illustration or an icon may be used instead. According to such an embodiment, the user can understand the type of correction process, which is user-friendly.

The method of correction is not particularly limited, and at least one of brightness, dynamic range, contrast, gamma, saturation, hue, etc. may be appropriately corrected. It is also preferable that the correction is performed based on what is easily visible to the human eye.

Conversely, image data IM that is difficult for the user to see may be realized. For example, from the viewpoint of protecting personal information, a function may be realized that blurs the faces of an unspecified number of people when they are photographed. Mosaicking may be performed at least when recording, when a man is recognized, or when a woman is recognized. A function may also be realized that blurs the license plates of other vehicles. The targets to be blurred may be determined by artificial intelligence or other methods. Such a blurring process is an example of a correction process that makes an image dirty after it has been made clean. Mosaicking is an example of "making it difficult to see," and may be realized by other methods.

Furthermore, unlike correction by an ISP (imaging signal processor) that removes aberrations and noise due to optical elements and optical systems, this correction process is a further correction process performed on image data IM obtained through an ISP. Specifically, this may include color correction such as gamma correction, color emphasis, and color balance, and edge emphasis. Conversely, the original image data IM refers to image data IM obtained through correction by an ISP. This type of correction process may be performed after correction by the ISP and before encoding into an appropriate file format such as H264, or may be performed by correcting image data IM that has already been encoded and encoding it again.

[Modification 1]
Next, a first modification of the fourth embodiment will be described. In the first modification, the acquisition unit 231 may acquire various information other than the specific information IF1 from an external device in addition to the specific information IF1. In the first modification, for example, the specific information IF1 may be the position of a moving body. The acquisition unit 231 may acquire the road network information IF2 as an acquisition step, and the correction unit 233 may execute a first correction process on the image data IM based on the position and the road network information IF2 as a first correction step. Specifically, the control unit 23 may acquire the road network information IF2 by reading out a dedicated program stored in the storage unit 22, and further execute the first correction process using the road network information IF2.

Road network information IF2 has various information as attribute data, such as speed limits, road width, weekday 12-hour traffic volume, travel speed (for a certain section, the distance of the section divided by the time required to travel, the required time including stops due to traffic lights, congestion, etc.), and number of lanes. Based on this information, correction unit 233 may determine an appropriate type or range of correction and execute the first correction process. According to this aspect, appropriate image processing can be applied depending on the position and the road conditions at that time, thereby further improving usability.

[Modification 2]
Next, a second modification of the fourth embodiment will be described. In the second modification, similar to the first modification, the acquisition unit 231 may acquire various information other than the specific information IF1 from an external device in addition to the specific information IF1. In the second modification, for example, the specific information IF1 may be the position of the moving body. The acquisition unit 231 may acquire the topographical information IF3, and the correction unit 233 may execute a first correction process on the image data IM based on the position and the topographical information IF3 as a first correction step. Specifically, the control unit 23 may acquire the topographical information IF3 by reading out a dedicated program stored in the storage unit 22, and further execute the first correction process using the topographical information IF3.

The terrain information IF3 may be obtained, for example, from an external server related to an institution that issues map data. The terrain information IF3 may be information including information on the terrain of urban areas, suburban areas, expressways, general roads, gravel roads, mountains, inside tunnels, etc. The correction unit 233 may determine an appropriate type of correction or correction range based on this information and execute the first correction process. For example, in a downtown area, the influence of neon lights is strong, so a correction process may be executed to alleviate this. For example, in a forest, a correction process may be executed to emphasize the overall brightness. According to this aspect, appropriate image processing can be executed according to the position and the terrain and environment at that time, thereby further improving usability.

[Modification 3]
Next, a third modification of the fourth embodiment will be described. In the third modification, the correction unit 233 may execute a preparation step and a processing step. The control unit 23 may be configured to be able to call a first program P1 stored in advance and a second program P2 different from the first program P1. In particular, the first program P1 may be a player application recommended for playing image data IM, and the second program P2 may be a general-purpose player application having a correction processing function. In other words, since the first program P1, which is a standard viewer, becomes bloated if too many functions are incorporated, it can be substituted by free software such as VLC or image processing in a cloud service provided by various companies. FIG. 44 is an activity diagram showing the flow of information processing related to the third modification of the fourth embodiment. Below, this information processing will be outlined along this activity diagram.

First, when the reception unit 232 of the information display terminal 3800 receives an input from the user for playing a video, the control unit 23 may call up the first program P1 stored in the storage unit 22 (activity A201). Next, the control unit 23 may cause the first program P1 to read image data IM (activity A202). Next, as an operation based on the first program P1, the acquisition unit 231 may extract multiple parameters PT included in the image data IM and acquire them (activity A203).

If the second program P2 has not been installed in advance on the information display terminal 3800, the installation of the second program P2 may be suggested. In other words, if the second program P2 has not been installed, the display control unit 235 may display a screen recommending its installation. The recommendation screen preferably includes the name of the second program P2, and more preferably includes the URL of the installation destination. According to this aspect, even if the second program has not been installed, the first correction process can be performed via the second program.

Specifically, when the first program P1, which is a PC viewer, is installed, the user may be asked whether to install an external player, and if instructed to do so, the external player may be installed. During installation or immediately after startup, a button to start the external player may be displayed. Cautions when using the external player may be displayed. Cautions when using the external player may include the following: when a button on the external player is operated, there is a possibility that the display state of the first program P1, which is a PC viewer, may differ; in that case, it is recommended to close the second program P2, which is an external player, once and then start the second program P2, which is an external player, again from the first program P1, which is a PC viewer.

If the second program P2 is installed in advance in the information display terminal 3800, the control unit 23 may call up the second program P2 stored in the storage unit 22 (activity A205). That is, an external player start button or the like may be on the UI of the first program P1, which is the viewer, and pressing this button may start the second program P2, such as VLC. Then, as an operation based on the second program P2, the acquisition unit 231 may acquire a plurality of parameters PT contained in the image data IM (activity A206). Here, the parameters PT may include at least one of hue, brightness, saturation, contrast, gamma, sharpness, playback start time, playback end time, and the presence or absence of a loop. According to this aspect, since each parameter is a general parameter that image data has, a more general-purpose image processing program can be used as the second program, and a device with high usability can be realized. Furthermore, as an operation based on the second program P2, the acquisition unit 231 may acquire operation information IF4 related to the operation of the second program P2 (activity A207).

Specifically, instead of incorporating libvlc.dll or the like into the viewer, the file name of the video and video effect settings, etc. may be passed as "command line arguments" to start the viewer.

vlc. exe 20200101090000. avi --hue=0 --brightness=1.5 --contrast=0.6 --saturation=1.9 --gamma=0.3 --sharpen-sigma=0.9

Also, the playback start time, playback end time, whether or not to loop, etc. may be passed. A parameter for disabling at least a part of the operation section of the second program P2, which is an external player, may be passed. In addition, the second program P2, which is an external player, may be controlled by sending a control command from the first program P1, which is a viewer. For example, if the second program P2 is a VLC player, it may be controlled from the first program P1, which is a viewer, using http, telnet, etc.

In other words, the first program P1 may execute a preparatory step. Then, as a preparatory step, the correction unit 233 may extract parameters PT included in the image data IM and pass the values of the extracted parameters PT to the second program P2. Also, as a preparatory step, the correction unit 233 may pass operation information IF4 related to the operation of the second program P2 to the second program P2.

Then, as a processing step, the correction unit 233 may change the value of the parameter PT by executing a first correction process on the parameter PT (activity A208). Then, the display control unit 235 may display image data IM based on the value of the parameter PT changed in the processing unit in a reproducible manner. According to this aspect, a separate program that is more specialized for the correction process can be used, which contributes to reducing calculation costs and improving image processing performance.

Preferably, the display control unit 235 may also restrict at least some of the functions of the second program P2 based on the operation information IF4 to display the image data IM in a reproducible manner (activity A209). According to this aspect, the second program can be used solely for correction and reproduction, and the user does not need to use unnecessary functions of the second program, thereby realizing a device with high usability.

[Modification 4]
Next, a fourth modified example of the fourth embodiment will be described. In the third modified example described above, when the second program P2 is not installed, a screen recommending the installation of the second program P2 is displayed. In the fourth modified example, the second program P2 can be used via a network without being installed in the information display terminal 3800. In other words, the second program P2 may be a program stored in an external device. The control unit 23 may be configured to call the second program P2 via a network. According to this aspect, even when the second program is not installed, the first correction process can be performed via the second program in the form of a cloud service.

Specifically, the first program P1 should have a function for storing an ID or the like for automatically logging in to the cloud image sharpening service, an automatic sending function, an uploading function, a function for determining whether conversion is complete and then automatically downloading the image when conversion is complete or a function for notifying the user that downloading is possible, and a function for playing the downloaded image in sync with the image before conversion.

[Modification 5]
Next, a fifth modified example of the fourth embodiment will be described. The above-mentioned correction processes are various, and different correction processes may be performed in a superimposed manner. Different correction processes may be performed on one playback screen 4 at the same time, or different correction processes may be performed at different times. It is preferable that different correction processes are determined according to needs and situations.

In the former case, for example, when using an imaging device 3100 with a fisheye lens, the deck board of a moving object is often reflected near the center of the image data IM, and this portion is often unnecessary information. Therefore, by not performing correction processing on this central portion, the overall processing load can be reduced. Furthermore, since the information in uncorrected areas has poor visibility, some additional information 43 may be embedded in this position. For example, the original image data IM before correction may be embedded, or a recommended player, etc. may be presented.

In the latter case, for example, if a person rushes out from behind an object at the moment of a traffic accident, the focus should be on the person who gradually emerges from a dimly lit area, and correction processing should be performed so that the user can appropriately recognize the person. Different processing may be performed inside and outside the vehicle.

[Modification 6]
Next, a sixth modified example of the fourth embodiment will be described. A sharpening process may be performed to sharpen a part of an image captured by a photographing device using a fisheye lens (for example, one or both of the photographing devices 3100 and 3200). The sharpening process is an example of a correction process to correct a photographed image. In particular, a sharpening process may be performed to sharpen a part of an image captured by a photographing device using a circular fisheye lens (hereinafter referred to as a "circular fisheye image"). In this case, a processing pattern may be adopted in which the circular fisheye image is sharpened as it is and then a shape conversion (geometric conversion) is performed. In particular, the shape conversion may be a process of projecting the image onto a rectangle or the like, in other words, a process of converting the image into a planar image. In this way, the processing can be simplified in a configuration in which multiple locations are extracted from the circular fisheye image and multiple images are displayed on a rectangular screen. Alternatively, when a part of the circular fisheye image is extracted to display an image, and the image is displayed, the following ability to change the extracted area during playback can be improved. In addition, in cases where multiple extraction regions overlap (for example, multiple rectangular regions overlap, such as the shape of a "turn" or the shape of an "eye"), it is preferable to provide a function to reduce the sharpening process of the overlapping parts. In this way, the process can be simplified by the amount of overlapping sharpening process that is reduced.

In a processing pattern in which a portion of a circular fisheye image is extracted and then sharpened after shape transformation, it is advisable not to perform sharpening processing on areas other than the extracted area. In this way, the range of sharpening processing is limited, thereby reducing the processing load.

A portion of a circular fisheye image captured by a circular fisheye lens may be cut out, its shape may be transformed (for example, projected into a rectangle, etc.), and a sharpening process may be performed to sharpen that portion of the transformed image. This type of processing can be considered a double narrowing of the sharpening range.

It is preferable that the sharpening process is not performed on devices such as low-performance PCs. Whether a device such as a low-performance PC is a device can be switched automatically or manually by the user. Low performance means that the processing capacity of the device is below a certain specification, and in particular it is preferable that the sharpening process is not performed on devices such as PCs that are not equipped with a GPU.

It may also be equipped with a function to export the image after sharpening processing, and a function to upload the image for posting on social networking sites such as YouTube (registered trademark).

[Modification 7]
Next, a seventh modified example of the fourth embodiment will be described. In the above embodiment, the recorded image data IM is described as a moving object, but the image data IM may be recorded as a parking monitoring function of the above-mentioned image capturing device 3100. In such a case, the parking monitoring function of the image capturing device 3100 may execute a correction process different from that executed when the object is moving.

[Embodiment 5]
Next, a fifth embodiment will be described. The fifth embodiment may be implemented by the information processing system 3000 shown in Fig. 1. Fig. 45 is a functional block diagram realized by the control unit 33. The control unit 33 shown in Fig. 45 may be, for example, a control unit 3011 provided in the imaging device 3100 mounted on the vehicle 3400. The storage unit 32 shown in Fig. 45 may be, for example, a memory 3112 provided in the imaging device 3100 or the like.

The memory unit 32 stores various information defined by the above description. This may be implemented, for example, as a storage device such as a solid state drive (SSD) that stores various programs related to the imaging device 3100 executed by the control unit 33, or as a memory such as a random access memory (RAM) that stores temporarily required information (arguments, arrays, etc.) related to the program calculations. It may also be a combination of these. In particular, the memory unit 32 may store the third program P3 and the fourth program P4 as various programs related to the imaging device 3100 executed by the control unit 33.

The control unit 33 may function as an acquisition unit 331, a reception unit 332, a recording unit 333, a correction unit 334, and a display control unit 335 by executing various programs stored in the memory unit 32. In other words, information processing by the software stored in the memory unit 32 may be specifically realized by the control unit 33, which is an example of hardware, and may be executed as each functional unit included in the control unit 33. Each functional unit related to the control unit 33 will be described below.

The acquisition unit 331 is configured to acquire various types of information.

The reception unit 332 is configured to receive various types of information.

The recording unit 333 is configured to record data on a storage medium. For example, the recording unit 333 records video captured by a camera as image data IM. This will be described in more detail later.

The correction unit 334 is configured to perform the correction process.

The display control unit 335 is configured to control various visual information to be displayed on a display unit capable of displaying the same. Here, visual information may be information itself that is generated in a manner that is visible to the user, such as a screen, an image, an icon, text, etc., or may be rendering information for displaying a screen, an image, an icon, text, etc. For example, the display control unit 335 may display an image captured by a camera as a live view. This will be described in more detail later.

Next, an information processing method of the information processing system 3000 according to the fifth embodiment will be described. In the fifth embodiment, the control unit 33 may be configured to be able to call up a pre-stored third program P3 and a fourth program P4 different from the third program P3. In particular, the third program P3 may be a player application recommended for playing back image data IM, and the fourth program P4 may be a general-purpose player application. Figure 46 is an activity diagram showing the flow of information processing according to the fifth embodiment. Below, this information processing will be outlined with reference to this activity diagram.

While the moving object is traveling, the following activities A301 to A305 are sequentially repeated. First, while the moving object is traveling, the recording unit 333 may record the video captured by the image capturing unit 3015 in the imaging device 3100 as image data IM (activity A301). Specifically, the control unit 33 may read out a dedicated program stored in the memory unit 32, thereby sequentially storing the image data IM in the memory unit 32.

The image data IM may be, for example, video-format data having a predetermined frame rate. The frame rate of the image capture device 3100 is not particularly limited, but may be, for example, 10 fps or more, preferably 30 fps or more, and more preferably 60 fps or more, and specifically may be, for example, 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190, 200, 210, 220, 230, 240, or 250 fps, or may be within a range between any two of the numerical values exemplified here.

Next, the processing of activities A302 and A303 for displaying the image data IM on the display unit 3013 of the imaging device 3100, and the processing of activities A304 and A305 for later playing back the image data IM on the information display terminal 3800 are executed in parallel.

Specifically, the image data IM is recorded, and a first correction process is applied to each frame (activity A302), and the image is displayed as a live view (activity A303). In other words, the display control unit 335 displays the image captured by the camera as a live view. Here, the values of the parameter PT for the recorded image data IM and the displayed image may be set to be different.

Figure 47 is a schematic diagram showing aspects of image data IM displayed as a live view and image data IM recorded so that it can be played back later. As shown in Figure 47, when the recording unit 333 records the image data IM, the recording unit 333 records the image as image data IM without correcting the image, while the display control unit 335 corrects the image and displays it as a live view. According to this aspect, it is possible to prevent information loss due to processing of the image data to be recorded while improving the visibility of the image viewed as a live view.

On the other hand, when the image data IM displayed as a live view is recorded, the acquisition unit 331 may extract and acquire various parameters PT contained in the image data IM (activity A304). Then, the recording unit 333 may record the parameters PT of the displayed image as a parameter log PL in chronological order together with the image data IM (activity A305). According to this aspect, it is possible to preserve the visibility of the image viewed as a live view.

After the moving object has completed its travel, the image data IM may be played back via the information display terminal 3800 connected to the imaging device 3100 via a network. When playing back the image data IM on the information display terminal 3800, the reception unit 332 of the information display terminal 3800 receives a third input from the user for playing back the image data IM as a third reception step (activity A306). Next, the third input may trigger the control unit 23 to read out the image data IM (activity A307).

Next, if the third program P3 is installed in the information display terminal 3800, the control unit 23 may read the parameter log PL as an operation based on the third program P3 (activity A308). Next, as an operation based on the third program P3, the correction unit 233 may refer to the values of the parameter log PL and execute a first correction process to appropriately change the values of each parameter of the original image data IM based on the values (activity A309). Thereafter, as an operation based on the third program P3, the corrected image data IM may be displayed by the display control unit 235 (activity A311).

In other words, the display control unit 335 may read the image data IM and the parameter log PL based on the third input, and display the corrected image data IM in a reproducible manner. According to this aspect, when the user wishes, a highly visible image can be reproduced on the spot, and a device with high usability can be realized.

On the other hand, if the third program P3 is not installed in the information display terminal 3800, the control unit 23 may generate additional information 43 recommending playback in the third program P3 as an operation based on the fourth program P4 (activity A310). Thereafter, as an operation based on the fourth program P4, the display control unit 235 may display the additional information 43 superimposed on the image data IM in the recorded state (activity A311). In other words, when the image data IM is played in the fourth program P4, the display control unit 335 may display information encouraging playback in the third program P3. According to this aspect, even if the image data IM is played in an unintended program, playback in a preferred program can be induced, and appropriate information can be provided to the user.

Figure 48 is a schematic diagram showing an aspect in which additional information 43 encouraging playback of the recommended third program P3 is superimposed on the playback screen 4. As shown in Figure 48, the information encouraging playback may be displayed together with the playback screen of the image data IM. In this manner, it is possible to provide the user with appropriate information that the current appearance is undesirable, but that there is no problem with the image data itself.

In addition, a QR code (registered trademark) encoding the URL of the third program P3 may be embedded and recorded along with the character string, or an image of a screen containing this information may be recorded at the beginning of the playback video file.

In summary, the imaging device 3100, which is an example of an information processing device according to this embodiment, includes a control unit 33. The control unit 33 includes the following units: The recording unit 333 records the image captured by the camera as image data IM. The display control unit 335 displays the image captured by the camera as a live view. Here, the parameters PT of the recorded image data IM and the displayed image are set to be different. According to this aspect, by making the image viewed as a live view different from the image data to be recorded, it is possible to ensure both visibility and customizability.

In other words, various parameters PT related to image quality, such as the parameters of the image sensor and ISP of the imaging device 3100 such as a drive recorder, can be recorded in association with the video data of the drive recorder, and playback can be performed using these parameters when playing back on the viewer or main unit.

[Modification 1]
Next, a first modification of the fifth embodiment will be described. In the first modification, the image data IM to be recorded is, unlike the fifth embodiment, image data IM that has been subjected to a correction process and has high visibility, substantially similar to that used for live view. In contrast, if it is desired to reproduce the original image data IM before correction later, the image data IM may be generated by applying a correction process to the image data IM using an information display terminal 3800 or the like. The correction process is preferably an accurate inverse conversion process, but a correction process that is assumed to be an inverse conversion may also be applied.

[Modification 2]
Next, a second modification of the fifth embodiment will be described. In this second modification, two types of parameters PT are recorded. The parameters PT include parameters that are constant throughout the entire time series of image data IM, and parameters that change on a frame-by-frame basis. The former are recorded on an image data IM basis, and the latter are recorded on a frame-by-frame basis of image data IM, thereby making it possible to reduce the processing load and perform correction processing efficiently.

In other words, the parameters PT are recorded separately as static parameters that do not change during shooting, and dynamic parameters that change depending on the video conditions during shooting, etc. For example, one static parameter is recorded for each file, and one dynamic parameter is recorded at least at the change point, preferably across multiple frames such as all frames, and the viewer can record these at different times,
Each may be read and processed separately.

[others]
A program that causes a computer to function as a control unit of an information processing device, for example, the control unit 33 in the imaging device 3100 or the control unit 23 in the information display terminal 3800, may be provided.

Since there is a problem that a mechanism is required to switch which of different algorithms such as fog, HDR, and sharpening is to be applied to the currently played video, the following aspect may be adopted for the correction process.
1. Obtain the position and direction of the sun from the GPS and change or modify the correction content used to correct the image.
2. Based on GPS location information and map information, the system determines whether the vehicle is in a city, a forest, etc., and changes the applicable algorithm.
3. If fog lamp on action is detected, apply fog algorithm.
4. If the contrast difference is small and the image is white, it is determined to be foggy and a fog removal algorithm is applied.
5. When there is a lot of black, it is judged as darkness and made brighter.
6. CLAHE, gamma, and contrast are varied as a set.

The system may be equipped with a function for writing out images recorded by an imaging device 3100 such as a drive recorder, after image quality correction has been applied by the imaging device 3100, to a file separate from the recorded image.

The drive recorder may also have a function to export video with image quality correction by performing a specified operation while playing the video on the drive recorder itself.

It is a good idea to continuously notify the driver not to turn off the engine while the image with image quality correction is being exported, and it is especially recommended to repeat this message by voice.

The viewer may be provided with an operation unit, such as a radio button, check box, pull-down menu, slider, etc., for adjusting image quality parameters that can be operated by the user, and may display the image quality adjustment results according to the operation of the operation unit while playing back the recorded video. When a fixation instruction is given to fix the operation unit in the displayed state, such as a check box that gives an instruction to fix, the operation of the operation unit may be prohibited from being changed, and the image quality adjustment results of the fixed operation unit may be displayed.

The viewer may be provided with an operation unit that allows the user to adjust image quality parameters, such as radio buttons, check boxes, pull-down menus, sliders, etc., and may have a function for recording the user's operation of the operation unit in synchronization with the time of the video. The viewer may also have a function for adjusting image quality according to the recorded user operation of the operation unit and playing it back.

The imaging device 3100 such as a drive recorder or the information display terminal 3800 on which the first program P1 serving as a viewer is installed may be provided with a function for writing out images to be posted with the location information deleted, and a function for writing out images to be posted without deleting the location information.

A selection function may be provided that displays the corrected video of the recorded video as multiple streams with different correction states, and allows the user to select the stream to be exported or played back from the multiple streams displayed on the display screen. For example, multiple candidate video windows may be opened and played back in chronological synchronization with the original video window, and the user may select the best one from among them using a selection button, etc.

The relationship between the user's selection result or selection state and the information of the original video at that time may be used to determine image quality adjustment parameters, etc., when playing another video. Here, the relationship may be recorded, or a learning model may be trained and recorded.

When exporting video, a function is provided for selecting the geographical range into which the video is to be exported using a rectangle or the like, and image quality correction is performed only within the selected range before re-encoding and exporting.

For example, it would be good to have an export function for posting to SNS such as YouTube, and a function for storing the original video and the processed video in a single file and uploading it. Both videos may be arranged side-by-side as a stream, or the two videos may be arranged one after the other in terms of time as a video stream. In this case, it would be good to embed a display indicating that each video is the "original video" and the "processed video" before exporting them.

For images captured by spherical or hemispherical cameras, it is advisable to unfold the image, convert the coordinates of a portion of it, and then perform processing on the displayed image. This can reduce the processing load. Alternatively, for example, processing can be performed on a doughnut-shaped portion including the outermost periphery, while no processing or a different processing can be performed on the hole in the doughnut, the circular area inside.

Furthermore, it can also be provided in the following form:

In the information processing device, the screen includes an object, and the control unit receives an operation instruction for the object selected by a user, receives an instruction to display a processing result related to the image corresponding to the operation instruction, and controls the information processing device to display a screen including the processing result related to the image.

According to this aspect, any image processing corresponding to the manipulation of the object can be executed at the user's choice.

In the information processing device, the screen includes objects, and the objects differ for different processing results related to the image, and the control unit receives an instruction to display the processing results related to the image that correspond to the object selected by the user, and controls the information processing device to display a screen including the processing results related to the image.

According to this aspect, any image processing corresponding to the object can be performed at the user's choice.

In the information processing device, when a first object and a second object are selected via the screen, the control unit receives an instruction to display a first processing result for the image corresponding to the first object and an instruction to display a second processing result for the image corresponding to the second object, and controls the information processing device to display a screen including the first processing result for the image and the second processing result for the image.

According to this aspect, multiple arbitrary image processes can be executed simultaneously.

In the information processing device, the image processing related to the first object and the image processing related to the second object are performed on different areas of the image that are the subject of image processing.

According to this aspect, multiple arbitrary image processes can be performed simultaneously on different areas.

In the information processing device, the object displayed on the screen is an object related to image processing related to weather.

This allows for appropriate image processing to be performed according to the weather.

In the information processing device, the control unit changes the object displayed on the screen based on the state of the image.

This aspect allows appropriate image processing to be performed based on the information within the image.

In the information processing device, the control unit controls the display of an object on the screen that suggests image processing for the image based on road network information.

According to this aspect, it is possible to perform appropriate image processing based on road network information.

In the information processing device, the control unit controls the display of an object on the screen that suggests image processing for the image based on information from an acceleration sensor that detects the acceleration of a moving body having the image capture unit.

This aspect makes it possible to perform appropriate image processing based on information from the acceleration sensor of the moving object.

In the information processing device, the control unit controls the display of an object on the screen that suggests image processing for the image based on position information of a moving object having the image capture unit.

This aspect makes it possible to perform appropriate image processing based on the position information of the moving object.

In the information processing device, the control unit controls the display of an object on the screen that suggests image processing for the image based on location information of a moving object having the image capture unit and weather information for the date and time the image was captured.

According to this aspect, it is possible to perform appropriate image processing based on the location information of the moving body and weather information for the date and time at that location.

In the information processing device, the control unit controls the display of an object on the screen that suggests image processing for the image based on operation information of a moving object having the image capture unit.

According to this aspect, it is possible to perform appropriate image processing based on the operation information of the moving object.

In the information processing device, the control unit controls the display of an object on the screen that suggests image processing for the image based on the date and time the image was captured.

This aspect allows appropriate image processing to be performed based on the date and time the image was captured.

In the information processing device, the image processing for the image is image processing for a portion of the image captured by the image capture unit.

This aspect makes it possible to perform image processing on a portion of the captured image.

In the information processing device, the part of the image is a set area of the entire area of the image captured by the image capture unit.

According to this aspect, it is possible to set the area in which image processing is performed.

In the information processing device, the part of the image is an image captured by the image capture unit during a set time period.

This aspect makes it possible to perform appropriate image processing on a portion of an image captured during a set time period.

In the information processing device, the part of the image is an image captured within a range in which the position information of the moving body having the image capturing unit is set.

This aspect makes it possible to perform appropriate image processing on a partial area of an image captured by a moving object moving within a set position.

In the information processing device, the part of the image is an image showing the weather, the weather information of the date and time when the image was taken at the location of the moving body having the image capture unit is set.

According to this aspect, it is possible to perform appropriate image processing on a portion of the captured image based on the weather at the location of the moving body.

In the information processing device, the part of the image is an image in which sensor information of a moving body having the image capture unit is within a set threshold.

According to this aspect, it is possible to perform appropriate image processing on a portion of an image captured based on sensor information from a moving object.

In the information processing device, the screen includes information on whether the range in which the image processing is to be performed has been selected automatically or manually.

According to this aspect, it is possible to identify whether the scope of the image processing performed was set automatically or selected manually.

In the information processing device, the screen includes an image captured by the image capture unit and an image after one or more image processes.

This allows you to display a comparison of the image before and after processing.

In the information processing device, the control unit adjusts image processing parameters based on learning information regarding the selection of the one or more images after image processing.

According to this aspect, the selected image processing results can be learned, enabling subsequent image processing to be performed more appropriately.

In the information processing device, the screen includes an image after image processing, and the control unit stores the image captured by the image capture unit in a storage area.

According to this embodiment, the original image can be stored while the processed image is displayed on the screen.

In the information processing device, the screen includes information about what kind of image processing has been performed on the image captured by the imaging unit.

This aspect allows you to check what kind of image processing has been performed.

In the information processing device, the control unit stores information about what kind of image processing was performed on the image captured by the imaging unit in a storage area as log information.

This aspect allows you to check what kind of image processing has been performed.

In the information processing device, the control unit converts the image after image processing to an image before image processing based on the log information and outputs the image to the screen.

According to this aspect, the image after image processing can be restored to the original image before image processing.

In the information processing device, the control unit changes the objects included on the screen in accordance with the object detected from the image.

According to this aspect, the image processing method can be changed based on the object in the image.

In the information processing device, the screen includes an adjustment object that adjusts the degree of the image processing, and the control unit adjusts the degree of the image processing in response to an operation of the adjustment object.

This aspect allows you to adjust the degree of image processing, such as brightness and darkness.

In the information processing device, the control unit stores the timing of the operation of the adjustment object and the timing of recording the image or the timing of playing back the image in association with each other.

This aspect makes it possible to know at what timing the degree of image processing was adjusted.

In the information processing device, the screen includes a fixed instruction object that prohibits adjustment by the adjustment object, and the control unit prohibits adjustment by the adjustment object in response to an operation of the fixed instruction object.

In this manner, the degree of image processing can be fixed.

In the information processing device, the control unit adjusts the degree of image processing based on learning information about the past degree of adjustment of the image processing.

According to this aspect, the results of the adjustment of the degree of image processing can be learned, and subsequent image processing can be performed more appropriately.

In the information processing device, the screen includes an image output object, and the control unit deletes the position information of the image and outputs the image in response to an operation of the image output object.

According to this embodiment, it is possible to perform an operation to delete the location information associated with the image.

In the information processing device, if the processed image contains information that can identify an individual, the control unit requests that a mask process be performed on the information that can identify an individual.

This aspect allows image processing to be performed appropriately so that personally identifiable information is not included.

In the information processing device, the control unit executes image processing based on operation instructions via the screen.

According to this aspect, the information processing device can perform image processing.

An image processing device capable of communicating with the information processing device, the image processing device having a control unit, the control unit performing image processing based on a request from the information processing device.

A program for causing a computer to function as a control unit of the information processing device.

Of course, this is not limited to these.

The scope of the present invention is not limited to the configurations explicitly described in the specification, but includes combinations of various aspects of the present invention disclosed in this specification. The configurations of the present invention that are sought to be patented are specified in the attached claims, but it is our intention to include configurations disclosed in this specification in the claims in the future, even if they are not currently specified in the claims.

The present invention is not limited to the configurations described in the above-mentioned embodiments. The components of each of the above-mentioned embodiments and variations may be arbitrarily selected and combined. Any components of each of the embodiments and variations may be arbitrarily combined with any components described in the means for solving the invention or any components that embody any components described in the means for solving the invention. We intend to acquire rights to these as well through amendments to this application or divisional applications, etc. Even if there is a description such as "in the case of" or "when", this is not intended to describe a configuration that is limited to that case or time. We also disclose configurations that are not in these cases or times, and we intend to acquire rights to them. Furthermore, the parts that are described in order are not limited to this order. We also disclose configurations in which some parts are deleted or the order is changed, and we intend to acquire rights to them.

In addition, by converting to a design registration application, it is intended to obtain rights to the overall design or partial design. The drawings show the entire device in solid lines, but they include not only the overall design but also partial designs claimed for a portion of the device. For example, it is not only a drawing that includes a portion of the device as a partial design, but also a drawing that includes a portion of the device as a partial design regardless of the portion. A portion of the device may be a portion of the device, or a portion of that portion. It is intended to obtain rights to not only the overall design, but also partial designs in which any portion of the solid line portion of the drawing is shown as a broken line portion. In addition, the modules, materials, parts, etc. inside the device's casing, all of which are shown in the drawings, are subject to trade independently, and it is intended to obtain rights to them by converting to a design registration application in the same way.

11: Control unit 22: Memory unit 23: Control unit 231: Acquisition unit 232: Acceptance unit 233: Correction unit 234: Release unit 235: Display control unit 32: Memory unit 33: Control unit 331: Acquisition unit 332: Acceptance unit 333: Recording unit 334: Correction unit 335: Display control unit 4: Playback screen 41: Correction area 42: Non-correction area 43: Additional information 5: Seek bar 51: Correction time IF1: Specific information IF2: Road network information IF3: Topography information IF4: Operation information IM: Image data P1: First program P2: Second program P3: Third program P4: Fourth program PL: Parameter log PT: Parameter R : Area of interest 3000: Information processing system 3011: Control unit 3013: Display unit 3014: Audio output unit 3015: Photographing unit 3111: Processor 3100: Photographing device 3200: Photographing device 3400: Vehicle 3500: Recording medium 3700: Server 3710: Control unit 3711: Image processing unit 3712: Communication control unit 3713: Storage control unit 3720: Storage unit 3730: Communication unit 3800: Information display terminal 3810: Control unit 3811: Display control unit 3812: Communication control unit 3813: Storage control unit 3820: Storage unit 3830: Communication unit 3840: Input unit 3850: Output unit 3900: Network 4100: Playback window 4101: File button 41 02: Display button 4103: Play button 4104: Tool button 4105: Information button 4110: Video display area 4120: Acceleration graph display area 4130: Map image display area 4131: Current location icon 4140: Fog correction button 4141: HDR button 4142: Sharpness button 4143: Other processing button 4144: Partial processing button 4145: Stream display button 4146: Operation button 4150: Date and time display area 4160: Vehicle status display area 4170: Speed graph display area 4180: Playlist display area 4200: Playback window 4210: Video display area 4240: Fog correction button 4300: Playback window 4310: Video display area 4311: Fog correction area 4312: Processing contents display area 4313: Sharpening area 4314: Processing contents display area 4340: Fog correction button 4342: Sharpening button 4344: Partial processing button 4400: Playback window 4440: Sunny correction button 4441: Cloud correction button 4442: Rain correction button 4443: Fog correction button 4500: Playback window 4510: Video display area 4540: Fog correction button 4600: Playback window 4610: Video display area 4630: Map image display area 4631: Current location icon 4640: Tunnel correction button 4700: Playback window 4720: Acceleration graph display area 4721: Acceleration display area 4740: Acceleration correction button 4770: Speed graph display area 4800: Playback window 4810: Video display area 4830: Map image display area 4831: Current location icon 4832: Processing area 4840: Fog correction button 4900: Playback window 4910: Video display area 4911: Weather information 4941: Rain correction button 5000: Playback window 5010: Video display area 5030: Map image display area 5031: Current location icon 5040: Brake correction button 5070: Speed graph display area 5100: Playback window 5110: Video display area 5140: Night correction button 5150: Date and time display area 5200: Playback window 5210: Video display area 5211: Fog correction area 5240: Fog correction button 5300: Playback window 5310: Video display area 5311: Night correction area 5340: Night correction button 5350: Date and time display area 5400: Playback window 5410: Video display area 5411: Fog correction area 5430: Map image display area 5431: Current location icon 5432: Processing area 5440: Fog correction button 5500: Playback window 5510: Video display area 5511: Fog correction area 5512: Weather information 5540: Fog correction button 5541: Rain correction button 5600: Playback window 5610: Video display area 5611: Acceleration correction area 5620: Acceleration graph table display area 5621: acceleration display section 5640: acceleration correction button 5700: playback window 5710: video display area 5711: automatic fog correction area 5712: processing content display section 5713: manual fog correction area 5714: processing content display section 5740: fog correction button 5745: stream display button 5800: playback window 5810: video display area 5811: original image display section 5812: original display section 5813: processing 1 result display section 5814: processing 1 content display section 5815: processing 2 result display section 5816: processing 2 content display section 5817: processing 3 result display section 5818: processing 3 content display section 5846: operation button 5900: playback window 59 10: Video display area 5911: Processing content display section 5940: Fog correction button 6000: Playback window 6080: Playlist display area 6081: Playlist 6100: Playback window 6110: Video display area 6111: Detection information display section 6140: Fog correction button 6200: Playback window 6290: Image processing adjustment area 6291: Hue adjustment object 6292: Brightness adjustment object 6293: Contrast adjustment object 6294: Saturation adjustment object 6295: Gamma value adjustment object 6296: Sigma value adjustment object 6297: Radius adjustment object 6298: Variation amount adjustment object 6300: Playback window 6390: Image processing adjustment area 6391: Image adjustment Fixed object 6392: Sharpness adjustment Fixed object 6393: Banding noise removal adjustment Fixed object 6394: Film grain adjustment Fixed object 6400: Playback window 6401: File button 6410: Video display area 6500: Playback window 6510: Video display area 6511: Other vehicles 6512: License plate information 6540: Personal information processing button 6541: HDR button 6600: Information processing system 6800: Information processing terminal 6811: Display control unit 6812: Communication control unit 6813: Storage control unit 6814: Image processing unit

Claims (4)

A system comprising a controller,
The control unit is configured to execute the following steps:
In the recording step, the image captured by the camera is recorded as image data.
In the display control step, the image captured by the camera is displayed as a live view.
The parameters of the recorded image data and the displayed image are set to be different from each other ;
In the recording step, the image is recorded as the image data without correction;
In the display control step, the image is corrected and displayed as the live view ,
In the recording step, parameters of the displayed image are recorded in chronological order together with the image data as a parameter log. 2. The system of claim 1 ,
The control unit is configured to further execute a receiving step,
In the receiving step, an input by a user is received,
The display control step includes reading out the image data and the parameter log based on the input , and displaying the corrected image data in a reproducible manner. 1. An information processing method, comprising:
A method comprising the steps of the system according to claim 1 or 2 . A program,
A computer that functions as a control unit of the system of claim 1 or claim 2 .