JP7713705B2 - Terminals, photographic equipment, programs, etc. - Google Patents

Description

The present invention relates to terminals, photographing equipment, programs, etc.

JP 2020-173844 A (Patent Document 1) is a background technology in this technical field. This publication states that "the device includes an observation means for observing the situation in which a moving object operated by a user is placed, and a control means for controlling the sending of a message to the user to switch the psychological state that the user would have to another psychological state when the observation result by the observation means satisfies a predetermined condition" (see abstract).

JP 2020-173844 A

The above-mentioned Patent Document 1 describes a device having a function of monitoring the rear of a vehicle from an image and detecting tailgating by a following vehicle, and a function of sending a message to a driver when tailgating by a following vehicle is detected. However, the above-mentioned Patent Document 1 does not consider a mechanism for a terminal or the like that stores image information transmitted from a photographing device when the state of the vehicle satisfies a specific recording condition.
Therefore, the present invention provides a function that is highly convenient for users, for example, a mechanism for a terminal or the like that stores image information transmitted from an imaging device when the vehicle condition satisfies specific recording conditions.

In order to solve the above problems, for example, the configurations described in the claims are adopted.
The present application includes multiple means for solving the above-mentioned problems, and one example is characterized in that a terminal has a communication control means for receiving multiple pieces of image information transmitted from a photographing device installed in a vehicle, and an image information recording means for storing the multiple pieces of image information received from the photographing device in a storage means for the terminal when the state of the vehicle satisfies a predetermined recording condition.

The present invention can provide a function that is highly convenient for users. For example, it can provide a mechanism for a terminal or the like that stores multiple pieces of image information transmitted from a photographing device when the state of the vehicle meets specific recording conditions. Problems, configurations, and effects other than those described above will become clear from the description of the embodiments below.

FIG. 1 is an explanatory diagram illustrating an example of the installation position of a photographing device 101 and a user terminal 102 in a vehicle 10 according to the present embodiment. FIG. 2 is an example of a connection diagram of the photographing device 101, the user terminal 102, and the external server 103. FIG. 3 shows an example of the hardware configuration of the user terminal 102. FIG. 4 shows an example of the hardware configuration of the photographing device 101. As shown in FIG. FIG. 5 shows an example of the hardware configuration of the external server 103 . FIG. 6 shows an example of a first flow 60 of image recording, event notification information output, and index information addition by the photographing device 101. FIG. 7 shows an example of a second flow 70 of image recording, event notification information output, and index information addition by the photographing device 101. FIG. 8 shows an example of a flow 80 of recording image information of the photographing device 101 by the user terminal 102. FIG. 9 shows an example of a setting flow 90 for the photographing device 101 by the user terminal 102. FIG. 10 shows an example of a flow 100 for stopping the display of image information by the user terminal 102. FIG. 11 is an example of a screen showing some of the operations for setting reference lines 1101 and 1102 for event determination. FIG. 12 is an example of a screen showing a part of the operation for setting reference lines 1201 and 1202 for event determination, which follows the example of the operation shown in FIG. FIG. 13 is an example showing appropriately set reference lines 1301, 1302, and D along with imaginary decision lines A, B, and C. FIG. 14 shows an example of switching between display layouts 1420 and 1430 in accordance with the orientation of the screen of the user terminal 102. In FIG. FIG. 15 shows an example of a preview screen 1500 of the user terminal 102 . FIG. 16 shows an example of an operation for displaying setting screens 1601 and 1602 of the photographing device 101 from the preview screen. FIG. 17 shows an example of setting menus 1702 to 1704 that are displayed when the "Smartphone event record setting" menu shown in FIG. 16 is selected. FIG. 18 is an example of a screen 1800 on which events are displayed. FIG. 19 shows an example of a setting screen 1900 of the user terminal 102 . FIG. 20 shows an example of a setting screen 2000 for two types of screen settings. FIG. 21 shows an example of a camera image display time selection screen 2100. FIG. 22 shows an example of a power saving screen transition time selection screen 2200. FIG. 23 shows examples of screen displays 2301 and 2302 before and after the camera image display time has elapsed. FIG. 24 shows examples of screen displays 2401 and 2402 before and after the power-saving screen transition time has elapsed. FIG. 25 shows an example of selection screens 2502 to 2504 for events to be warned about. FIG. 26 shows an example of the response of the photographing device 101 and the indicator display when rear approach is detected. FIG. 27 shows an example of the response of the photographing device 101 and the indicator display when rear approach and abnormal rear approach are detected. FIG. 28 shows an example of a selection screen 2800 for a terminal storage unit. FIG. 29 shows an example of a selection screen 2900 for selecting the storage time of the image information received from the photographing device 101. FIG. 30 shows examples of display screens 3001 and 3002 in which image information received from the photographing device 101 is being stored. FIG. 31 is an example of a settings screen 3100 for event storage selection and event recording settings. FIG. 32 shows an example of a notification screen 3200 regarding a shortage of capacity of the auxiliary storage device 302 . FIG. 33 shows an example of a notification screen 3300 for notifying that the photographing device 101 is accepting an operation via an operation element provided on the photographing device 101 . FIG. 34 is a diagram showing two examples of the external appearance of the photographing device 101 (A) and (B). FIG. 35 is a diagram showing another example of the appearance of the photographing device 101. In FIG.

Below, a description will be given of a mode for carrying out the present invention with reference to the drawings. Note that the embodiment shown below is one embodiment of the present invention, and the contents of the present invention should not be interpreted as being limited based on the following description.

FIG. 1 is an example of an explanatory diagram illustrating the installation positions of photographing devices 101 (101F, 101R) and a user terminal (terminal) 102 in a vehicle 10 (hereinafter, also referred to as the host vehicle 10) according to this embodiment.
FIG. 1( a ) is a side view of a vehicle 10 , and FIG.

1(a) and 1(b), the vehicle 10 has an image capturing device 101F at the front of the vehicle interior and an image capturing device 101R at the rear of the vehicle interior. The image capturing device 101 is an electronic device having an image capturing function, and may be, for example, a drive recorder, a front camera, a rear camera, or a side camera that cooperates with the drive recorder, or a camera provided for crime prevention, surveillance, or other sensing purposes.
The vehicle 10 may be configured to include only the photographing device 101F for the front of the vehicle 10 or the photographing device 101R for the rear of the vehicle 10. Alternatively, the vehicle 10 may include three or more photographing devices. The photographing devices 101F and 101R may be simply referred to as the photographing devices 101, regardless of their installation positions.

1 is, for example, a drive recorder, and is an imaging device arranged on the front side of the vehicle 10. In this example, the imaging device (drive recorder) 101F is arranged on the front side inside the vehicle 10 (i.e., the passenger compartment) and attached to a predetermined portion such as the windshield. The imaging device (drive recorder) 101F images the outside of the vehicle 10 in front.
The photographing device 101 (101R) is, for example, a rear camera, and is a photographing device arranged on the rear side of the vehicle 10. In this example, the photographing device (rear camera) 101R is arranged on the rear side inside the vehicle 10 and attached to a predetermined portion such as the rear window. The photographing device (rear camera) 101R photographs the outside rear of the vehicle 10.

The photographing device 101F and the photographing device 101R are connected via a cable 11. The cable 11 is a cable for electrically connecting the photographing device 101F and the photographing device 101R. The cable 11 includes, for example, a power line for supplying power from the photographing device 101F to the photographing device 101R, and a signal line for transmitting various signals. The photographing device 101R operates by receiving power supply via the cable 11.
The photographing device 101R has a function of photographing an image and a function of outputting image data representing the photographed image to the photographing device 101F via the cable 11. The photographing device 101F has a function of photographing an image, a function of recording image data representing the photographed image, and a function of recording image data received from the photographing device 101R.
As described later, the functions, configurations, etc. of the photographing devices 101F and 101R in this embodiment are not limited to the above example, and the total number of photographing devices 101 may be one or three or more.

The user terminal 102 may be, for example, a portable terminal (mobile terminal) such as a smartphone, tablet, mobile phone, or personal digital assistant (PDA), or may be a wearable terminal such as glasses, a wristwatch, or clothing. It may also be a stationary or portable computer, or a server located on the cloud or a network. In terms of functionality, it may be a VR (Virtual Reality) terminal, an AR (Augmented Reality) terminal, or an MR (Mixed Reality) terminal. Alternatively, it may be a combination of multiple of these terminals. For example, a combination of one smartphone and one wearable terminal may logically function as one terminal. It may also be an information processing terminal other than these.

The user terminal 102 comprises a processor 303 that executes an operating system, application programs, etc., a main memory device 301 such as a RAM (Random Access Memory), an auxiliary memory device 302 such as an IC card, a hard disk drive, an SSD (Solid State Drive), or a flash memory (SD card, etc.), a communication control unit 307 such as a network card, a wireless communication module, or a mobile communication module, an input device 304 such as a touch panel, a keyboard, a mouse, voice input, or input by motion detection using images captured by a camera unit, and an output device 305 such as a monitor, display, or speaker. The output device 305 may be a device or terminal that transmits information to be output to an external monitor, display, speaker, printer, device, etc.

The main memory device 301 stores various programs, applications, etc. (modules), and the processor executes these programs and applications to realize each functional element of the overall system. Each of these modules may be implemented in hardware, for example by integration. Each module may be an independent program or application, or may be implemented as a subprogram or function within a single integrated program or application.

In this specification, each module is described as an entity (subject) that performs processing, but in reality, a processor that processes various programs, applications, etc. (modules) executes the processing.
Various databases (DB) are stored in the auxiliary storage device. A "database" is a functional element (storage unit) that stores a data set so that it can handle any data operation (e.g., extraction, addition, deletion, overwriting, etc.) from a processor or an external computer. The method of implementing the database is not limited, and may be, for example, a database management system, a spreadsheet software, or a text file such as XML or JSON.

FIG. 2 is an example of a connection diagram of the photographing devices 101 (101F, 101R), the user terminal 102, and the external server 103.
The photographing device 101, the user terminal 102, and the external server 103 are connected to each other via a network 104. As described later, in this embodiment, of the photographing devices 101F and 101R, the photographing device 101F connects to the network 104 and performs communication. When data transmission/reception is required between the photographing device 101R and the network 104, or when data transmission/reception is required between the photographing device 101R and the user terminal 102, the photographing device 101F performs communication to relay data transmission/reception between the photographing device 101R and the network 104. However, the photographing device 101R may directly transmit and receive data to and from the user terminal 102. The network 104 may be wired or wireless, and each user terminal 102 can transmit and receive information via the network 104. The network 104 is not limited to the Internet, and may be a combination of networks with different protocols.

As described above, the image capturing device 101 is a device that is mounted on, for example, the vehicle 10 and records images while driving.
However, although the vehicle 10 to be mounted is a four-wheeled automobile in this embodiment, it is not limited to a four-wheeled automobile, and may be any vehicle in which the imaging device 101 can be installed. For example, large transport vehicles with four or more wheels such as buses and trucks, two-wheeled vehicles such as motorcycles and bicycles, and other vehicles may be targets for mounting. In addition, for example, transportation vehicles such as trains, monorails, and linear motor cars may also be targets for mounting.
The photographing device 101 can transmit various acquired information, information relating to the results of event determination (described later), and the like, to the user terminal 102 via the network 104 .

The user terminal 102 is a terminal used by, for example, a user who drives a car. The user terminal 102 is connected to the photographing device 101 via a mobile data communication network or an in-vehicle network 104 such as Wi-Fi (registered trademark) of the car, and can operate the photographing device 101.
In addition, the user terminal 102 is connected to an external server 103 via these networks 104 and can transmit various types of information to the external server 103 .

FIG. 3 shows an example of the hardware configuration of the user terminal 102.
The user terminal 102 is composed of a terminal such as a smartphone, a tablet, a notebook PC, or a desktop PC.
The main memory device 301 stores programs and applications such as an image information recording module 310, a warning module 311, a display control module 312, a photographing device linkage module 313, and a photographing device control module 314, and the various functional elements of the user terminal 102 are realized by the processor 303 executing these programs and applications.

The image information recording module 310 stores the image information transmitted from the photographing device 101 via the network 104 in the auxiliary storage device 302 for the terminal. The auxiliary storage device 302 is an example of a storage means for the terminal. The storage means for the terminal is a storage means reserved for the user terminal 102, and means a storage means that the user terminal 102 can use to record data. The storage means for the terminal may be a storage means inside the user terminal 102 or a storage means outside the user terminal 102. The storage means outside the user terminal 102 may be, for example, a storage means that the user terminal 102 accesses using a communication means, for example, a storage means reserved on a network.

The warning module 311 uses the output device 305 such as a display and a speaker to warn the user about various events determined by the photographing device 101 .
A display control module 312 controls the display content and display layout on the output device 305 such as a display of the user terminal 102 .
The photographing device cooperation module 313 executes connection and functional cooperation with one or more photographing devices 101 .
The photographing device control module 314 executes control from the user terminal 102 to the photographing device 101 that is connected and functionally linked via the photographing device linkage module 313 .
The specific functions of these individual modules will be described in detail using other drawings.

As described above, the main memory device 301 stores applications, programs, etc. (modules) for displaying and recording image information transmitted from the image capture device 101. Such applications are sometimes referred to as APPs in the drawings.

The display control module 312 displays an operation area and a display area on the output device 305 of the user terminal 102. The operation area is, for example, an area for displaying operation elements (operation parts) for accepting operations and settings for the user terminal 102 and the photographing device 101. The display area is, for example, an area for displaying image information transmitted from the photographing device 101. The operation area and the display area will be described in more detail in relation to FIG. 14. Various examples of the operation area and the display area are also shown in other drawings.

The auxiliary storage device 302 saves setting information for each module stored in the main storage device 301 and image information received from and stored in the image capture device. In this specification, image information refers to, for example, still image information, video information, and the like.
The image information recording module 310 stores still images in still image file formats such as JPEG, PNG, and GIF, and stores videos in video file formats such as MOV, MP4, AVI, WMV, and WemM.

FIG. 4 shows an example of the hardware configuration of the photographing device 101. As shown in FIG.
The photographing equipment controller 404 is connected to the camera 402, the acceleration sensor 405, the SD card 406, the GNSS sensor 407, and other sensors (not shown), and controls them. Based on information received from these elements, the photographing equipment controller 404 generates status information indicating the state of the photographing equipment controller 404. The photographing equipment controller 404 also transmits the generated status information to the microcomputer 401.

The microcomputer 401 performs various processes, for example, as follows.
- Controls the communication processing unit 414.
The communication processing unit 414 notifies the photographing device controller 404 of the acquired location information.
The status information acquired from the photographing device controller 404 and the position information acquired from the communication processing unit 414 are converted into, for example, 16-byte data.
・Performs sleep control when powered by a secondary battery.
・Execute test mode during manufacturing and testing.

The microcomputer 401 is programmable, and stores programs for executing the various processes described above, and the various processes described above are realized by executing these programs with a processing unit of the microcomputer 401. In this embodiment, the processing unit of the microcomputer 401 or the microcomputer 401 itself may be referred to as a control unit.
The front photographing equipment 101F receives image information from the rear photographing equipment 101R via a network 104 such as a cable 11, and the photographing equipment controller 404 and the microcomputer 401 of the front photographing equipment 101F execute various processes on the received image information.
However, the rear photographing device 101R may also be configured to include the photographing device controller 404 and the microcomputer 401, and the rear photographing device 101R itself may perform the above-mentioned processing on the image information.

The photographing device controller 404, the microcomputer 401, and the communication processing unit 414 can each be implemented as an integrated circuit (chip) such as a SoC (System on a chip), or a configuration in which a plurality of these are implemented together on a single chip may be used. In particular, in this embodiment, the photographing device controller 404 and the microcomputer 401 work in conjunction with each other, so they may be implemented together on a single chip.

Also, the microcomputer 401 may be configured to execute some of the functions of the photographing device controller 404, or the photographing device controller 404 may be configured to execute some of the functions of the microcomputer 401. In this embodiment, the photographing device controller 404 and the microcomputer 401 are collectively referred to as the control unit 403. The control unit 403 may be a single chip, or may be configured to be composed of multiple chips that separate the functions of the photographing device controller 404 and the microcomputer 401.

In addition, by equipping the image sensor of the camera 402 with an AI processing function for performing pre-processing, it is possible to perform, for example, the following output in addition to or instead of generating a normal captured image. The AI processing function has a function for performing image recognition based on the images captured by these cameras.
・Output the object from the image as metadata.
- Outputs images in an ISP (Image Signal Processor) output format, for example, YUV or RGB.
・Output an image that is cropped from a specific area.

By performing preprocessing on the image sensor side in this way, it is possible to reduce the amount of data and enable real-time tracking of objects using high-speed AI processing.
For example, if a plurality of photographing devices 101 (101F, 101R) are provided in a vehicle and each of the photographing devices 101F, 101R is equipped with an AI processing function that performs pre-processing on the image sensor of the camera 402, each of the photographing devices 101F, 101R can independently perform video processing and image processing, so that each of the photographing devices 101F, 101R can create image information with ideal image quality. In addition, it is possible to avoid concentration of the computational burden of video processing and image processing on one of the photographing devices 101F, 101R.

For example, if photographing device 101F is a front camera and photographing device 101R is a rear camera, and each performs image processing, then compared to a system in which only the front camera processes images from the front and rear cameras, the front camera processes only the front image, resulting in reduced load and stable operation, and the rear camera processes images using its own image processing engine, enabling ideal image quality to be achieved.

As the network 104 that handles communication between the photographing equipment 101F and the photographing equipment 101R, it is preferable to use one that complies with the communication standard of digital transmission (Ethernet (registered trademark) system). When the photographing equipment 101F and the photographing equipment 101R are connected with a cable 11 that complies with the Ethernet standard and image information is transmitted from the photographing equipment 101R to the photographing equipment 101F, for example, it is possible to record clearly without degradation even with the same FULL HD image quality. For example, such digital transmission makes it possible to record clearer and sharper rearward images than with conventional analog transmission models.

The acceleration sensor 405 detects acceleration and generates acceleration information. For example, when a sudden change in acceleration occurs due to an impact, abrupt steering, a sudden stop, etc., the control unit 403 detects the occurrence of an event. The control unit 403 can infer the circumstances of an accident or the like that has occurred to an automobile or the like by analyzing the change in the acceleration information.
The SD card 406 is a storage device that records various information such as image information and recording information corresponding to the image information. Note that the SD card is not limited to an SD card, and any configuration that can store data will do, but a flash memory with high vibration resistance is preferable for mounting on a vehicle.
In addition, if there are multiple photographing devices 101F, 101R in the vehicle 10 and the multiple photographing devices 101F, 101R can share a storage device, it is not necessary for all of the photographing devices 101F, 101R to have a storage device such as an SD card 406.

For example, if the photographing device 101F is a drive recorder and the photographing device 101R is a rear camera without a recording function, the rear camera does not need to have a storage device such as the SD card 406.
Furthermore, the configuration for determining the vehicle condition described below, the configuration for storing image information captured by the photographing equipment 101F, 101R, the configuration for transmitting image information to the user terminal 102, etc. may be concentrated in the front drive recorder, and these configurations may be omitted in the rear camera, with the rear camera being configured to at least transmit image information to the drive recorder.

As for other configurations, functions, etc., if the configurations, functions, etc. can be shared among multiple photographing devices 101, it is also possible to omit some or all of the configurations, functions, etc. for each photographing device.
In this embodiment, the photographing device 101F basically performs advanced functions such as image processing and transmission processing to the user terminal 102 and has a storage device, and the rear side is described as having a simple photographing function, but conversely, the rear side may have advanced functions concentrated, and the front side may be a front camera that only performs simple photographing.

The communication processing unit 414 transmits information about the photographing device 101 to the user terminal 102 via the wireless communication antenna 408 and the network.
The GNSS sensor 407 has a location information acquisition unit implemented by an integrated circuit or the like, and acquires location information based on a signal received from a GNSS antenna compatible with a GNSS (Global Navigation Satellite System) such as a Global Positioning System (GPS). The location information acquisition unit can also acquire time information, moving speed information, and moving direction information.

The photographing device 101 has a power supply 411 that is connected to a vehicle power supply 410 .
The power supply 411 supplies power received (or sometimes referred to as power reception) from the vehicle power supply 410 to the photographing device controller 404 , and also charges the secondary battery 412 .

The secondary battery 412 is a rechargeable battery such as a lithium ion battery, a nickel cadmium battery, a nickel metal hydride battery, or a lead storage battery. However, it may be a primary battery that is not rechargeable. When the accessory power supply (ACC power supply) is turned off, when the supply of power from the vehicle power supply 410 to the power supply 411 of the photographing device 101 is interrupted due to a fault such as a broken wire or a disconnected wire caused by an accident, or when the power supply 411 is unable to supply power due to a malfunction or the like, the secondary battery transmits (or may be called power transmission or power supply) power to the microcomputer 401, the communication processing unit 414, etc., and maintains at least some of the functions of the photographing device 101.

The control unit 403 acquires image information from the camera 402 of the photographing device 101, and stores the image information in a storage device such as an SD card 406. Note that in this specification, "storage" may also be referred to as "recording".
The control unit 403 of the photographing equipment 101 further detects the state of the vehicle 10 based on the image information from the camera 402, such as the distance or positional relationship to at least one of the vehicles in front of the vehicle 10 or the vehicle behind the vehicle 10, the length of time that either the front or rear vehicle is in a judgment area or other reference area or proximity area described below, the speed of the vehicle, and the speed of either the front or rear vehicle.

The control unit 403 determines whether or not the detected state of the vehicle 10 satisfies a specific event recording condition, which will be described later. The control unit 403 can also set individual event recording conditions for each of a plurality of different events, which will be described later.
The control unit 403 can also use the functions of a forward collision warning system (FCWS) or a rearward collision warning system (RCWS), for example, to measure the length of time that either the front or rear vehicle is in a specific reference area (proximity area).

FIG. 5 shows an example of the hardware configuration of the external server 103 .
5, the external server 103 is shown as an example of a server operated to provide an SNS service, but it may be an external server of another private company, public institution, etc. The external server 103 has a main memory device 501, an auxiliary memory device 502, a processor 503, an input device 504, an output device 505, and a communication control unit 506, and a bus connecting these to each other.

The main memory device 501 stores a command receiving module 510 and a content posting module 511, and the processor 503 executes these programs and applications to realize each functional element of the external server 103.

The command receiving module 510 receives and processes commands from users who use the SNS service. The command receiving module 510 stores data provided by the users, such as image data and video data, in the auxiliary storage device 502 as content information.
The content posting module 511 outputs the result of processing a command from a user by the command receiving module 510 in a state accessible by other users who use the SNS service.

Depending on the result of command processing from the user, the content posting module 511 can also output the content information stored in the auxiliary storage device 502 in a state accessible by other users who use the SNS service.
The auxiliary storage device 502 stores various types of program information 520 and content information 530 related to the SNS service.

Next, the operation of the present embodiment will be described. Among the operations described below, the operation of the user terminal 102 may be realized based on an application (a smartphone app for a photographing device) installed in the user terminal 102.
FIG. 6 shows an example of a first flow 60 of image recording, event notification information output, and index information addition by the photographing device 101.

The control unit 403 of the photographing device 101 executes the following process according to the first flow 60. Before explaining the first flow 60, the process before that will be explained. The photographing device 101 has two settings: a setting for connecting to the user terminal 102 via an in-vehicle network 104 such as Wi-Fi (registered trademark) when the photographing device 101 is turned on (for example, Wi-Fi ON mode), and a setting for not connecting to the user terminal 102 when the photographing device 101 is turned on (for example, Wi-Fi OFF mode). The latter is the default setting. In the latter case, the user can connect to the photographing device 101 by operating the user terminal 102. In the following explanation, the photographing device 101 may be set to either setting, but it is assumed that the photographing device 101 and the user terminal 102 are connected.

With these settings, the photographing equipment 101 can automatically or manually transmit data via a Wi-Fi (registered trademark) connection, and the user of the user terminal 102 can check the image information captured by the photographing equipment 101 at hand and post the confirmed image information directly to SNS, etc. in an appropriate manner.
Furthermore, by the following process, video recording is performed on the user terminal 102, for example a smartphone, by automatic transmission from the photographing device 101. The automatic transmission records video for 30 to 60 seconds after sensor detection, for example, video recording of aggressive driving. These points will be described in detail later.

In addition, when there are a plurality of photographing devices 101 in the vehicle as described above, only one photographing device 101, for example, the photographing device 101F, may execute the first flow 60, and the other photographing devices 101, for example, the photographing device 101R, may simply send image information to the photographing device 101F executing the first flow 60 via, for example, the cable 11 in Fig. 1. In this case, the photographing device 101F executing the first flow 60 may record the image information of the photographing device 101R and may communicate with the user terminal 102 to transmit the image information.
In the following embodiment, the photographing equipment 101 as the photographing equipment 101F connects to and communicates with the user terminal 102, and when data transmission and reception between the photographing equipment 101R and the user terminal 102 is required, the photographing equipment 101F communicates to relay the transmission and reception of data between the photographing equipment 101R and the user terminal 102.

Next, the first flow 60 will be described.
The control unit 403 acquires image information (including date, time, camera identification number, etc.) from the camera 402 (S600) and stores it (S610). The steps described below can be applied to various different image recording processes and video recording processes performed by the photographing device 101, and a detailed description of the image recording processes and video recording processes will be omitted here.

The communication processing unit 414 sequentially transmits a plurality of pieces of image information of the photographing device 101 to the user terminal 102 with which a connection has already been established via the in-vehicle network 104 such as the vehicle's Wi-Fi (registered trademark) based on, for example, event recording conditions described later (S620). The event recording conditions will be described in detail in Figs. 6 and 25, etc.
When there are two photographing devices 101F, 101R in the vehicle, the multiple image information transmitted by the photographing device 101 to the user terminal 102 may, for example, be transmitted by one of the photographing devices 101F, 101R, only that image information captured by the photographing devices 101F, 101R that is displayed on a preview screen described below, or the multiple image information transmitted by the photographing devices 101F, 101R may always be transmitted.

The control unit 403 measures driving information such as the speed and acceleration of the vehicle 10 using the acceleration sensor 405 (S630), and stores the speed and acceleration information (S640).
Step S620 may be performed after steps S630 and S640. Alternatively, in another simpler embodiment, S620 may be omitted. In the case where S620 is omitted, the photographing device 101 transmits image information with the start of the following S640.

The acceleration information in steps S630 and S640 is the forward/rearward component of the traveling direction of the vehicle 10 obtained from the acceleration sensor 405, but a three-axis composite value other than gravitational acceleration can also be used. More specifically, the control unit 403 generates recording information by combining the obtained speed and acceleration with information obtained from various sensors of the photographing device 101. The control unit 403 associates the recording information with the image information obtained from the camera 402 and stores them in the SD card 406 (storage unit) (S640). For example, the control unit 403 associates the date, time, camera identification number, front or rear type, and driving information (speed, acceleration, position coordinates, etc.) and stores them in the storage means.

The control unit 403 analyzes the acquired image information and executes a vehicle determination process to determine whether the vehicle 10 satisfies the event recording conditions (S650). The event recording conditions and the vehicle determination process will be described later with reference to another diagram. The event recording conditions are conditions that are determined based on the state of the vehicle 10, such as whether there is an approaching vehicle either in front or behind. However, as will be described later, the event recording conditions may be other conditions.

The control unit 403 determines whether the state of the vehicle 10 satisfies the event recording condition based on the acquired image information (S660). If the state of the vehicle 10 does not satisfy the event recording condition (No in S660), the control unit 403 repeats steps S630 to S650.
When the state of the vehicle 10 satisfies the event recording condition (Yes in S660), the control unit 403 transmits event notification information (notification information notifying that the recording condition has been satisfied, or information for storing the transmitted plurality of pieces of image information in the terminal) to the user terminal 102. The event notification information is notification information notifying that the vehicle 10 satisfies the recording condition, and is data that serves as a trigger for the image information recording module 310 of the user terminal 102 to start recording the image information transmitted from the photographing device 101 when the vehicle 10 satisfies the event recording condition.

Particularly advantageously, the vehicle determination process (S650) in this embodiment may perform an event determination on the images recorded by the image capture device 101, in particular, a determination of dangerous driving such as tailgating from behind. In the vehicle determination process (S650), the control unit 403 executes a vehicle determination process that, for example, analyzes the acquired image information and determines whether a vehicle is present in front of or behind the vehicle 10.

The determination of whether or not there is an adjacent vehicle is based on, but not limited to, detection by a forward collision warning system (FCWS) or a rear collision warning system (RCWS).
For example, when the size of a vehicle in an image captured by the camera 402 becomes equal to or larger than a predetermined size, it can be determined that the vehicle has approached within a distance of, for example, 30 m.
Alternatively, for example, proximity determination can be made by measuring distance using a distance measurement sensor that uses radio waves, sound waves, or the like, without using image processing.

The above vehicle determination process (S650) makes it possible to start recording an event on the terminal after the occurrence of an event detected by the forward collision warning system (FCWS) or the rear collision warning system (RCWS).
This makes it possible, for example, for the terminal to start recording when a rear collision warning system (RCWS) detects that a vehicle behind is approaching abnormally close, and for the terminal to start recording when a front collision warning system (FCWS) detects that a vehicle ahead is suddenly braking.
According to this embodiment, it is possible to reliably store image information of an event in which image information after the event occurs is more important.

The proximity range for determining whether or not a vehicle is approaching an obstacle may be variable. For example, the proximity range may be changed depending on the speed of the vehicle 10.
For example, when the vehicle 10 is traveling fast, the distance between the vehicles increases, so the proximity range can be set wide; when the vehicle is traveling slow, the distance between the vehicles decreases, so the proximity range can be set narrow.
A more specific example of the vehicle determination process will be described in more detail below, particularly in conjunction with FIG.

If the vehicle determination result indicates that the state of the vehicle 10 does not satisfy the event recording condition (No in S660), the control unit 403 returns the first flow 60 to S630.
If the vehicle determination result indicates that the state of the vehicle 10 satisfies the event recording condition (Yes in S660), the control unit 403 adds index information related to the event to the image information from the camera 402 stored in a storage unit of the photographing device 101 such as the SD card 406 (S670). The index information can be created based on information regarding the type of event that has occurred, values characteristic of each event, such as a sudden change in acceleration, distance to the preceding and succeeding vehicles, GPS coordinates of the location where the event has occurred, the date and time of the event occurrence, etc.

The index information related to an event makes it easy to call up photographing device image information related to each event stored in the memory unit of the user terminal 102 from the photographing device image information stored in the memory unit of the photographing device 101, such as an SD card 406.
The control unit 403 adds index information related to the event to the image information stored in the storage unit of the photographing device 101 (S670), and then transmits event notification information to the user terminal 102 (S680).

The control unit 403 of the photographing equipment 101 can use the index information that has already been attached to the photographing equipment image information recorded on the SD card 406 of the photographing equipment 101 as event notification information to be sent to the user terminal 102 or as part of the event notification information.
In this case, the control unit 403 of the photographing equipment 101 does not require the load of creating new event notification information to send to the user terminal 102, or such load is reduced, so this is significantly more advantageous in terms of computational load and manufacturing costs than a case in which a dedicated configuration is provided for creating notification information for the start of recording of event video to the auxiliary storage device 302 for the user terminal 102.

When index information is used as event notification information, for example, the photographing device control module 314 of the user terminal 102 can call up, from the photographing device 101, photographing device image information recorded on the SD card 406 of the photographing device 101 that has index information corresponding to the event notification information sent to the user terminal 102.

In the first flow 60, the index information is added to the captured image information after it is determined that the event recording conditions are met (S670), but it may also be added when the image is stored in the photographing device in S610.
Also, it is possible to configure the system so that index information is not used.

Based on the result of this comparison, the photographing device control module 314 can instruct the photographing device 101 to search and extract photographing device image information (second image information) recorded on the SD card 406 or the like of the photographing device 101 that corresponds to the event video recorded in the auxiliary storage device 302 of the user terminal 102, and if there is a plurality of corresponding photographing device image information, to list them, etc. After that, the photographing device control module 314 can instruct the photographing device 101 to transmit the searched photographing device image information to the user terminal 102. Also, if image information photographed by the photographing device 101 is stored on the cloud, it can also be searched for and displayed based on the index information.

The photographing device image information is, for example, image information including an event of an event video recorded in the auxiliary storage device 302 of the user terminal 102, and is image information of a longer duration or higher image quality, or of a longer duration and higher image quality. The photographing device image information may also include audio information, which is particularly preferable when the image information recording module 310 stores image information not including audio information in the auxiliary storage device 302.

In particular, the photographing device image information may be, for example, image information from a period before the state of the vehicle 10 satisfied the event recording condition, and therefore image information from before the event recorded in the auxiliary storage device 302 of the user terminal 102. The photographing device image information may further be image information captured during a period including before and after the state of the vehicle 10 satisfied the event recording condition.

The event notification information may be different data depending on the event to be recorded in the user terminal 102, the settings of the user terminal 102, and the settings of the photographing device 101. For example, when a rear abnormal approach is an event to be recorded in the user terminal 102, the control unit 403 can create the event notification information based on data regarding the distance to the rear vehicle, for example.
Also, for example, if a forward approach is an event to be recorded by the user terminal 102, the control unit 403 can create event notification information based on data regarding the distance to the vehicle ahead, for example.

The control unit 403 can notify the user terminal 102 of the occurrence of an event such as rearward collision or forward collision by transmitting event notification information related to abnormal rearward collision or forward abnormal frontward collision.
The control unit 403 can also select, for each event, whether or not to transmit event notification information to the user terminal 102 .
The event notification information does not necessarily have to be common to all events.

If the control unit 403 does not receive confirmation information of receipt of the event notification information from the user terminal 102 after sending the event notification information (No in S690), the control unit 403 returns the first flow 60 to S680, and the control unit 403 sends the event notification information to the user terminal 102 again.
When the control unit 403 receives the receipt confirmation information from the user terminal 102 in S690 (Yes in S690), the control unit 403 determines whether or not to end the first flow 60 (S695).

If the control unit 403 does not end the first flow 60 (No in S695), the control unit 403 returns to S630 of the first flow 60. If the control unit 403 ends the first flow 60 (Yes in S695), the control unit 403 ends the first flow 60. Examples of cases in which the control unit 403 ends the first flow 60 (Yes in S695) include a case in which the photographing device cooperation module 313 releases the connection with the user terminal 102 or a case in which the photographing device control module 314 outputs an end command for the first flow 60.

In the example of FIG. 6, the control unit 403 continues to execute image recording until the end of the first flow 60 (S610), regardless of whether an event occurs or whether notification information is sent to the user terminal 102 or not.
In particular, through S630-S660, the control unit 403 can determine whether at least one of an event recording condition based on at least an impact occurring to the vehicle 10 and an event recording condition based on the approach of another vehicle to the host vehicle 10 is satisfied. Then, the control unit 403 can transmit an event recording notification, which is a trigger for recording an event video in the auxiliary storage device 302 for the user terminal 102, to the user terminal 102 without requiring any operation by the user.

According to this embodiment, for example, while the driver is driving his/her vehicle 10, a sensor (VOICE: rear abnormal approach warning) detects aggressive driving by a following vehicle, and the driver or passenger becomes anxious or upset, saying, "Wow! They're tailgating me!". However, the rear image after the detection is automatically sent to and saved on the smartphone (the dashcam itself also records images of the front and rear), so the driver and passenger can check the smartphone and feel reassured that "everything is properly recorded, so there is evidence in case of an emergency."

FIG. 7 shows an example of a second flow 70 of image recording, event notification information output, and index information addition by the photographing device 101.
The steps S700, S720 to S750, S756, and S760 to S795 of the second flow 70 correspond to the steps S600, S620 to S650, S670, and S660 to S695 of the first flow 60 described above, respectively, and therefore will not be described.
Unlike the example of Figure 6, in the example of Figure 7, the control unit 403 acquires image information from the camera 402 (S600) and transmits the acquired image information to the user terminal 102 (S720) without storing the acquired image information in the memory unit of the photographing equipment 101 until the photographing equipment recording conditions are satisfied.

Then, similar to the example in FIG. 6, the control unit 403 measures driving information such as the speed and acceleration of the vehicle 10 (S730), stores this speed and acceleration information (S740), and performs a vehicle determination process (S750) to determine whether the vehicle 10 satisfies the imaging device recording conditions and the event recording conditions.

After the vehicle determination process (S750), if the vehicle 10 satisfies the photographing equipment recording conditions (first recording conditions) (Yes in S752), the control unit 403 records the image information acquired from the camera 402 in the memory unit of the photographing equipment 101 (S754), and further adds index information to the image information (S756), and proceeds to step S760.
If the vehicle 10 does not satisfy the photographing device recording condition (No in S752), the control unit 403 proceeds directly to step S760.

Then, the control unit 403 executes steps S760, S780, S790, and S795 in the same manner as the first flow 60 in FIG. 6, and if the vehicle 10 satisfies the event recording condition (second recording condition) (Yes in S760), it transmits event notification information to the user terminal 102 so as to reliably store, for example, multiple image information after the time when the event recording condition was satisfied in the auxiliary storage device 302 of the user terminal 102 (S780, S790).

In another embodiment, the control unit 403 may, for example, when an approaching vehicle from behind the vehicle 10 is set as an event recording condition, issue a warning using a sound effect such as a "beep" or a voice such as "Watch your rear" or "Dangerous driving to the rear has been detected" as described later with reference to FIG. 27, and may also transmit information for storing multiple image information in the auxiliary storage device 302 of the user terminal 102.

In the example of FIG. 7, the control unit 403 records image information in the memory unit of the photographing device 101 when the vehicle 10 satisfies the photographing device recording conditions, so that the photographing device 101 can use the capacity of the memory unit more efficiently than in the example of FIG. 6. In addition, as a result, for example, the image information recorded in the memory unit of the photographing device 101 can be recorded with higher image quality than the image information recorded in the memory unit of the photographing device 101 in the example of FIG. 6.

In the example of FIG. 7, the control unit 403 can separately set the photographing device recording conditions for recording image information in the memory unit of the photographing device 101 and the event recording conditions for recording image information in the auxiliary storage device 302 of the user terminal 102. In addition, it is preferable that each setting can be changed according to the content of each event.

As a result, for example, the control unit 403 can record image information related to various types of dangerous driving, such as "abnormally approaching from behind," "abnormally approaching from the front," "dangerously cutting in," "sudden approach," "dangerous cutting in" by other vehicles from the left and right lanes, "excessive honking" or "abnormal passing/high beams" by other vehicles traveling around the vehicle 10, and "obstruction of overtaking" by a vehicle in front, in the memory unit of the image capture device 101, and the user terminal 102 can record only image information related to, for example, "abnormally approaching from behind" from among these types of dangerous driving.

In yet another embodiment, the control unit 403 records the image of the photographing device after acquiring the image of the photographing device as in the example of FIG. 6 (S600, S610), and further, when an event occurs that corresponds to the photographing device recording condition of S752 in the example of FIG. 7, it is possible to change the recording image quality of the image information according to the priority of each event, and record image information of higher image quality than the image information recorded in S610.

By appropriately combining the embodiments of FIG. 6 and FIG. 7, the following operation becomes possible.
The photographing device 101 performs an event recording, and transmits trigger information for executing the event recording to cause the user terminal 102 to also execute the event recording.
When the photographing device 101 detects the occurrence of an event while it is continuously transmitting video information to the user terminal 102, it transmits information that serves as a trigger for executing event recording to the user terminal 102. In this embodiment, sequential transmission of video information means transmitting frames of image information captured by the photographing device 101 in chronological order. The transmitted image information is displayed in motion on a preview screen.

- For each event type, it is possible to set whether or not to transmit trigger information for executing event recording to the user terminal 102. Trigger information is transmitted only for the set type of event, and not for others. Depending on the type of event, information such as a video may be recorded only in the photographing device 101, or may be recorded in both the photographing device 101 and the user terminal 102. In this case, it is more desirable to transmit trigger information for tailgating from behind.
Since it is possible to cause the user terminal 102 to issue a warning, the transmission of trigger information for recording an event and the transmission of trigger information for outputting a warning can be switched. Note that in this embodiment, a warning may be used to mean a lower level of notification than an alarm.

- Switch between a setting for continuous transmission of image information from the photographing device 101 to the user terminal 102 when connected via Wi-Fi or the like, and a setting for starting transmission when the occurrence of a specific event is detected. For example, the photographing device 101 transmits image information only in the case of rear-facing shots, and the user terminal 102 executes an event record of the transmitted image information (and may also display it).

FIG. 8 shows an example of a flow 80 of recording image information of the photographing device 101 by the user terminal 102.
In the example of Fig. 8, the photographing device cooperation module 313 of the user terminal 102 searches for and connects to the corresponding photographing device 101 (S800). The connection to the corresponding photographing device 101 may be executed when the corresponding photographing device 101 is turned on, or may be executed when the image information recording module 310 of the user terminal 102 receives an input of a connection command to the photographing device 101. Please refer to the detailed explanation already given in relation to Fig. 6 for this point.

In this embodiment, at least one of the photographing devices 101 is described as being, for example, a drive recorder, but the photographing devices 101 are not limited to drive recorders, and the photographing device linking module 313 of the user terminal 102 can also establish a connection with three or more photographing devices 101. For example, the photographing device linking module 313 of the user terminal 102 can also establish a connection with a photographing device provided in a position that can determine whether the vehicle 10 is moving to the side.

If the connection between the user terminal 102 and the photographing device 101 is not established (NO in S805), the photographing device cooperation module 313 of the user terminal 102 repeats S800.
When a connection between the user terminal 102 and the photographing device 101 is established (Yes in S805), the user terminal 102 receives image information from the photographing device 101 using the communication control unit 307 and the display control module 312 (S810). At this point, the user terminal 102 has not stored the image information from the photographing device 101. Note that, in another embodiment, S810 may be omitted. When S810 is omitted, the image information from the photographing device is received, for example, after S820 described later.

The display control module 312 of the user terminal 102 displays the image information from the photographing device 101 on the output device 305 such as a display of the user terminal 102 (S815). The output device 305 such as a display of the user terminal 102 may hereinafter be simply referred to as a "screen."
In another embodiment, the display control module 312 may omit the execution of S815, or may selectively execute the process for each event.

The photographing device cooperation module 313 of the user terminal 102 receives the event notification information from the photographing device 101 via the communication control unit 307 (S820). In another embodiment, it is also possible to omit S820. When S820 is omitted, for example, an image analysis module (not shown) of the user terminal 102 may analyze the image received from the photographing device, and a vehicle determination module (not shown) of the user terminal 102 may determine the state of the vehicle 10.

When an event occurs in the vehicle 10 and the state of the vehicle 10 satisfies the event recording condition (Yes in S825), the user terminal 102 checks whether or not a warning setting has been made for the target event (S830).
In another embodiment, the reception of the event notification information in S820 may be set as a condition for the state of the host vehicle satisfying the event recording condition in S825 (Yes in S825).

The event is related to an event determined in the vehicle determination process S650 of the first flow 60, which has already been described with reference to Fig. 6. The event may be, for example, various types of dangerous driving, such as "abnormal approach from behind", "abnormal approach from the front", "dangerous approach to the side", "sudden approach", "dangerous cutting in" by other vehicles from the left and right lanes, "excessive honking" or "abnormal passing/high beams" by other vehicles traveling around the vehicle 10, and "obstruction of overtaking" by a vehicle ahead. The control unit 403 of the photographing device 101 can detect rearward tailgating, forward tailgating, side tailgating, sudden acceleration of a vehicle behind, and sudden braking of a vehicle ahead by determining "abnormal approach from behind", "abnormal approach from the front", "dangerous approach to the side", and "sudden approach".
Furthermore, the vehicle determination process S650 can also determine whether a tire has been punctured or whether a wheel has fallen off or run off the road.

The image information recording module 310 of the user terminal 102 may determine that the event recording condition is satisfied by receiving event notification information from the photographing device 101. In another embodiment, the image information recording module 310 may determine whether the state of the vehicle 10 satisfies the event recording condition by performing a process similar to the vehicle determination process S650 of the first flow 60 in FIG. 6 on the image information received from the photographing device 101.

If there is a setting to issue a warning for the event that has occurred (Yes in S805), the warning module 311 of the user terminal 102 issues a warning according to each event. For example, when event notification information about an approaching vehicle from behind is received, the user terminal 102 may output an audio warning such as "Watch your rear" from an output device 305 such as a speaker (S835).

After the warning is output or if there is no setting to issue a warning for the event that has occurred (No in S805), the image information recording module 310 of the user terminal 102 checks the event recording settings (S840).

If there is a setting to record image information for the event that has occurred (Yes in S840), the user terminal 102 uses the image information recording module 310 to store the image information of the event that has occurred in the auxiliary storage device 302 for the set length of time (the length of time of the multiple pieces of image information to be stored) (S845, S850).

After recording the image information or if there is no setting to record image information for the event that has occurred (No in S840), the image information recording module 310 of the user terminal 102 checks whether or not to end the image information recording flow 80 (S860). If the image information recording flow 80 is to be continued (No in S860), the image information recording module 310 of the user terminal 102 executes S815.
Examples of cases in which the image information recording flow 80 is terminated (Yes in S860) and the image information recording module 310 terminates the image information recording flow 80 (Yes in S860) include cases in which the photographing device linkage module 313 disconnects from the user terminal 102 or cases in which the image information recording module 310 receives a command to terminate the image information recording flow 80.

As a result of the above steps, in the example of FIG. 8, the photographing device 101 transmits image information to a user terminal 102 such as a smartphone, and the user terminal 102 can store the image information transmitted from the photographing device 101 in the auxiliary storage device 302. In particular, it becomes possible to automatically transfer and store images to a smartphone in relation to aggressive driving, strong impacts, sudden braking, abrupt steering, etc.

In particular, the user terminal 102 can store video of an event that has occurred immediately after the event has actually occurred. Such post-event video recording is particularly important with respect to reckless driving events, such as aggressive driving.
According to this embodiment, the user terminal 102 and the photographing device 101 can provide the user with important information after the occurrence of such an event, without the user having to operate the user terminal 102 by himself.
In addition, since the image information is stored in the user terminal 102, the user is not forced to take the trouble of removing the SD card from the photographing device 101, inserting it into a computer, and displaying it on a viewer, and the user can easily check the image photographed by the photographing device 101 using only the user terminal 102.

By S815, the user terminal 102 can provide the user with images of the area around the vehicle 10 captured by the photographing device 101 on the user terminal 102, and by S845, can record image information of the image that was displayed when the state of the vehicle 10 satisfies the event recording conditions for each event in a memory unit for the user terminal 102.
Furthermore, the user terminal 102 can also accept a setting as to whether or not to record image information in a storage unit for the user terminal 102 for each event.

When the vehicle 10 satisfies the event recording conditions (S825) based on the event notification information (S820) received from the photographing device 101, the image information recording module 310 of the user terminal 102 can record image information of the state in which the vehicle 10 satisfies the event recording conditions in the auxiliary storage device 302 for the terminal (S845).

Furthermore, the photographing device 101 can record image information corresponding to the event video stored in the user terminal 102, which is stored for a longer period of time and with higher image quality. For example, if one of the multiple photographing devices 101 is a drive recorder, multiple pieces of image information can be recorded from the start to the end of driving.

As described below, the photographing device control module 314 of the user terminal 102 can also instruct the photographing device 101 to transmit image information stored in the photographing device 101 that corresponds to the event video in the user terminal 102.

FIG. 9 shows an example of a setting flow 90 for the photographing device 101 by the user terminal 102.
In the example of Figure 9, the photographing device collaboration module 313 of the user terminal 102 accepts setting input by the user using the input device 304 and output device 305 of the user terminal 102 for the photographing device 101 connected to the user terminal 102 (S900).

The photographing device control module 314 of the user terminal 102 transmits the setting input received in S900 to the photographing device 101 via the communication control unit 307 and the network 104 (S910).
The photographing device control module 314 of the user terminal 102 receives a confirmation notification regarding the setting input from the photographing device 101 (S920), and ends the setting flow 90.

Examples of settings of the photographing device 101 by the user terminal 102 will be described in more detail with reference to FIGS.
In particular, the setting flow 90 in Fig. 9 allows the photographing device control module 314 of the user terminal 102 to perform various settings for individual events on the photographing device 101. This allows the user terminal 102 to provide the user with the possibility of easily changing the settings of the photographing device 101 according to various driving conditions and surrounding conditions.

FIG. 10 shows an example of a flow 100 for stopping the display of image information by the user terminal 102.
In the example of Figure 10, the display control module 312 of the user terminal 102 displays image information received from the photographing equipment 101 in the display area of a screen (hereinafter sometimes simply referred to as the "screen") which is the output device 305 of the user terminal 102, and displays operation elements in the operation area of the screen of the user terminal 102 (S1000).
Examples of the display area and the operation area will be described in more detail with reference to FIG.

If the first set time has not elapsed (No in S1010), the display control module 312 continues to display the image information on the output device 305. If the no-operation time has elapsed after the first set time (Yes in S1010), the display control module 312 stops displaying the display area while continuing to display the operation area (S1020). For example, the display control module 312 may determine whether the first set time has elapsed since starting to display the screen, or may determine whether the no-operation time, which is the time during which no operation is performed on the user terminal 102 (input device 304), has elapsed after the first set time.

If the second set time has not elapsed (No in S1030), the display control module 312 continues to display only the operation area on the output device 305. If the second set time has elapsed (Yes in S1030), the display control module 312 stops displaying the operation area and the display area (S1040). For example, the display control module 312 may determine whether the second set time has elapsed since starting to display the screen, or may determine whether the no-operation time, which is the time during which no operation is performed on the user terminal 102 (input device 304), has elapsed the second set time.

The display stop flow 100 shown in FIG. 10 will be described in more detail with reference to FIGS.
According to this embodiment, the user terminal 102 is able to record events that are difficult for the driver to directly visually confirm without using a mirror or monitor, particularly rear-end collisions, without requiring any active operation or action by the user.

In addition, when the vehicle 10 is subjected to dangerous driving such as tailgating, the driver is forced to pay the utmost attention to ensuring the safe running of the vehicle, and therefore it is extremely difficult to record event information corresponding to the memory unit for the user terminal 102 in order to back up image information related to the dangerous driving or to secure evidence that can be submitted immediately. Even in such a case, it is not legally permitted for the driver to operate the user terminal 102 while driving.

However, according to this embodiment, even when the driver is exposed to such dangerous driving, the user terminal 102 can reliably record the corresponding event video without requiring any active operation or action by the driver.

FIG. 11 is an example of a screen showing some of the operations for setting reference lines 1101 and 1102 for event determination.
FIG. 12 is an example of a screen showing a part of the operation for setting reference lines 1201 and 1202 for event determination, which follows the example of the operation shown in FIG.

The display control module 312 can accept the user's selection of "Front Reference Position Setting" or "Rear Reference Position Setting" in the "Active Safety Setting" menu (see FIG. 26).
In the "front reference position setting" and "rear reference position setting", the photographing device cooperation module 313 can receive input for setting the reference lines 1101, 1102, 1201, and 1202 for event determination by the photographing device 101. Then, the photographing device control module 314 sets the received setting input in the photographing device 101.

The procedure for "setting the front reference position" is explained below using the examples of Figures 11 and 12, but the imaging device control module 314 can also perform "setting the rear reference position" using a similar procedure.

When the display control module 312 accepts the user's selection of "Front reference position setting" in the "Active Safety Settings" menu 1100, it displays image information received from the photographing device 101 connected via the photographing device linkage module 313 in the display area 1110 on the output device 305 of the user terminal 102.
In the examples of Figures 11 and 12, the display control module 312 displays, in addition to the image transmitted from the photographing device 101 within the display area 1110, red lines 1101, 1201 on the screen, which are reference lines for event determination, and white lines 1102, 1202 (two dashed lines in the shape of a V) on the screen, which are also reference lines.

The display control module 312 further displays the operation elements, buttons 1104 to 1109, a decision button, and a cancel button, in the operation area 1120 on the output device 305 of the user terminal 102.

In response to user input via buttons 1104 to 1109, the photographing equipment control module 314 of the user terminal 102 can move two V-shaped white lines (reference lines) 1102 and one red line (reference line) 1101 extending horizontally, which are displayed superimposed on the image from the photographing equipment 101 in the display area 1110, up and down and left and right, and can also narrow or widen the width of the two white lines (reference lines) 1102.
For proper event determination, it is preferable that the two white lines (reference lines) 1102 overlap with the white lines 1103 on the road, and it is also preferable that the red line (reference line) 1101 overlap with the horizon (horizontal line).

When the photographing equipment control module 314 receives an input to the "Confirm" button in the operation area 1120, it confirms the input setting of the reference line to the photographing equipment 101, and ends the input setting.
When the photographing device control module 314 finishes inputting and setting the reference line to the photographing device 101, the display control module 312 displays the "Active Safety Settings" menu 1100 again (see FIG. 26).

For example, if the photographing device 101 is a drive recorder with an LCD (an example of a display unit), the photographing device control module 314 of the user terminal 102 can adjust the image displayed on the LCD and the reference line for event determination in accordance with the user input results from the buttons 1104 to 1106 displayed in the operation area 1120 of the user terminal 102, in a manner similar to the procedure illustrated in Figures 11 and 12.

FIG. 13 is an example showing appropriately set reference lines 1301, 1302, and D along with imaginary decision lines A, B, and C.
The example in FIG. 13 shows an actual image after a reference line has been set for an image transmitted from the rear photographing device 101R, and below that, illustrations of a "good example" and a "bad example" for the reference line setting are shown.

After the reference line is appropriately set by the user terminal 102, the photographing device 101 judges dangerous driving such as tailgating from behind using the set reference line and virtual judgment lines A, B, and C set between them. Note that the virtual judgment lines A, B, and C are not actually displayed on the display of the user terminal 102, and are not included in the recorded image information. However, these virtual judgment lines can be displayed or set depending on the design and settings.
Moreover, line D shown in the example of FIG. 13 is a reference line D corresponding to the red lines (reference lines) 1101 and 1102 in the screen shown in FIGS. 11 and 12, and is preferably adjusted so as to overlap with the horizontal line.

When a rear vehicle is present in the area defined by the two reference lines 1301, 1302, the reference line D, and the judgment line C, the photographing device 101 determines that an appropriate distance is maintained between the vehicle 10 and the rear vehicle.

The control unit 403 of the photographing device 101 can use the first and second judgment times in addition to the reference line to judge dangerous driving. The control unit 403 of the photographing device 101 can set the first and second judgment times to fixed values, for example, 5 seconds. FIG. 27 described later shows an example in which the first and second judgment times are both set to 5 seconds.
For example, the control unit 403 of the photographing equipment 101 can further strengthen measures against aggressive driving by two-stage discrimination using a first judgment time and a second judgment time.

In another embodiment, the first and second determination times may be variable values. The variable values may be changed using a reference table or the like depending on, for example, the speed of the vehicle 10, the speed difference between the vehicle 10 and the vehicle behind, the distance or positional relationship between the vehicle 10 and the vehicle behind, the driving environment such as whether the vehicle is on a general road or a highway, etc.

For example, when a rear vehicle exists in a first determination area defined by the reference line and the determination lines B and C for a first determination time, the photographing device 101 determines that a rear approaching vehicle exists.
The control unit 403 can also change the distance between the judgment lines B and C, the area of the first judgment region and the first judgment time, for example by using a reference table, depending on the speed of the vehicle 10, the speed difference between the vehicle 10 and the rear vehicle, the distance and positional relationship between the vehicle 10 and the rear vehicle, the driving environment such as whether the road is a general road or a highway, etc.

Similarly, if a rear vehicle is present in a second judgment area defined by the reference line and judgment lines A and B for a predetermined second judgment time, the photographing equipment 101 determines that a vehicle approaching abnormally from behind is present.
The control unit 403 of the photographing equipment 101 can also change the distance between the judgment lines A and B, the area of the second judgment area and the second judgment time, for example by using a reference table, depending on the speed of the vehicle 10, the speed difference between the vehicle 10 and the rear vehicle, the distance and positional relationship between the vehicle 10 and the rear vehicle, the driving environment such as whether the vehicle is on a general road or a highway, etc.

As an example of rear vehicle approach judgment using the first judgment area and the second judgment area, when the vehicle 10 is traveling at a speed between 50 km/h and 80 km/h, if a rear vehicle is detected within the second judgment area for 5 seconds, a rear approach warning is issued, and if the vehicle is detected for another 5 seconds, i.e., a total of 10 seconds, a rear abnormal approach warning is issued and an event is recorded. Also, when the vehicle 10 is traveling at a speed of 81 km/h or faster, if a rear vehicle is detected within the first and second judgment areas for 5 seconds, a rear abnormal approach warning is issued and an event is recorded if the vehicle is detected for another 5 seconds, i.e., a total of 10 seconds.

In this way, it is possible to issue a warning that there is a rear vehicle in the area depending on the speed and approach distance, and by using the event recording setting described below, image information related to rear approach warnings and rear abnormal approach warnings can be stored in the memory section for the user terminal 102.
The control unit 403 of the front photographing equipment 101F performs the approach determination of a rear approaching vehicle by performing image processing and judgment on the video information received from the rear photographing equipment 101R, but if the rear photographing equipment 101R is configured to have a control unit 403, the rear photographing equipment 101R may also perform the approach determination of a rear approaching vehicle.

FIG. 14 shows an example of switching between display layouts 1420 and 1430 in accordance with the orientation of the screen of the user terminal 102. In FIG.
The example of Figure 14 shows switching between the display layout of a display area 1401 and an operation area 1402 when the screen of the user terminal 102 is in portrait orientation and the display layout of the display area 1401 and multiple operation elements (operation parts) 1403 to 1410 in the operation area when the screen of the user terminal 102 is in landscape orientation.

The display control module 312 of the user terminal 102 displays a display area 1401 and an operation area 1402 (1403 to 1410) on the output device 305 such as a screen (display) of the user terminal 102.
The operation area 1402 is an area made up of operation elements 1403 to 1410 for inputting operations for the image information recording module 310 and settings for the photographing device 101, for example.
The display area 1401 is an area for displaying, for example, image information transmitted from the photographing device 101. Note that various examples of the operation area and the display area are also shown in other drawings.

Depending on the orientation of the user terminal 102, the display control module 312 of the user terminal 102 switches between placing the display area 1401 and the operation area 1402 of the image information recording module 310 in separate areas on the screen of the user terminal 102, or placing the operation area superimposed on the display area.
More specifically, when the screen of the user terminal 102 is in portrait orientation, the display control module 312 displays the display area 1401 and the operation area 1402 in a display layout in which they are arranged in separate areas within the screen, and when the screen of the user terminal 102 is in landscape orientation, the display control module 312 displays the operation area 1402 in a display layout in which it is arranged on top of the display area 1401.

Furthermore, when the operation area is composed of multiple operation elements (operation parts) 1403-1410, the display control module 312 can also arrange each of the multiple operation elements 1403-1410 on the display area 1401, spaced apart from each other.

In particular, when an image is displayed when an event occurs, the parts that are important or of high interest to the user are often located in the center. Therefore, when the screen is in landscape orientation, the display control module 312 can display the individual operation elements 1403-1410 arranged along the four sides of the screen, as in the example of Figure 14, thereby maximizing the visible area and maximizing the visibility of the important or highly interesting parts.

FIG. 15 shows an example of a preview screen 1500 of the user terminal 102 .
15 shows a preview screen of the image information recording module 310 of the user terminal 102 when the output device 305 such as a display is in portrait orientation. In this preview screen, the display control module 312 can change the display layout of the display area 1510 and the operation areas 1520 and 1530 according to the orientation of the screen of the user terminal 102, similar to FIG.

The display control module 312 can call up and display a setting screen or the like of the image information recording module 310 from a preview according to the example of FIG.
Furthermore, when the user terminal 102 is connected to a plurality of photographing devices 101, the display control module 312 can receive settings regarding which photographing device 101 the image from which is to be displayed on the preview screen.

The display control module 312 can also display images from multiple image capture devices 101 (101F, 101R) on the output device 305 in picture-in-picture format. For example, if the image capture device 101F is a front camera and the image capture device 101R is a rear camera, the display control module 312 can display image information from the rear camera in the entire display area and display image information from the front camera superimposed on a portion of the image display from the rear camera.

The image information displayed on the output device 305 can be switched in various ways, for example, as described below.
- The front photographing equipment 101F aggregates image information from the front photographing equipment 101F and the rear photographing equipment 101R and transmits it to the user terminal 102, and the display control module 312 of the user terminal 102 selects one of the image information captured by the front photographing equipment 101F and the rear photographing equipment 101R and transmitted from the front photographing equipment 101F, and switches the displayed image.
The front photographing device 101F and the rear photographing device 101R each transmit image information to the user terminal 102, and the display control module 312 of the user terminal 102 selects either the image information captured by the front photographing device 101F and transmitted from the front photographing device 101F, or the image information captured by the rear photographing device 101R and transmitted from the rear photographing device 101R, and switches the displayed image.
The front photographing device 101F aggregates image information from the front photographing device 101F and the rear photographing device 101R, and the front photographing device 101F executes image switching processing. In other words, the front photographing device 101F selects one of the image information photographed by the front photographing device 101F and the rear photographing device 101R, transmits the selected image information to the user terminal 102, and the user terminal 102 displays the transmitted image information.
Furthermore, the display control module 312 may use the above configurations depending on the event recording settings described below.

For example, the display control module 312 displays the camera image (front, rear, or picture-in-picture) selected by the user on the preview screen, but can also forcibly switch to the rear camera image if approaching from behind is detected.

The three buttons 1501 to 1503 arranged at the bottom of FIG. 15 will be described.
Data confirmation button 1501: displays a confirmation screen for files in the auxiliary storage device 302 of the user terminal 102.
One-touch record button 1502: Saves the image sent from the image capturing device 101 and displayed on the preview screen.
Settings button 1503: Calls up a screen for making various settings. Examples of settings screens that can be called up directly or indirectly from the settings button 1503 will be described in more detail in conjunction with FIGS.

The four buttons 1504 to 1507 arranged at the top of FIG. 15 will now be described.
Screen switching button 1504: Switches between a plurality of image capturing devices 101. For example, the screen is switched in the order of front camera → rear camera → picture-in-picture.
Snapshot button 1505: Saves the image transmitted from the photographing device 101 and displayed on the preview screen as a still image.
Power saving mode button 1506: Decreases the brightness of the screen of the user terminal 102. In particular, darkens the screen of the user terminal 102 while keeping it displayed.
Microphone on/off button 1507: Switches whether or not audio is to be added to the data stored in the photographing device 101.

The display control module 312 has a display area and an operation area on the output device 305 of the user terminal 102 .
The photographing device cooperation module 313 can display the image from the photographing device 101 in the display area of the output device 305 of the user terminal 102. In addition, the photographing device control module 314 can receive setting input for the photographing device 101 from the user via a plurality of operation elements displayed in the operation area.
Note that the input from the user using the multiple operation elements displayed in the operation area may be a setting input related only to the user terminal 102 .

FIG. 16 shows an example of an operation for displaying setting screens 1601 and 1602 of the photographing device 101 from the preview screen.
16, the user terminal 102 can provide the user with the possibility of setting the photographing device 101 in a simple and intuitive manner. In particular, the user terminal 102 can provide the user with the possibility of setting the photographing device 101 via the setting screens 1601 and 1602, if only the settings for various modules of the user terminal 102.

FIG. 17 shows an example of setting menus 1702 to 1704 that are displayed when the "Smartphone event record setting" menu shown in FIG. 16 is selected.
In the example of FIG. 17, when "Smartphone event record settings" is selected from the "System settings" menu 1701, the display control module 312 displays a setting menu 1702 for "Smartphone event record selection" and "Smartphone event record settings."
The “Smartphone Event Recording” menu provides options for events that can be recorded on the user terminal 102 .
In the "smartphone event record setting" setting menu 1703, the image information recording module 310 accepts the selection input of an event to be saved. In the "smartphone event record setting" setting menu 1704, "rear abnormal approach record" is set to ON.

The communication control unit 307 of the user terminal 102 can receive image information from the photographing device 101 with which a connection has been established by the photographing device cooperation module 313 via the network 104 before an event occurs or immediately after the event occurs.
The image information recording module 310 records image information of the selected event based on the "smartphone event recording selection."

In the example of FIG. 17, the options displayed in the "Smartphone event record selection" menu are "G-sensor record", "One-touch record", and "Rear-end abnormal approach record".
An example of an event related to the "G sensor" is an event in which the G sensor detects an impact of a certain level or more, such as a collision of the vehicle 10.
An example of an event related to "one-touch recording" is an event caused by a button operation by the user.
An example of an event related to the "abnormal rear approach record" is an event in which a rear vehicle is detected for 10 seconds while driving. More specifically, an example of an event in which a rear vehicle is present in the first or second determination area for a certain period of time, for example, 10 seconds, as already described with reference to FIG. 13 is an event.

It is also possible to add other events. For example, the "Smartphone Event Record Selection" menu may also include various types of dangerous driving as candidates for events to be automatically saved, such as "abnormal approach from ahead," "sudden approach (sudden acceleration of the vehicle behind, sudden braking of the vehicle in front)," "dangerous cutting in close to the side," "dangerous cutting in," "excessive honking," "abnormal passing/high beams," "sudden braking of the vehicle in front," and "obstruction of overtaking by the vehicle in front."

For example, for each event set as dangerous driving in the photographing device 101, the image information recording module 310 can individually receive an ON/OFF setting for automatic saving.
In this specification, the term "automatic saving" refers to saving of image information without requiring the user to operate the user terminal 102 during or after the occurrence of an event.

FIG. 18 is an example of a screen 1800 on which events are displayed.
In the example of Figure 18, the user terminal 102 activates an image information recording module 310 for recording video from the photographing equipment 101, and an example of image information is shown that is displayed on the screen of the user terminal 102 when the photographing equipment 101 detects the occurrence of a set or selected event.

In the image screen with the text "image of being tailgated" in the figure, an image showing a state where the vehicle is being tailgated by a vehicle behind is displayed, for example, transmitted from the photographing device 101R that actually photographed the event. For example, when another image information from the other photographing device (another photographing device, front camera) 101F is displayed in the display area of the preview screen, the display control module 312 switches the image displayed in the display area of the preview screen to the image transmitted from the photographing device (rear camera) 101R that photographed the event.
To clearly indicate that event recording is being performed, the user terminal 102 may display a screen such as that shown in FIG.

When an event occurs for which the recording setting is on, the display control module 312 of the user terminal 102 can display the corresponding event on the output device 305 of the user terminal 102 without requiring any operation by the user. When the user terminal 102 is in an appropriate position, for example, in the driver's line of sight while driving, the user terminal 102 can provide the driver with an opportunity to check video related to the event without requiring any operation by the driver himself.

When an event such as rear-end collision occurs, it is extremely useful and important from the perspective of driving safety for users, especially drivers, to be able to check information about the event without having to perform any additional operations themselves.

FIG. 19 shows an example of a setting screen 1900 of the user terminal 102 .
In the example of FIG. 19, a setting screen 1900 displays a selection menu of the image information recording module 310 relating to "Video" and "System".
The image information recording module 310 can execute “smartphone event recording settings” and “smartphone APP screen settings” for a system for recording images from the photographing device 101 .

When the display control module 312 accepts the selection of "Smartphone event record settings", the setting screen 1900 transitions to, for example, the setting screen 3100 of "Event storage selection" and "Event record settings" in FIG. 31. Also, when the display control module 312 accepts the selection of "Smartphone APP screen settings", the setting screen transitions to, for example, the setting screen 2000 of "Smartphone APP screen settings" in FIG. 20.

The display control module 312 can present the user with the possibility of setting the recording target, recording time, image display time, and the like of the image information from the photographing device 101 on these setting screens 2000 and 3100 .
The setting screens 2000 and 3100 will be described later with reference to FIGS.

FIG. 20 shows an example of a setting screen 2000 for two types of screen settings.
20, the display control module 312 of the user terminal 102 displays a "smartphone APP screen settings" menu. The "smartphone APP screen settings" menu has "camera video display time" and "power saving screen transition time."

The "camera image display time" is the length of time for which the image received from the photographing device 101 is displayed on the preview screen of the user terminal 102. The "power-saving screen transition time" is the length of time from the start-up of the image information recording module 310 until the entire screen of the user terminal 102 is switched to a state with less power consumption, for example, until the entire screen is made invisible.
In FIG. 20, the camera image display time is set to 1 minute, and the power saving screen transition time is set to 30 seconds.

FIG. 21 shows an example of a camera image display time selection screen 2100.
21, the display control module 312 can accept a user's selection of a desired camera image display time from among 30 seconds, 1 minute, 2 minutes, 5 minutes, 10 minutes, 30 minutes, and continuous display. In another embodiment, it is also possible to accept a direct input of a camera image display time by the user himself.

This camera image display time may correspond to the first set time of the display stop flow 100 in FIG. 10. When the selected camera image display time has elapsed, the display control module 312 stops displaying the image information from the photographing device 101 on the screen of the user terminal 102. If the user selects continuous display, the image from the photographing device 101 is displayed continuously on the preview screen of the user terminal 102.

FIG. 22 shows an example of a power saving screen transition time selection screen 2200.
22, the display control module 312 can accept a user's selection of a desired power saving screen transition time from among 30 seconds, 1 minute, 2 minutes, 5 minutes, 10 minutes, 30 minutes, and continuous display. In another embodiment, it is also possible to accept a direct input of a camera image display time by the user.

This power-saving screen transition time may correspond to the second set time of the display stop flow 100 in FIG. 10. When the selected power-saving screen transition time has elapsed, the display control module 312 stops displaying the entire screen of the user terminal 102, or reduces the brightness of the entire screen (keeps the screen displayed but darkens it). In this case, the "reducing brightness" process also includes making the screen completely black. If the user selects continuous display, these transitions are not made, and the image from the photographing device 101 is continuously displayed on the preview screen of the user terminal 102.

The options shown in FIG. 21 and FIG. 22 may increase or decrease in number depending on the selection result of one of the options. For example, if the user selects 5 minutes in the example of FIG. 22, the options shown in the example of FIG. 21 may only be 30 seconds, 1 minute, 2 minutes, and continuous display.

When the image information recording module 310 receives settings relating to the “camera image display time” and the “power saving screen transition time”, it stores each setting as setting information 320 in the auxiliary storage device 302 .
The display control module can call up the saved setting information regarding the "camera image display time" and "power saving screen transition time" in order to execute the display stop flow 100 of FIG.

FIG. 23 shows examples of screen displays 2301 and 2302 before and after the camera image display time has elapsed.
23 , after the camera video display time has elapsed, the user terminal 102 displays a screen display 2302 of image information that consumes less power for screen display than the image information from the photographing device 101, instead of a screen display 2301 of image information from the photographing device 101. For example, after the camera video display time has elapsed, the display control module 312 stops displaying the image information from the photographing device 101 in the display area, and turns the display black.
The display control module 312 can also add text information such as "Video Display" to the image information with lower power consumption, as shown in Fig. 23. The user terminal 102 can also display a notification screen that is more intuitively recognizable by using marks, symbols, images, or the like other than text.

The display control module 312 can switch from a screen display 2302 of image information consuming less power to a screen display 2301 of image information from the photographing device 101 in response to a trigger from the photographing device 101 or a user tapping on the screen.
In the example of Fig. 23, the display control module 312 can reduce power consumption associated with displaying an image from the photographing device 101 in the display area 2304 while still providing the user with the possibility of inputting using the operation area 2303. Note that the user terminal 102 can record a corresponding event when the event occurs in any of the screen states of Fig. 23.

FIG. 24 shows examples of screen displays 2401 and 2402 before and after the power-saving screen transition time has elapsed.
24, after the power-saving screen transition time has elapsed, the user terminal 102 changes not only the image information from the photographing device 101 but also the screen display 2401 of the user terminal 102 to a screen display 2402 in a state in which power consumption is lower, for example, in which the brightness of the entire screen is reduced. It is particularly preferable that the user terminal 102 turns off the display of the entire screen of the user terminal 102, thereby reducing power consumption related to the screen display of the user terminal 102 as much as possible.

The display control module 312 can also return the entire screen display in the lower power consumption state to the preview screen in FIG. 15 or the screen displayed immediately before transitioning to the screen in the lower power consumption state, in response to a trigger from the photographing device 101 or a user tapping on the screen.
In the example of Fig. 24, the user terminal 102 can reduce power consumption related to screen display as much as possible. Note that the user terminal 102 can record a corresponding event when an event occurs in any of the screen states of Fig. 24.

FIG. 25 shows an example of selection screens 2502 to 2504 for events to be warned about.
In the example of Figure 25, the display control module 312 of the user terminal 102 displays a "Settings" menu 2501 on the output device 305 of the user terminal 102 having the options "Video Resolution,""Settings," and "Active Safety," and accepts the user's selection of "Active Safety" by tapping, and transitions to an "Active Safety settings screen" 2502 having the options "Front reference position setting,""Rear reference position setting,""Forward vehicle distance warning,""Departure delay warning,""Lane departure warning,""Rear approach warning,""Rear abnormal approach warning," and "Rear abnormal approach record."

On the “Active Safety setting screen” 2502, the photographing device control module 314 of the user terminal 102 can accept settings for the safe driving support function (Active Safety) of the photographing device 101 with which a connection has been established by the photographing device cooperation module 313.
The safe driving support function is a function executed by the image capturing device 101 for the purpose of assisting the driver in making decisions and reducing the driving burden.

On the "Active Safety Settings Screen", each of the safe driving support functions, "Forward vehicle distance warning", "Departure delay warning", "Lane departure warning", "Rear approach warning", "Abnormal rear approach warning", and "Abnormal rear approach record", can be individually set to ON or OFF. For example, a warning for a safe driving support function that is set to ON can be output by at least one of the image capture device 101 or the user terminal 102.

In the selection screen 2503, all event warnings and rear abnormal approach record are set to OFF, and in the selection screen 2504, all event warnings and rear abnormal approach record are set to ON.
When the "rear abnormal approach recording" shown in Figure 25 is set to ON, the image information recording module 310 of the user terminal 102 can record image information of, for example, tailgating from behind in the auxiliary memory device 302 when the vehicle 10 satisfies the event recording conditions, in accordance with the image information recording flow 80 of Figure 8.

In addition to the “Rear Abnormal Approach Record,” the “Active Safety Settings Screens” 2502-2504 may have options for recording different events, such as, for example, “Front Abnormal Approach Record,” “Rapid Approach Record,” “Side Abnormal Approach Record,” “Abnormal Cut-In Record,” and “Overtaking Obstruction Record.”
In this case, the image information recording module 310 can also receive a priority setting for a plurality of events, regarding which event is to be given priority in recording when those events occur simultaneously.

FIG. 26 shows an example of the response of the photographing device 101 and the indicator display when rear approach is detected.
In the example of Fig. 26, when the front photographing device 101F detects the approach of a vehicle from behind based on the image captured by the rear photographing device 101R, the photographing device 101F emits a "beep" sound effect and blinks an indicator attached to the photographing device 101F. The sound effect and the display of the indicator can be set based on the recording settings of the photographing device 101F or the user terminal 102, the type of event, etc., and in this embodiment, the recording lamp indicating recording is blinked quickly in blue.

For example, when the photographing device 101F recognizes that a vehicle is approaching from behind, in addition to or instead of the photographing device 101F making the above-mentioned response and displaying an indicator, the photographing device 101R at the rear may make a similar response and display an indicator.

In addition, in the configuration of FIG. 26, an image of the photographing equipment 101R at the rear is transmitted to the photographing equipment 101F at the front, and the photographing equipment 101F at the front performs event determination such as proximity, but the configuration may be such that this event determination is performed by the photographing equipment 101R at the rear.
In addition, if the front photographing device 101F outputs a warning, for example, by voice, when a vehicle approaches from the front, and if the rear photographing device 101R outputs a warning, for example, by voice, when a vehicle approaches from the rear, the driver and passengers can easily recognize that an oncoming vehicle is approaching from the direction of the warning sound.

These warnings enable the photographing device 101 to notify the user, for example by outputting sound or light from the photographing device 101, that recording of the corresponding event has begun on the user terminal 102 or the photographing device 101.

FIG. 27 shows an example of the response of the photographing device 101 and the indicator display when rear approach and abnormal rear approach are detected.
27, when the photographing device 101 continues to detect the approach of a vehicle from behind for five seconds, the photographing device 101 issues a voice warning saying, "Watch your rear." At this point, the user terminal 102 has not yet started event recording.

After that, if the photographing device 101 continues to detect the approach of a vehicle from behind for another 5 seconds, the photographing device 101F issues a voice warning saying "Dangerous driving from behind has been detected" and changes the display of the indicator to a warning state. The photographing device 101 also changes the display of the indicator to a recording state, and the user terminal 102 starts recording the event.
26, when an event is detected, the user terminal 102 can also notify the event by sound, light, screen display, etc., in addition to the photographing device 101. For example, instead of a sound warning, the user terminal 102 can output an event sound such as a "beep."

In the example of Figure 27, the user terminal 102 starts event recording after the second warning (abnormal rear approach warning), but in another embodiment, the user terminal 102 may perform event recording both after the first warning (rear approach warning) and after the second warning (rear approach warning).
In still another embodiment, the first warning (rear approach warning) may be omitted, and a warning and event recording may be performed at the timing of the abnormal rear approach warning.

In yet another embodiment, when the warning module 311 of the user terminal 102 detects an event that is the subject of a warning in Figures 26 and 27, it may provide a notification similar to the warning provided by the photographing device 101, for example, by displaying a screen using sound, light, text, etc.
Furthermore, the photographing device cooperation module 313 of the user terminal 102 can set for each event and for each photographing device 101 which event the notification should be sent for.

FIG. 28 shows an example of a selection screen 2800 for a terminal storage unit.
In the example of FIG. 28, the image information recording module 310 can accept a user's selection from the options of "folder in smartphone" and "folder in microSD" shown in the "smartphone event save destination" menu.

However, the "Smartphone Event Storage Destination" menu may have another auxiliary storage device 302 as an option. The other auxiliary storage device 302 may be, for example, a cloud storage device connected to the user terminal 102 via the network 104. Furthermore, in another embodiment, the image information recording module 310 may also accept the user's own addition of a storage destination for events.

FIG. 29 shows an example of a selection screen 2900 for selecting the storage time of the image information received from the photographing device 101.
In the example of FIG. 29, the "smartphone event recording time" can be selected from 10 seconds, 20 seconds, 30 seconds, and 60 seconds.

In another embodiment, the user can input and set the recording time. For example, if the SNS service to which the user is registered has set a maximum video duration for uploaded video, the image information recording module 310 may record the event at that preset duration. This allows the user terminal 102 to record image information that meets the requirements of the upload destination without the user having to process the image information during or after the event occurs.

Furthermore, the selection screen 2900 in FIG. 29 may present as options suitable recording times for various SNS services, or suitable recording times designated by public institutions to which evidence images are to be submitted, etc. By accepting such options from the user in advance, even if the user encounters an event such as dangerous driving, the user can record the data he or she desires in the auxiliary storage device 302 of the user terminal 102 without imposing the psychological burden of editing or checking the images of the dangerous driving.

The communication control unit 307 of the user terminal 102 can also upload image information stored in an appropriate format for the purpose of posting content to an external server 103 that provides SNS services, etc., reporting or submitting evidence to an external server of a public institution, etc., providing information or backing up data to an external server of a private company, etc.
In any embodiment, the user terminal 102 does not require the user to process the image information after or during the event in order to upload the image information. Also, the image information recording module 310 of the user terminal 102 can accept settings on the setting screen of the image information recording module 310 regarding which event is to be transmitted to which external server.

Furthermore, when uploading image information to the external server 103, the user terminal 102 may upload the image information by adding index information, for example, as described in FIG. 7. It is particularly advantageous for the user terminal 102 to perform processing such that at least a portion of the index information can be confirmed on the image information before uploading the image information.

As a result, even if the user terminal 102 or the photographing device 101 is damaged and it becomes difficult to transfer data after the event, the user terminal 102 can reliably save, submit, and publish image information of important events and the circumstances under which the event occurred, without the user having to operate the user terminal 102 during or after the event.

In another embodiment, when the user terminal 102 uploads image information to an external server, the user terminal 102 may add information stored in the auxiliary storage device 302 of the user terminal 102 as setting information 320 to the image information to be uploaded. The added setting information 320 may be information on physical characteristics such as height, weight, blood type, medical examination history, medications taken, allergies, etc. of the user who owns the user terminal 102 and his/her family, friends, or passengers, or may be emergency contact information, etc.
This allows the user terminal 102 to add information previously stored in the user terminal 102 to the image information captured by the image capturing device 101 and upload the image information to an external server.

For example, if the vehicle 10 falls off its wheels, image information about the fallen wheel can be sent to a server of an organization that provides road services to which the driver subscribes, along with index information consisting of GPS coordinates indicating the current location of the vehicle 10.

The image information recorded on the user terminal 102 is event record image information triggered by the occurrence of an event, so the user does not have to process the image information and can easily create impactful image information, which can be sent from a smartphone to an external server and posted. For example, it is possible to post video suitable for publication on a social networking service (SNS). Event record video is impactful and short in length, making it suitable for SNS.

FIG. 30 shows examples of display screens 3001 and 3002 in which image information received from the photographing device 101 is being stored.
During event recording, the corresponding event video is displayed on the display screen.
In the example of FIG. 30, both pieces of image information received from two photographing devices 101 (for example, the front photographing device 101F is called the front camera, and the rear photographing device 101R is called the rear camera) are displayed in a picture-in-picture format. As already described with reference to FIG. 15, the display of the picture-in-picture image can be switched, for example, from a front camera image to a rear camera image to a picture-in-picture image in which an image from the rear camera is displayed within an image from the front camera to a picture-in-picture image in which an image from the front camera is displayed within an image from the rear camera, by tapping a screen switching button. Various combinations of switching patterns are possible depending on the number and type of photographing devices 101, and the user himself can also set a desired pattern.

For example, when the user terminal 102 receives image information from four photographing devices, the screen can be divided into four parts for display. Alternatively, the user terminal 102 can display only image information related to the event.
To notify that an event recording is actually being performed, the user terminal 102 can display notification information such as "Event recording in progress" or "Steam recording" along with a countdown of the set event recording time. For example, in FIG. 30, a display screen 3001 displaying the text information "Event recording in progress 18" notifies that 18 seconds of event recording remain. Also, a display screen 3002 displaying the text information "Steam recording 28" notifies that 28 seconds of event recording remain.

The screen of the user terminal 102 can also display a notification screen that is more intuitively recognizable by using marks, symbols, images, etc. other than text. In addition to displaying a notification screen on the screen, the user terminal 102 can also output corresponding voices, sound effects, etc.

FIG. 31 is an example of a settings screen 3100 for event storage selection and event recording settings.
In the example of Fig. 31, the user terminal 102 displays two items, "Select smartphone event record" and "Select smartphone event record" on the screen. The "Select smartphone event record" menu is the same as that described in relation to Fig. 17, and therefore a description thereof will be omitted.

The "Smartphone event recording selection" menu has the options "Smartphone event recording time," "Smartphone event overwrite," and "Smartphone event save destination." The image information recording module 310 accepts the user's selection of these options.

In the example of FIG. 31, the image information recording module 310 accepts the selection of 30 seconds for "smartphone event recording time," ON for "smartphone event overwrite," and the smartphone itself for "smartphone event save destination."
The image information recording module 310 can provide the user with very concise options via the “Smartphone Event Recording Selection” menu and the “Smartphone Event Recording Selection” menu regarding the recording settings for individual events sent from the photographing equipment 101 and the method of recording in the auxiliary storage device 302 for the user terminal 102.

FIG. 32 shows an example of a notification screen 3200 regarding a shortage of capacity of the auxiliary storage device 302 .
In FIG. 32, the screen of the user terminal 102 displays a notification screen having text information stating, "Event cannot be recorded because the smartphone's storage capacity is insufficient," but the content of the text information can be set as appropriate.

The screen of the user terminal 102 can also display a more intuitively recognizable notification screen using marks, symbols, images, etc. other than text. In addition to displaying a notification screen on the screen, the user terminal 102 can also output voices, sound effects, etc. corresponding to the notification screen.
By displaying such a notification screen, the user terminal 102 can promptly and reliably notify the user of the operating status of the photographing device 101 .

FIG. 33 is an example of a notification screen 3300 for notifying that the photographing device 101 is accepting an operation via an operation element provided on the photographing device 101 .
The operation elements include, for example, buttons, switches, slide knobs, rotary knobs, etc., provided on the photographing devices 101F and 101R, a touch panel type operation screen, and buttons, icons, etc. displayed thereon.
In Figure 33, the screen of the user terminal 102 displays a notification screen 3300 having text information (predetermined display information, information related to the operational status of the photographing equipment) saying "Photographing equipment in operation," but the content of the text information can be set as appropriate.
The text information may be information notifying the process being executed by the photographing device 101, such as "SD card formatting."

While the notification screen 3300 is displayed, the display control module 312 restricts input of operations to the photographing device 101 through the operation area of the user terminal 102. In another embodiment, while the notification screen 3300 is displayed, the display control module 312 may always be able to accept some operations, such as operation elements in the operation area that do not conflict with operations of the photographing device 101, for example, operations that can only be operated through the user terminal 102. In addition, the display control module 312 may not accept operations for only a part of the period during which the photographing device 101 is being operated.

The screen of the user terminal 102 can also display a more intuitively recognizable notification screen using marks, symbols, images, etc. other than text. In addition to displaying a notification screen on the screen, the user terminal 102 can also output voices, sound effects, etc. corresponding to the notification screen.
By displaying such a notification screen, the user terminal 102 can promptly and reliably notify the user of the operating status of the photographing device 101 .

FIG. 34 is a diagram showing two examples of the external appearance of the photographing device 101F (A) and (B).
In the example of the external appearance (A) in Fig. 34, the photographing device 101F is designed as a drive recorder 3401 for photographing the front of the vehicle 10. The drive recorder 3401 is shown as seen from the front side of the vehicle 10 through the windshield. The drive recorder 3401 is connected to a cable 3402 for electrically connecting to the photographing device 101R at the rear, for example, via a connection part 3402a. This cable 3402 corresponds to the cable 11 in Fig. 1, and is connected to the photographing device 101R at the rear on the opposite side to the drive recorder 3401.

The housing of the photographing device 101F is generally columnar (or cylindrical). An imaging lens 3404 of the photographing device 101F (drive recorder 3401) is disposed on the vehicle direction side of the photographing device 101. On the housing of the photographing device 101F, LEDs for notifying various information, operation buttons, and a microphone hole are disposed on the opposite side of the position where the imaging lens 3404 is disposed.
The drive recorder 3401 is further connected to a cable 3403 for connecting to a multi-battery or a power supply unit with a voltage monitoring function via a connection section 3403 a, but can function without the cable 3403 .

34B, the photographing device 101F is also designed as a drive recorder 3411 for photographing the area ahead of the vehicle 10. The drive recorder 3411, the cable 3412, and the connection section 3412a have configurations similar to those in the external view (A), and therefore description thereof will be omitted.
As is clear from a comparison with the example of the appearance (A) in FIG. 34, various variations in the appearance design of the photographing device 101 are possible.

[Configuration of Image Recording Apparatus]
FIG. 35 is a diagram showing another example of the appearance of the image capturing device 101, which is a drive recorder with an LCD as the display unit described above.
The operation unit 3515 is one of the user interfaces for inputting various instructions to the photographing device 101. As an example of the operation unit 3515, as shown in Fig. 35, there are various operation buttons arranged on a predetermined surface of the housing 3524. Details will be described later.

The display unit 3516 is, for example, a liquid crystal display (LCD). The control unit 403 controls the display surface of the display unit 3516 to display, in real time, image data captured by the image capture unit (camera) 402, or to read and display image data recorded in a storage device such as an SD card 406.

The power supply 411 receives power from an external source and supplies a predetermined power supply voltage to devices and equipment within the photographing device 101. The power supply 411 includes a DC jack 3531 for attaching a power cable for connecting to an external power source, for example.

As shown in FIG. 35, the external configuration of the photographing device 101 includes a flat, approximately rectangular housing 3524. The housing 3524 stands upright when attached to a predetermined position on a vehicle, such as a car. In the normal position, a lens 3542 of the built-in photographing unit (camera) 402 is disposed on a front surface 3541 located forward in the direction of travel of the vehicle. The photographing unit (camera) 402 photographs the area ahead in the direction of travel of the vehicle. Above the lens 3542 on the front surface 3541 of the housing 3524, a through hole 3581 for a speaker 409 built into the housing 3524 is provided. In addition, below the lens 3542 on the front surface 3541 of the housing 3524, a through hole 3571 for a microphone built into the housing 3524 is provided.

A joint rail 3523 and a camera jack 3521 are provided on the upper surface 3544 of the housing 3524. One end of a bracket, which is an example of an attachment member not shown, is removably attached to the joint rail 3523. The other end of the bracket serves as an attachment part for attachment to, for example, a predetermined position above the windshield of the vehicle, the vehicle's rearview mirror, or the ceiling inside the vehicle cabin. This attachment part is affixed and fixed to, for example, the windshield of the vehicle, which is an example of an attachment part. This positions the photographing device 101 at a predetermined position inside the vehicle cabin. Furthermore, when the photographing device 101 is placed at the predetermined position and placed in the upright position as described above, the front surface 3541 faces the windshield of the vehicle, and the photographing unit (camera) 402 photographs the area in front of the vehicle.

The camera jack 3521 is used to connect one end of a camera cable (not shown). The other end of the camera cable is connected to another external imaging device. The external imaging device may, for example, be one that captures an area different from the imaging area of the imaging unit (camera) 402, and in this embodiment, a cable for communicating with the imaging device 102R is connected to the camera jack 3521. The camera jack 3521 may be compatible with, for example, the USB Type C standard, and may be a terminal for communicating with the imaging device 101R according to the Ethernet (registered trademark) standard.

The rear surface of the housing 3524 is provided with a display surface of the display unit 3516, an operation unit 3515, and a power lamp. In the example shown in FIG. 35(b), the operation unit 3515 has a plurality of operation buttons arranged vertically in the area to the right of the display surface of the display unit 3516. In this embodiment, four operation buttons are arranged from top to bottom, namely, a first operation button 3501, a second operation button 3502, a third operation button 3503, and a fourth operation button 3504.

The first operation button 3501 is a button for switching the image displayed on the display surface of the display unit 3516, and a button for instructing the formatting of a storage device such as the SD card 406. As described above, the photographing device 101 has a function of connecting a separate external photographing device (not shown) using a camera cable connected to the camera jack 3521, and displaying image data captured by the external photographing device on the display surface of the display unit 3516. When the first operation button 3501 is pressed briefly, the control unit 403 switches the display between an image captured by the photographing unit (camera) 402 built into the photographing device 101 and an image captured by the external photographing device. When the first operation button 3501 is pressed for a long time, the control unit 403 executes the formatting of a storage device such as the SD card 406.

The second operation button 3502 is a button that displays a selection screen for selecting image data to be played. When the second operation button 3502 is pressed, the control unit 403 displays a selection screen for selecting and playing video data recorded on the SD card 406.

The third operation button 3503 is a button that displays a setting menu. This setting menu is hierarchical, and by appropriately operating each operation button, a lower-level menu screen is displayed, and various settings can be made on the desired setting screen.

The fourth operation button 3504 is a button for starting/stopping recording. For example, when the fourth operation button 3504 is pressed briefly during recording using the continuous recording function, the control unit 403 detects this and pauses recording, and when the fourth operation button 3504 is pressed briefly during the pause, the control unit 403 resumes recording. In addition, when the fourth operation button 3504 is pressed for a long time, the control unit 403 changes the frame rate during recording.

A DC jack 3531 and a recording medium insertion port 3525 are provided on the right side surface 3543 of the housing 3524. The DC jack 3531 is used to supply power to the photographing device 101. For example, a cigarette plug cord connector can be connected to the DC jack 3531, and the cigarette plug cord can be connected to a cigarette lighter socket of a vehicle to receive power. It is advisable to use an OBD adapter that can be connected to an OBDII connector of a vehicle instead of the cigarette plug cord.

The OBDII connector, also known as a fault diagnosis connector, is connected to the vehicle's ECU and periodically outputs various vehicle information. Therefore, by connecting this OBDII adapter to the OBDII connector on the vehicle body and connecting the connector attached to the end of the OBDII adapter's connection cord to the DC jack 201, power is supplied to the imaging device 101 and the control unit 403 periodically acquires various vehicle information. The recording medium insertion port 3525 is an insertion port for inserting an SD card, which is an example of a recording medium. A reader/writer is mounted at the back of this recording medium insertion port 3525, making recording media such as the SD card 406 removable from the reader/writer.

A fifth operation button 3505, which is an example of the operation unit 3515, is provided on the left side surface 3545 of the housing 3524. This fifth operation button 3505 is an input button for manually instructing the start of recording, and when this fifth operation button 3505 is pressed, the control unit 403 may perform event recording to start recording image data captured by the imaging unit (camera) 402, etc., and sound collected by the microphone, etc.

Furthermore, the program according to the present invention causes the user terminal 102 to realize the above-mentioned functions of each module (each means) of the user terminal 102. Also, another program according to the present invention causes the photographing device 101 to realize the above-mentioned functions of each module (each means) of the photographing device 101.

[Other embodiments]
Furthermore, the embodiments of the present invention may be as follows: The embodiments may be appropriately combined.
(A) In one embodiment of the present invention, when both a warning from the photographing equipment 101 (in this embodiment, the photographing equipment 101F) and a warning from the user terminal 102 (more specifically, the warning module 311) are provided, a problem occurs in that when sound (e.g., a warning voice) is emitted from both the photographing equipment 101 and the user terminal 102, a sound for the same event may be emitted twice, with a time difference, by the photographing equipment 101 and the user terminal 102.

Therefore, it is preferable that both the photographing device 101 and the user terminal 102 emit sounds in response to the same event, but that the sounds (i.e., the notification contents) are different. Note that although event notification information is normally sent from the photographing device 101 to the user terminal 102, it may also be sent from the user terminal 102 to the photographing device 101. In this way, the number of sensors for detecting events in the photographing device 101 (for example, a G sensor such as an acceleration sensor) can be reduced, simplifying the configuration and reducing costs.

Both the photographing device 101 and the user terminal 102 may be configured to detect an event, send event notification information to each other, and record image information based on the event notification information received from another device (here, the other device).
In addition, both the photographing device 101 and the user terminal 102 each emit sound in response to the same event, exchange event notification information with each other, and record image information, etc. In this case, it is preferable that the device that detects the event emits a sound, and the device that has not detected the event and received event notification information from the other device does not emit a sound.

It is preferable to provide a function that allows the user to set which device, the photographing device 101 or the user terminal 102, will emit sound for each event. This setting function may be provided, for example, in the user terminal 102, which may then notify the photographing device 101 of this setting. At this time, it is preferable to provide a setting item so that when one of the devices is selected to emit sound, the other devices are automatically set to not emit sound.

For events detected by the photographing device 101, a notification (e.g., warning) threshold value set to a higher threshold value than the detection threshold value for the occurrence of the event (e.g., acceleration) may be provided on the user terminal 102, and the user terminal 102 may not notify an event that exceeds the event detection threshold value but does not exceed the notification threshold value. Both threshold values may be set from an application program installed on the user terminal 102, and the user terminal 102 may have a function for sending threshold value information to the photographing device 101.

It is preferable to have a function of automatically matching the set language for notifications of the photographing device 101 to the set language of the user terminal 102. The set language of the user terminal 102 can be set, for example, from an application program installed in the user terminal 102. In this way, it is possible to output sounds such as warnings from each device in a language that the user can understand, without having to set a set language for each device individually.
In this way, a warning function that is highly convenient for the user can be provided.

(B) In one embodiment of the present invention, the user terminal 102 may be equipped with the following functions related to voice recognition.
The photographing device 101 does not have a voice recognition function, or even if it has a voice recognition function, it has a function of recognizing a voice different from that of the user terminal 102. The photographing device 101 may have a function of performing processing in the photographing device 101 based on the result of voice recognition on the application program side of the user terminal 102.

Based on the results of the voice recognition, the user terminal 102 may generate a command to cause the photographing device 101 to execute a process, and transmit the command to the photographing device 101. An example of the process is event recording. In this case, when a predetermined phrase such as "Record it!" is recognized in an application program of the user terminal 102, event recording may be executed. In particular, the photographing device 101 or the user terminal 102 may record the image information with the recognized phrase attached as a tag. For example, if the photographing device 101 or the user terminal 102 recognizes "tailgating," it may attach the tag "tailgating" to the image information. In this way, for example, it is possible to search for image information using the tag as a search key.

Recently, price competition for packet communication fees has intensified for smartphones and the like, and constant tethering is no longer a problem. If a smartphone or the like is brought into a vehicle, it can be used as a router, so it is easy to use the voice recognition function in a cloud computing environment, for example. Therefore, by performing voice recognition on the user terminal 102 side, the voice is converted into a character string, and the character string is sent to the image capture device 101. Then, on the image capture device 101 side, the received tag is added to the image including the range at that time or immediately before, and the image information is recorded, so that the image with the tag added can be searched and displayed using a viewer or the like.

The present invention is not limited to the above-described embodiments, but includes various modified examples. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and are not necessarily limited to those having all of the configurations described. It is also possible to replace part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. It is also possible to add, delete, or replace part of the configuration of each embodiment with other configurations.

Furthermore, the above-mentioned configurations, functions, processing units, processing means, etc. may be realized in hardware, in part or in whole, for example by designing them as integrated circuits. Furthermore, the above-mentioned configurations, functions, etc. may be realized in software, by a processor interpreting and executing a program that realizes each function. Information on the programs, tables, files, etc. that realize each function can be stored in a memory, a storage device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or DVD.

In addition, the control lines and information lines shown are those that are considered necessary for the explanation, and not all control lines and information lines in the product are necessarily shown. In reality, it can be considered that almost all components are connected to each other.
The above-described embodiments disclose at least the configurations described in the claims.

The above-mentioned embodiment discloses at least the following configuration.
(1)
A terminal,
A communication control means for receiving a plurality of pieces of image information transmitted from a photographing device disposed in the vehicle;
an image information recording means for storing the plurality of pieces of image information received from the photographing device in a storage means for the terminal when the state of the vehicle satisfies a predetermined recording condition;
A terminal having the above configuration.
With this configuration, image information captured by the imaging device installed in the vehicle when the recording conditions are met can be recorded on the terminal without the user having to operate the terminal to instruct the recording. Therefore, for example, the user can use the terminal to check the image when the recording conditions are met without having to remove the recording medium such as an SD card attached to the imaging device and attach it to a PC, and display the image on a viewer installed on the PC, which is convenient for the user.
Even in the case of an event such as a rear-end collision that is difficult for the driver to directly visually confirm without using a mirror or monitor, the terminal can record image information related to the event in a storage device for the terminal without the driver having to operate the terminal or the photographing equipment after or during the event.
In addition, for example, since the driver does not need to operate the terminal himself to record image information, he or she can concentrate on driving and can respond to dangerous driving such as tailgating in a safer condition and with a more reassured or calm mind.
(2)
the terminal has a display means for displaying a screen and a display control means;
The communication control means sequentially receives the plurality of pieces of image information from the photographing device,
The display control means displays the received plurality of pieces of image information in a display area on the screen,
a terminal that stores the displayed plurality of pieces of image information in a storage means for the terminal when the recording condition is satisfied while the plurality of pieces of image information is being displayed;
With this configuration, the user can check the images of the vehicle surroundings captured by the image capturing device on a terminal at hand, for example, a smartphone, without operating the image capturing device. Also, the user can leave the image information of the image displayed when the recording conditions were satisfied in the storage unit for the terminal, without having to check the image capturing device again. For example, when a serious event such as aggressive driving or an accident occurs, the situation can be easily checked and displayed on a terminal such as a smartphone.
(3)
The image information recording means is a terminal that stores the plurality of pieces of image information when notification information notifying that the recording condition is satisfied is received from the photographing device.
By configuring the terminal in this manner, when the terminal receives notification information, the reception of the notification information is used as a trigger to record the image information received from the photographing device as image information that satisfies the recording conditions, allowing the user to easily check the image information later.
(4)
The display control means is a terminal that displays, on the screen, an operation area for operating the photographing device or the terminal.
With this configuration, the terminal can provide the user with operations for not only the terminal itself but also the photographing device via the screen, allowing the user to operate and set the terminal and the photographing device as if they were an integrated system.
(5)
The display control means is a terminal that stops displaying the display area after a first set time has elapsed, while continuing to display the operation area.
With this configuration, the terminal can reduce power consumption related to displaying the display area after the first set time has elapsed. Since the main power consumption related to the screen display is caused by displaying image information in the display area, the terminal can reduce the main power consumption related to the screen display while maintaining a state in which the terminal can accept operations and settings by the user via the displayed operation area. As a result, it is possible to prevent the occurrence of problems such as image information not being stored in the terminal storage unit due to the terminal running out of battery.
(6)
The display control means stops displaying the display area and the operation area after a second set time has elapsed.
With this configuration, the terminal can reduce power consumption related to the display of the display area and the operation area after the second set time has elapsed. When the user does not look at the terminal screen for a long period of time, such as while driving, the display of the operation area as well as the display area can be stopped, thereby enabling more effective power saving.
(7)
The display control means is a terminal that accepts changes to the first set time and the second set time.
With this configuration, the user can set the power consumption saving that is desired each time according to his/her own driving style, driving conditions, remaining battery capacity of the terminal, etc., through simple operations.
(8)
When the photographing equipment is operated using an operating element on the photographing equipment side, the display control means displays specified display information in the display area and restricts input of operations to the photographing equipment using the operation area of the terminal.
This configuration can prevent interference between the terminal and the photographing device when inputting operations to the photographing device. The photographing device is often placed in a position where the user's line of sight is not obstructed, such as near the top edge of the front windshield or rear windshield, and is difficult for the user to reach. This allows the user to easily check the operation status and processing status of the photographing device at hand, and is convenient because it does not require the user to directly operate the photographing device itself. For example, the user can easily and reliably know the status of the rear camera without having to move to check the status of the photographing device himself.
(9)
The display control means is a terminal that displays information related to an operating state of the photographing device as the predetermined display information.
With this configuration, the user can know on the terminal whether the photographing device is being operated by the operation element on the photographing device side. For example, when a passenger is operating the rear camera, the driver can check whether the operation is being performed as intended while remaining seated in the driver's seat.
(10)
A terminal in which the display control means switches between arranging the operation area and the display area in separate areas on the screen, or arranging the operation area overlapping the display area, depending on the orientation of the terminal.
By configuring the terminal in this way, the display area and the operation area can be easily switched in a state where the user can see and operate more easily according to the orientation of the screen. For example, when taking a screenshot, the user can easily obtain an event image from a photographing device displayed on the entire terminal screen by simply changing the orientation of the terminal. This is significantly more advantageous for the user in terms of effort, image quality, safety, etc. than the case where a tailgating image or the like displayed on the LCD screen of a conventional liquid crystal display (LCD)-equipped drive recorder is directly photographed with the camera of the terminal.
(11)
The operation area has a plurality of operation portions,
A terminal in which, when the display control means displays the operation area superimposed on the display area, the display control means arranges each of the multiple operation parts so as to be spaced apart from each other within the display area and superimposed on the display area.
With this configuration, when the operation area is displayed overlapping the display area, the display area can be displayed with higher visibility. Also, since the interval between the operation parts of the operation area is wider, the user feels less cramped in each operation, so that the terminal can provide the user with comfortable operability and good visibility at the same time.
Alternatively, displaying an operation area (operation portion) superimposed on a display area can significantly improve the degree of freedom in layout design and the degree of freedom in customization by the user.
(12)
The terminal has a photographing device linking means, and sets the photographing device based on inputs to the display area and the operation area.
By configuring in this way, the terminal can provide the user with the possibility of individual settings of the photographing device as well as the possibility of setting the terminal. For example, the photographing device may not have a high performance user interface compared to the terminal, and the photographing device itself may not have an output device such as a display, but the user can easily set various settings of the photographing device at hand via the display of the terminal.
(13)
The photographing device linking means is a terminal that sets a reference line in the photographing device for outputting that the state of the vehicle satisfies a predetermined recording condition based on an input to the operation area.
By configuring it in this manner, even if the photographing device itself is designed not to have an output device such as a display, the user can easily make detailed settings for the reference line of the photographing device at hand via the display of the terminal.
The user can set the settings in a state where it is easier to see or check (e.g., less affected by ambient light) or in a more comfortable position, compared to when using the screen of a photographing device fixed to a vehicle. In addition, since the terminal UI of a smartphone or the like is often more complete than that of a photographing device, the operability of the user is improved.
(14)
The photographing equipment determines at least one of the distance or positional relationship to at least one of a vehicle in front of the vehicle or a vehicle behind the vehicle based on the set reference line, and notifies the terminal that the state of the vehicle meets specified recording conditions.
Approaching other vehicles is one of the circumstances that is particularly necessary to record because it often causes trouble or accidents, and by configuring in this way, the terminal can provide a notification according to the judgment based on the performance of the imaging device. Also, the user can not only receive notifications from the terminal, but also easily recognize that image information of the event related to such notifications has been recorded.
(15)
When the state of the vehicle satisfies a specified recording condition while another image information from another photographing device is being displayed in the display area, the display control means switches the image information to be displayed in the display area to the image information from the photographing device that has detected that the state of the vehicle has satisfied the specified recording condition.
With this configuration, the terminal can display image information that is problematic or of great interest to the user on the screen. For example, even in the case of an event such as a rear-end collision that is difficult for the driver to directly check visually without using a mirror or monitor, the user can check image information related to the event on the terminal at hand without having to operate the terminal individually each time, which can provide the user with good driving safety and a sense of security.
(16)
The display control means is capable of displaying images from a camera in front of the vehicle and images from a camera in rear of the vehicle, and if the approach of another vehicle from behind the vehicle is detected while the image from the camera in front of the vehicle is being displayed, the terminal switches to and displays the image from the camera in rear of the vehicle.
By configuring in this way, the terminal can record image information related to rear-end tailgating in the storage device for the terminal, even if the rear-end tailgating is difficult for a user such as a driver or a passenger to directly visually confirm without using a mirror or a monitor, without the user operating the terminal or the photographing device after the rear-end tailgating occurs or while the rear-end tailgating occurs. The user can check the rear-end tailgating detected with high accuracy on the screen display at hand while keeping his/her attention on the road while driving, without continuously checking the behavior of the following vehicle suspected of rear-end tailgating, and can further save the image information in the terminal. With conventional terminals, especially when there is only one driver in the vehicle, even if the driver drives while paying attention to the rear and recognizes the rear-end tailgating early, it is difficult to record the image information on the terminal at hand on the spot. By using this terminal, the user does not need to pay more attention to the rear than necessary, nor does he/she need to operate to start recording the image information by himself/herself.
(17)
the control means of the photographing device determines whether or not a recording condition based on at least an impact occurring to the vehicle is satisfied, and whether or not a recording condition based on an approach of another vehicle to the vehicle is satisfied,
The image information recording means is a terminal that records the plurality of pieces of image information in a storage means for the terminal when a recording condition based on an approach of another vehicle to the vehicle is satisfied.
By configuring in this way, it is possible to selectively use recording conditions based on the impact of a collision accident, etc., and recording conditions based on proximity before a collision accident, which is a situation where an accident is likely to occur. For example, image information of an event where the post-event occurrence is more important, such as road rage or dangerous driving, can be preferentially stored in the storage unit for the terminal.
(18)
The terminal has a warning means for outputting a warning that the state of the vehicle satisfies a predetermined recording condition by using at least one of sound, light, and text.
By configuring in this way, the terminal can warn the user about the vehicle condition determined by the photographing device. For example, if the photographing device is a rear camera, the user can more reliably receive a warning related to the rear camera from the terminal at hand. Furthermore, if image information of an event related to a warning about aggressive driving, dangerous driving, etc. is recorded in the terminal, the user (driver) is freed from paying attention to securing evidence, etc., and can focus on safe driving.
(19)
The image information recording means is a terminal that stores the image information not including audio information in the storage means.
With this configuration, the terminal can store smaller amounts of image information, and the capacity of the terminal storage unit can be used efficiently.
(20)
the image information recording means stores the image information to which the index information has been added in the storage means;
A terminal that acquires second image information corresponding to the plurality of image information stored by the photographing device based on the index information.
With this configuration, the terminal can easily search for and acquire image information related to the image information stored in the terminal storage unit from the image information stored in the storage unit for the photographing device based on the index information. Also, the user can easily recognize the contents of the image information recorded in the terminal from the index information even from the list on the storage screen.
(21)
A terminal in which the second image information includes audio information.
With this configuration, even if a lower capacity image information not including audio information is stored in the terminal storage unit, for example, when it becomes necessary to grasp a more detailed situation from the video later, the terminal can obtain the audio information from the second image information. Since the user has both image information including audio information and image information not including audio information related to the same event, the user does not need to process the image information regarding the presence or absence of audio information.
(22)
The second image information is image information for a period before the state of the vehicle satisfied a predetermined recording condition.
With this configuration, the terminal can obtain image information from a period prior to the image information stored in the terminal storage unit, so the user can check the behavior of the target vehicle before it is determined to be aggressive driving, and can submit it as evidence if necessary. For example, even if it is difficult to determine whether the target event was intentional or negligent based on image information only after the event occurred, the behavior prior to that can be added to the criteria for judgment. The user can understand the situation when the recording conditions were met in more detail.
(23)
The image information recording means is a terminal that sets a time length for storing the plurality of pieces of image information, and controls a time for storing the plurality of pieces of image information based on the set time length.
With this configuration, the terminal can provide image information of a length desired by the user without the user having to process the length of the image information after storing the image information in the terminal storage unit. This also enables the user to upload image information of an appropriate playback length to SNS or the like without processing it.
(24)
The communication control means is a terminal that transfers the stored image information to an external server.
With this configuration, the terminal can also leave image information stored in the terminal storage unit on an external server, so that even if the terminal or the photographing device is damaged after an event occurs and image information serving as records or evidence is lost, image information of important scenes can be reliably left. For example, if the external server that can be set on the terminal is operated by a public investigative agency or a private road service, it is possible to quickly report or contact the relevant party along with the specific situation.
(25)
A program for causing the terminal to realize the functions of each of the means described above.
With this configuration, the above control can be performed by providing a program to an existing terminal.
(26)
A photographing device disposed in a vehicle,
A transmission means for transmitting information on a plurality of captured images to an external terminal;
A detection means for detecting a state of the vehicle;
and a control means for transmitting information for storing the transmitted plurality of pieces of image information in the terminal when the state of the vehicle satisfies a predetermined recording condition.
With this configuration, image information captured by the imaging device installed in the vehicle when the recording conditions are met can be recorded on the terminal without the user having to operate the terminal to instruct the recording. Therefore, for example, the user can use the terminal to check the image when the recording conditions are met without having to remove the recording medium such as an SD card attached to the imaging device and attach it to a PC, and display the image on a viewer installed on the PC, which is convenient for the user.
The photographing device can record image information related to an event on the terminal without the driver operating the photographing device or the terminal after or during the event, even in the case of an event such as tailgating that is difficult for the driver to directly check with the naked eye without using a mirror or a monitor. This allows the driver to concentrate on driving since he does not need to operate the photographing device to record the image information on the terminal, and he can deal with dangerous driving such as tailgating in a safer state and with a more reassured or calm mind.
(27)
the information for storing the plurality of pieces of image information in the terminal is notification information for notifying that the recording condition is satisfied,
The transmission means sequentially transmits the captured image information,
The control means is an image capturing device that transmits the notification information when the recording condition is satisfied.
With this configuration, the photographing device notifies the terminal by notification information that the recording conditions have already been met, which serves as a trigger for the terminal to start storing image information, so that the terminal can store image information without the need to determine the recording conditions at the terminal.
(28)
The control means is a photographing device that, when the state of the vehicle satisfies the specified recording condition, stores the multiple image information captured during a period including before and after the point at which the recording condition was satisfied in a storage means for the photographing device.
With this configuration, the image capturing device can store longer image information captured including the period before and after the image information stored in the terminal. This allows the user to obtain image information from the period before and after the image information stored in the terminal storage unit. The user can check, for example, the behavior of the target vehicle and the words and actions of the user of the target vehicle before and after it is determined that the vehicle is tailgating, and can submit the information as evidence if necessary.
For example, even if it is difficult to judge whether the event in question was intentional or accidental based on image information only after the event occurred or for a short period of time before and after the event, the behavior before the event can be added to the information for judgment. Furthermore, if the behavior of the driver of the target vehicle becomes an issue at a later date, the necessary image information can be provided to the user at that stage.
(29)
The control means
storing the plurality of pieces of image information in a storage means for the photographing device when a first recording condition is satisfied;
The photographing device transmits, when a second recording condition is satisfied, information for causing the terminal to store the plurality of pieces of image information after the second recording condition is satisfied.
With this configuration, the photographing device can record image information on the photographing device and the terminal under different recording conditions, so that the user can set more appropriate conditions depending on the entity (the photographing device or the terminal) that will store the information. For example, the user can set the second recording condition to a relatively extreme condition, such as a very dangerous condition, so that image information that is expected to have a large impact on SNS or the like can be preferentially saved on the terminal.
(30)
The photographing device issues a predetermined warning when a vehicle approaches from behind the vehicle as the second recording condition, and transmits information for storing the plurality of pieces of image information in the terminal.
By configuring in this manner, the photographing device can issue a warning about the approach of a rear vehicle and prompt the terminal to store image information related to the approach of the rear vehicle. Therefore, not only can the photographing device issue a warning about a detected event, but the terminal can also record image information of a detected event such as rear-end tailgating based on a highly accurate judgment based on the performance of the photographing device, even if the terminal itself does not have the configuration or function to determine the distance or positional relationship to the rear vehicle.
This not only enables the user to quickly recognize events such as rear-end collisions that the user tends to be slow to recognize due to warnings, but also allows the user to secure image information of high interest on a nearby terminal without having to operate the terminal individually, allowing the user to concentrate on safe driving with peace of mind that the image information that serves as evidence is secured.
(31)
A program for causing the photographing device to realize the functions of each of the means described above.
With this configuration, the above control can be performed by providing a program to existing photographing equipment.

10...vehicle, 101, 101F, 101R...photographing equipment, 102...user terminal, 103...external server, 104...network, 301...main memory device, 302...auxiliary memory device, 310...image information recording module, 311...warning module, 312...display control module, 313...photographing equipment linkage module, 314...photographing equipment control module, 320...setting information, 321...image information, 303...processor, 304...input device, 305...output device, 306...camera unit, 307...communication control unit, 401...microcomputer, 402...camera, 403...control unit, 404...photographing equipment controller

Claims (15)

A portable terminal,
A communication control means having a function of receiving a plurality of pieces of image information constituting a video transmitted from a photographing device disposed in the vehicle;
an image information recording means for storing the plurality of pieces of image information received from the photographing device in a storage means for the portable terminal when the state of the vehicle satisfies a predetermined recording condition;
having
the portable terminal has a display means for displaying a screen and a display control means,
the communication control means has a function of successively receiving the plurality of pieces of image information from the photographing device;
the display control means has a function of displaying the received plurality of pieces of image information in a display area on the screen;
a function of storing the displayed plurality of pieces of image information in a storage means for the portable terminal when the recording condition is satisfied while the plurality of pieces of image information is being displayed;
the display control means has a function of displaying an operation area for operating the photographing device or the portable terminal on the screen,
When the photographing device is operated by an operation element on the photographing device side, the display control means has a function of displaying predetermined display information in the display area and restricting input of an operation to the photographing device through the operation area of the portable terminal.
Portable device. 2. The portable terminal according to claim 1 , wherein the display control means displays information relating to an operating state of the photographing device as the predetermined display information. A portable terminal,
A communication control means having a function of receiving a plurality of pieces of image information constituting a video transmitted from a photographing device disposed in the vehicle;
an image information recording means for storing the plurality of pieces of image information received from the photographing device in a storage means for the portable terminal when the state of the vehicle satisfies a predetermined recording condition;
having
the portable terminal has a display means for displaying a screen and a display control means,
the communication control means has a function of successively receiving the plurality of pieces of image information from the photographing device;
the display control means has a function of displaying the received plurality of pieces of image information in a display area on the screen;
a function of storing the displayed plurality of pieces of image information in a storage means for the portable terminal when the recording condition is satisfied while the plurality of pieces of image information is being displayed;
the display control means has a function of displaying an operation area for operating the photographing device or the portable terminal on the screen,
the portable terminal has a photographing device linking means and has a function of setting the photographing device based on inputs to the display area and the operation area;
The photographing device linking means has a function of setting a reference line for notifying the photographing device that the state of the vehicle satisfies the predetermined recording condition based on an input to the operation area.
Portable device. A portable terminal,
A communication control means having a function of receiving a plurality of pieces of image information constituting a video transmitted from a photographing device disposed in the vehicle;
an image information recording means for storing the plurality of pieces of image information received from the photographing device in a storage means for the portable terminal when the state of the vehicle satisfies a predetermined recording condition;
having
the image information recording means stores the image information to which the index information has been added in the storage means for the portable terminal;
a function of acquiring second image information, which is information stored in a storage means for the photographing device by the photographing device based on the index information and corresponds to the plurality of image information;
Portable device. the image information recording means has a function of storing the image information not including audio information in the storage means,
The image information to which the index information is added does not include audio,
The second image information includes audio information.
The portable terminal according to claim 4. 6. The portable terminal according to claim 4 , wherein the second image information includes image information for a period before the state of the vehicle satisfied the predetermined recording condition. A program for causing the portable terminal to realize the functions of each of the means according to any one of claims 1 to 6 . A photographing device disposed in a vehicle,
A transmission means having a function of transmitting a plurality of pieces of image information constituting a moving image, which have been taken, to a portable terminal;
A detection means having a function of detecting a state of the vehicle;
a control means having a function of transmitting information for storing the transmitted plurality of pieces of image information in the portable terminal when the state of the vehicle satisfies a predetermined recording condition;
an operating element having a function of operating the photographing device;
having
the control means has a function of executing an operation of the photographing device based on an operation input to the photographing device from the portable terminal,
The control means has a function of transmitting, to the portable terminal, predetermined information for restricting an input of an operation of the photographing device based on an operation input from the portable terminal to the photographing device when the photographing device is being operated by the operation element.
Filming equipment. the predetermined information is information related to an operational state of the photographing device;
9. The imaging device according to claim 8. A photographing device disposed in a vehicle,
A transmission means having a function of transmitting a plurality of pieces of image information constituting a moving image, which have been taken, to a portable terminal;
A detection means having a function of detecting a state of the vehicle;
a control means having a function of transmitting information for storing the transmitted plurality of pieces of image information in the portable terminal when the state of the vehicle satisfies a predetermined recording condition;
having
the control means has a function of receiving an input for setting the photographing device from the portable terminal,
the control means has a function of setting a reference line for notifying that the state of the vehicle satisfies the predetermined recording condition in response to an input from the portable terminal;
Filming equipment. The photographing device according to claim 10, wherein the photographing device notifies the portable terminal that the state of the vehicle satisfies the specified recording condition by determining at least one of a distance or a positional relationship to at least one of a vehicle in front of the vehicle or a vehicle behind the vehicle based on the set reference line. A photographing device disposed in a vehicle,
A transmission means having a function of transmitting a plurality of pieces of image information constituting a moving image, which have been taken, to a portable terminal;
A detection means having a function of detecting a state of the vehicle;
a control means having a function of transmitting information for storing the transmitted plurality of pieces of image information in the portable terminal when the state of the vehicle satisfies a predetermined recording condition;
having
the transmitting means has a function of transmitting the image information to which the index information has been added to the portable terminal;
the control means has a function of receiving, from the portable terminal, transmission command information for second image information based on the index information, and transmitting, to the portable terminal, the second image information stored in the storage means for the photographing device, the second image information corresponding to the plurality of image information to which the index information has been added.
Filming equipment. the transmitting means has a function of transmitting the image information not including audio information to the portable terminal,
The image information to which the index information is added does not include audio information,
The second image information includes audio information.
13. The imaging device according to claim 12. 14. The photographing device according to claim 12 , wherein the second image information includes image information for a period before the state of the vehicle satisfied the predetermined recording condition. A program for causing the photographing device to realize the functions of each of the means according to any one of claims 8 to 14 .