JP7779146B2 - Vehicle recording control device and recording method - Google Patents

Description

The present invention relates to a vehicle recording control device and recording method, and more particularly to a vehicle recording control device and recording method used in vehicles such as automobiles.

Some vehicle recording devices, known as drive recorders, record events using voice commands in addition to detecting impacts using acceleration sensors (see, for example, Non-Patent Document 1). Recording events using voice commands allows for safe event recording without the need to operate a touch panel or the like while driving, such as when recording an accident in which the driver was not involved. Patent Document 1 discloses a drive recorder that records events by issuing voice commands in response to event detection based on acceleration.

Japanese Patent Application Laid-Open No. 2020-154904

DRV-MR760 [Retrieved September 24, 2021], Internet: <https://www.kenwood.com/jp/car/drive-recorders/products/drv-mr760/>

If a drive recorder is equipped with a function for recording events via voice commands, there may be few opportunities to record events, and so the driver may not be able to immediately remember what voice command to use to record an event. Or, the driver may accidentally speak a similar phrase, preventing the voice command from being executed. In such cases, the timing to record an event may be missed, or the driver may have to use their hands to perform the event recording operation while driving the vehicle, which could compromise safety.

The present invention provides a vehicle recording control device and recording method that can appropriately record events using voice commands.

The vehicle recording control device of the present invention includes an image data acquisition unit that acquires first image data captured by a first camera that captures images of the vehicle's surroundings; an emotion detection unit that detects the emotion of an occupant of the vehicle; a voice recognition unit that recognizes a voice command instructing event recording; an output control unit that, when the emotion detection unit detects that the occupant's emotion is negative, outputs information indicating the voice command to the occupant; and a recording control unit that, when the voice recognition unit recognizes the voice command, saves the first image data as event data.

The recording method of the present invention is performed by a vehicle recording control device, which includes an image data acquisition unit step of acquiring first image data captured by a first camera that captures images of the vehicle's surroundings, an emotion detection step of detecting the emotion of an occupant of the vehicle, a voice recognition step of recognizing a voice command instructing event recording, an output control step of outputting information indicating the voice command to the occupant if the emotion of the occupant is detected to be negative in the emotion detection step, and a recording control step of saving the first image data as event data if the voice command is recognized in the voice recognition step.

According to the present invention, events can be recorded appropriately using voice commands.

1 is a block diagram showing a configuration of a vehicle-mounted recording device according to a first embodiment of the present invention; 3 is a flowchart illustrating an example of processing performed by the vehicle recording control device according to the first embodiment of the present invention. FIG. 10 is a block diagram showing the configuration of a vehicle-mounted recording device according to a second embodiment of the present invention. 10 is a flowchart illustrating an example of processing performed by a vehicle recording control device according to a second embodiment of the present invention.

Embodiments of the present invention will now be described with reference to the drawings. In the following description, a drive recorder used in a moving vehicle, such as an automobile, will be used as an example of a vehicle recording device according to this embodiment. However, the present invention is not limited to this. For example, the present invention can be applied to a variety of moving objects, including various vehicles such as motorcycles and bicycles, trains, ships, robots, and even people. Furthermore, the present invention is not limited to the following embodiments.

A first embodiment of the present invention will now be described with reference to Figures 1 and 2. Figure 1 is a block diagram showing the configuration of a vehicle recording device 100 according to the first embodiment of the present invention. The vehicle recording device 100 is a so-called drive recorder. The vehicle recording device 100 is attached to the top of the windshield of the vehicle, and detects impacts corresponding to accidents, i.e., accelerations corresponding to accidents that have occurred on the vehicle, and saves video footage and other images taken when an accident is detected as event data.

The vehicle recording device 100 shown in Figure 1 is not limited to being installed in a vehicle as a standalone device, but can be configured in a variety of ways, such as being implemented as a function of a navigation device or being pre-installed in a vehicle.

In FIG. 1, the vehicle recording device 100 includes a control unit 200 as a vehicle recording control device, a first camera 102, a second camera 104, a recording unit 106, an output unit 108, a microphone 110, an acceleration sensor 112, an operation unit 114, and a GNSS (Global Navigation Satellite System) receiving unit 116. The output unit 108 is composed of a display unit 108a and an audio output unit 108b.

The control unit 200 includes one or more central processing units (CPUs) that perform various data processing.
The control unit 200 is composed of a graphics processing unit (GPU), a random access memory (RAM), etc., and executes various processes according to programs. The control unit 200 at least includes, as its configuration or realized functions, an image capture data acquisition unit 202, a buffer memory 204, an image capture data processing unit 206, a recording control unit 208, a playback control unit 210, an output control unit 212, a voice recognition unit 216, an event detection unit 218, an operation control unit 220, a position information acquisition unit 222, and an emotion detection unit 224. The output control unit 212 is composed of a display control unit 212a and an audio output control unit 212b.

The control unit 200 is a vehicle recording control device that executes the recording method of the present invention, and is a computer that runs the program of the present invention.

The first camera 102 captures images of the surroundings of the vehicle, which is a moving object. The first camera 102 may be provided integrally with the vehicle recording device 100, or may be connected as a separate device via a wired or wireless connection. Although not shown, the first camera 102 includes a lens, an image sensor, an A-D (Analog to Digital) conversion element, etc. The first camera 102 outputs the captured first image data to the image data acquisition unit 202.

The first camera 102 is provided inside the vehicle cabin above the windshield of the vehicle recording device 100, and if it is provided integrally with the vehicle recording device 100, it is provided on the windshield side of the vehicle recording device 100.

The first camera 102 is not limited to a single camera, but may be composed of multiple cameras. For example, it may be composed of multiple cameras in any combination that capture images of the front, rear, sides, interior of the vehicle, etc., or it may be all cameras.

The second camera 104 captures images of the interior of the vehicle, which is a moving body. The second camera 104 may be provided integrally with the vehicle recording device 100, or may be connected as a separate device via a wired or wireless connection. The second camera 104 includes a lens, an image sensor, an A-D conversion element, etc., although not shown. The second camera 104 outputs the captured second image data to the image data acquisition unit 202.

The second camera 104 is provided in a position where it can capture the face of the vehicle occupant and detect the facial expression or line of sight of the vehicle occupant. The second camera 104 is provided on the interior side of the vehicle above the windshield when the vehicle recording device 100 is installed, and if the second camera 104 is provided integrally with the vehicle recording device 100, it is provided on the interior side of the vehicle recording device 100.

When the second camera 104 is located in a position where the facial expression or gaze of the driver, who is a vehicle occupant, can be detected, it is located, for example, on the vehicle's instrument panel, so that it can capture the direction in which the face of the vehicle driver is located. When the second camera 104 is located in a position where the facial expressions or gaze of other vehicle occupants in addition to the driver can be detected, it may be located, for example, near or inside the vehicle's rearview mirror, so that it can capture the entire interior of the vehicle.

The second camera 104 is not limited to a single camera, and may be composed of multiple cameras. Furthermore, the second camera 104 may be a camera that captures visible light, a camera that captures infrared light, or a combination of these.

In FIG. 1, the first camera 102 and the second camera 104 are described as separate cameras, but a single camera may be used if it can capture the range captured by the first camera 102 and the second camera 104 described above. A single camera is, for example, an all-around camera capable of capturing 180-degree or 360-degree images.

The image data captured by the first camera 102 and the second camera 104 is video image data. This image data may also include audio data in addition to video image data.

The recording unit 106 is a non-volatile memory, such as a memory card, for recording the first image data captured by the first camera 102 under the control of the recording control unit 208. The first image data and event data recorded in the recording unit 106 are played back under the control of the playback control unit 210. The recording unit 106 may be replaced with a separate recording and playback device that includes the recording control unit 208 and playback control unit 210 in addition to the recording unit 106. The recording unit 106 may be provided as an integrated recording and playback device, or may be connected as a separate device via a wired or wireless connection.

The output unit 108 outputs information indicating a voice command to a vehicle occupant. The output unit 108 is composed of a display unit 108a and an audio output unit 108b. When the display unit 108a operates as the output unit 108, the display unit 108a displays information such as text information indicating the voice command on the display unit 108a as information indicating the voice command to a vehicle occupant. When the audio output unit 108b operates as the output unit 108, the audio output unit 108b outputs audio indicating the voice command.

The display unit 108a is a display device that displays various information under the control of the display control unit 212. The display unit 108a has a display panel such as a liquid crystal panel or an organic EL (organic electro-luminescence) panel. The display unit 108a may be provided integrally as part of the vehicle recording device 100, or may be connected as a separate device via a wired or wireless connection. The display unit 108a may also be another device that is connected wirelessly and has the display control unit 212a. The display unit 108a is positioned so that it can be easily seen by vehicle occupants.

The display unit 108a is provided on the interior side of the vehicle above the windshield when the vehicle recording device 100 is mounted, and if it is provided integrally with the vehicle recording device 100, it is provided on the interior side of the vehicle recording device 100. In this case, the display unit 108a is provided near the second camera 104.

The audio output unit 108b outputs various sounds under the control of the audio output control unit 212b. Specifically, the audio output unit 108b is composed of an audio amplifier and a speaker. The audio output unit 108b outputs the same sound as the voice command to the vehicle occupants as information indicating the voice command.

When outputting information indicating a voice command, the voice output unit 108b outputs the voice so that it is not mistakenly recognized as a voice command. For this reason, the voice output unit 108b is configured to be easy for vehicle occupants to hear, but not input to the microphone 110. The voice output unit 108b is, for example, a speaker provided above the seat in which the vehicle occupant is seated. Alternatively, the voice output unit 108b is a neck speaker worn by the vehicle occupant.

The microphone 110 acquires the voice spoken by a vehicle driver or other passenger. The microphone 110 is provided in the second camera 104, and the voice spoken by the vehicle driver or other passenger may be included in the second captured data. The microphone 110 is positioned so that it can acquire the voice spoken by the vehicle driver or other passenger. The microphone 110 may have directionality that allows it to acquire the voice spoken by the vehicle driver or other passenger.

The acceleration sensor 112 detects acceleration applied to the vehicle recording device 100 or the vehicle. The acceleration sensor 112 is, for example, a three-axis acceleration sensor that detects acceleration applied in the x-axis direction (the longitudinal direction of the vehicle), the y-axis direction (the lateral direction of the vehicle), and the z-axis direction (the vertical direction of the vehicle). The acceleration sensor 112 outputs the detected acceleration information to the event detection unit 218. The acceleration sensor 112 may be provided integrally as part of the vehicle recording device 100, or may be connected as a separate entity via a wire or wirelessly. The acceleration sensor 112 may also be a separate entity that forms a unit including the acceleration sensor 112 and the event detection unit 218.

The operation unit 114 is an interface that accepts operations on the vehicle recording device 100 and outputs the accepted operation information to the operation control unit 220. The operation unit 114 is composed of various buttons provided on the vehicle recording device 100 or a touch panel provided on the display unit 108a, and accepts operations by the user. The operation unit 114 may also accept operations from other wirelessly connected devices. For example, the operation unit 114 accepts an operation by the user to start event recording. The operation unit 114 also accepts an operation by the user to play back event data.

The GNSS receiving unit 116 is composed of, for example, a receiving device and antenna that receives signals from GPS (Global Positioning System) satellites. The GNSS receiving unit 116 outputs the received signals to the location information acquisition unit 222. The GNSS receiving unit 116 may be provided integrally as part of the vehicle recording device 100, or may be connected as a separate unit via a wired or wireless connection. The GNSS receiving unit 116 may also be a separate unit that includes the GNSS receiving unit 116 and the location information acquisition unit 222.

The image capture data acquisition unit 202 acquires first image capture data captured by the first camera 102 and second image capture data captured by the second camera 104. The image capture data acquisition unit 202 outputs the first image capture data acquired from the first camera 102 to the buffer memory 204. If the emotion detection unit 224 detects emotions based on the facial expressions of vehicle occupants, the image capture data acquisition unit 202 outputs the second image capture data acquired from the second camera 104 to the emotion detection unit 224.

The image capture data acquisition unit 202 also functions as an image capture control unit that controls image capture by the first camera 102 and the second camera 104. The image capture data acquisition unit 202 causes the first camera 102 and the second camera 104 to capture images as necessary while the vehicle recording device 100 is operating.

The buffer memory 204 is an internal memory provided in the control unit 200, and temporarily stores, while updating, a certain period of shooting data acquired by the shooting data acquisition unit 202.

The shooting data processing unit 206 converts the shooting data temporarily stored in the buffer memory 204 into any file format, such as MP4 format, encoded using any codec, such as H.264 or MPEG-4 (Moving Picture Experts Group). The shooting data processing unit 206 generates shooting data files for a certain period of time from the shooting data temporarily stored in the buffer memory 204. As a specific example, the shooting data processing unit 206 generates 60 seconds of shooting data as files in the order of shooting from the shooting data temporarily stored in the buffer memory 204. The shooting data processing unit 206 outputs the generated shooting data to the recording control unit 208. The shooting data processing unit 206 also outputs the generated shooting data to the display control unit 212a. The period of the shooting data generated as files is set to 60 seconds as an example, but is not limited to this.

The recording control unit 208 controls the recording unit 106 to record the imaging data filed by the imaging data processing unit 206. While the vehicle recording device 100 is operating, the recording control unit 208 records the imaging data filed by the imaging data processing unit 206 in the recording unit 106 as overwritable imaging data. When the recording capacity of the recording unit 106 reaches its upper limit, the recording control unit 208 overwrites and records new imaging data from the recording area where older imaging data is recorded among the overwritable imaging data recorded in the recording unit 106. The imaging data recorded by the recording control unit 208 in the recording unit 106 is the first imaging data captured by the first camera 102.

The recording control unit 208 continues to record image data based on the first image data in the recording unit 106 while the vehicle recording device 100 is operating. This type of recording process is called normal recording or loop recording.

When the event detection unit 218 determines that an event has occurred, the recording control unit 208 saves the first captured data for a predetermined period of time, including the time when the event occurred, as event data that is prohibited from being overwritten. The event data is, for example, captured data for 15 seconds before and after the time when the event occurred.

When the operation control unit 220 accepts an operation to perform event recording, the recording control unit 208 saves the first captured data for a predetermined period of time, including the time when the operation to perform event recording was performed, as event data that is prohibited from being overwritten. The event data is, for example, captured data for 15 seconds before and after the time when the operation to perform event recording was performed.

When the voice recognition unit 216 recognizes a voice command to perform event recording, the recording control unit 208 saves the first captured data for a predetermined period of time, including the time when the voice command to perform event recording was recognized, as event data that is prohibited from being overwritten. The event data is, for example, captured data for 15 seconds before and after the time when the voice command to perform event recording was recognized.

The recording control unit 208 may store the event data in any manner. For example, an overwrite prohibition flag may be added to the header or payload of the section of the first captured data that is to be overwritten, and the data may be stored in the recording unit 106. Alternatively, the section of the first captured data that is to be overwritten may be stored in an overwrite prohibition area of the recording unit 106. Alternatively, the section of the first captured data that is to be overwritten may be transmitted to another device and stored there.

The playback control unit 210 controls the playback of the shooting data and event data recorded in the recording unit 106. In addition to playing back overwritable shooting data recorded in the recording unit 106, the playback control unit 210 also plays back event data that is not overwritable, and outputs the data to the display control unit 212a.

The output control unit 212 performs processing to output various information to vehicle occupants. Specifically, when the emotion detection unit 224 detects that the emotion of the vehicle occupant is negative, the output control unit 212 outputs information indicating a voice command to the occupant. The output control unit 212 is composed of a display control unit 202a and an audio output control unit 212b.

The display control unit 212a controls the display unit 108a to display various information on the display unit 108a. When the display control unit 212a functions as the output control unit 212, if the emotion detection unit 224 detects that the passenger's emotion is a negative emotion, the display control unit 212a causes the display unit 108a to display information indicating a voice command instructing event recording. If the voice command instructing event recording is "start recording," the display control unit 212 causes the display unit 108a to display information indicating the voice command, such as "start recording" or "recording start."

In addition to functioning as the output control unit 212, the display control unit 212a may also cause the display unit 108a to display the first video data acquired by the shooting data acquisition unit 202. In this case, the video being shot by the first camera 102 is displayed on the display unit 108a. The display control unit 212a also causes the display unit 108a to display the shooting data or event data played back by the playback control unit 210.

The audio output control unit 212b causes the audio output unit 108b to output various sounds. When the audio output control unit 212b functions as the output control unit 212, if the emotion detection unit 224 detects that the passenger's emotion is negative, the audio output control unit 212b causes the audio output unit 108b to output a sound indicating a voice command instructing event recording. If the voice command instructing event recording is "start recording," the audio output control unit 212b causes the audio output unit 108b to output the sound "start recording."

In addition to functioning as the output control unit 212, the audio output control unit 212b causes the audio output unit 108b to output the audio contained in the shooting data or event data played back by the playback control unit 210.

The voice recognition unit 216 analyzes the voice captured by the microphone 110 and recognizes the content of the speech. The voice recognition unit 216 performs acoustic model analysis on the voice captured by the microphone 110 for each phoneme or word, and compares the results with phoneme models or language models to recognize the content of the speech. The voice recognition unit 216 recognizes voice commands for the vehicular recording device 100. Voice commands recognized by the voice recognition unit 216 include, for example, "start recording," which is a voice command for recording an event, and "take a photo," which is a voice command for taking a still image.

When the voice recognition unit 216 determines through phoneme model analysis of the voice acquired by the microphone 110 that each consecutive phoneme is "roku ga kai shi," it accepts the voice command "start recording" and outputs an instruction to the recording control unit 208 to record the event.

The event detection unit 218 acquires acceleration information detected by the acceleration sensor 112, and if acceleration corresponding to an event is detected, it determines that an event has been detected. Acceleration corresponding to an event is acceleration corresponding to the acceleration that occurs when an object, such as another vehicle, collides with the vehicle on which the vehicular recording device 100 is installed. The detection of acceleration corresponding to an event may be weighted in each of the x-axis direction, y-axis direction, and z-axis direction. If the event detection unit 218 determines that an event has been detected, it outputs an instruction to the recording control unit 208 to record the event.

The operation control unit 220 acquires operation information received by the operation unit 114 and outputs operation instructions based on the operation information to each component. When the operation control unit 220 receives an operation of a button or icon to perform event recording from the operation unit 114, it outputs an instruction to perform event recording to the recording control unit 208. In addition to the above, the operation control unit 220 acquires operation information indicating operations such as selecting event data, playing, erasing, pausing, and various setting operations of the vehicle recording device 100, and issues instructions for processing corresponding to the acquired operation information.

The location information acquisition unit 222 calculates the vehicle's current location information using a known method based on the radio waves received by the GNSS receiving unit 116. The location information calculated by the location information acquisition unit 222 is recorded by the recording control unit 208 together with the shooting data and event data.

The emotion detection unit 224 performs processing to detect the emotions of vehicle occupants. For example, the emotion detection unit 224 detects the emotions of vehicle occupants by detecting vehicle occupants from the second captured data and analyzing the facial expressions of the detected occupants. The emotion detection unit 224 also detects the emotions of vehicle occupants by analyzing the speech of the vehicle occupants acquired by the voice recognition unit 216. Publicly known methods can be applied to either method.

When detecting the emotion of a vehicle occupant from the second captured data, the emotion detection unit 224 acquires the eye and mouth movements that make up the occupant's face, and detects emotions such as joy, serenity, anger, and fear by using a trained model that has undergone machine learning to learn the movements of various facial components. If the emotion detected from the facial expression of the vehicle occupant is anger or fear, the emotion detection unit 224 determines that this is a negative emotion.

When detecting the emotions of a vehicle occupant from their speech, the emotion detection unit 224 acquires the content and intonation of the occupant's speech, and detects emotions such as joy, serenity, anger, fear, and surprise by using a trained model that has undergone machine learning of various speech contents and intonations. If the emotion detected from the speech of the vehicle occupant is anger or fear, the emotion detection unit 224 determines that this is a negative emotion.

The emotion detection unit 224 may determine that a vehicle occupant's emotion is negative if it continues to show anger, fear, or surprise for a predetermined period of time, such as 60 seconds or more.

Next, the flow of the recording process performed by the vehicle recording device 100 will be explained using Figure 2. The recording process performed by the vehicle recording device 100 is executed based on a program by the control unit 200, which is a computer operating as a vehicle recording control device.

The start of processing in Figure 2 means that the vehicle recording device 100 begins operating, for example, when the engine or power supply of a vehicle, which is a moving object, is turned on.

When the process starts, the vehicle recording device 100 starts the normal recording process and the gaze detection process (step S10). Specifically, the recording control unit 208 starts the normal recording process of recording the image data based on the first image data in the recording unit 106, and the gaze detection unit 214 starts detecting the emotions of the vehicle occupants from the second image data.

When the processing of step S10 begins, the vehicle recording device 100 determines whether an operation to perform event recording has been performed (step S11). Specifically, the operation control unit 220 determines whether an operation of a button or icon to perform event recording has been received on the operation unit 114. If it is determined that an operation to perform event recording has been performed (step S11: Yes), the process proceeds to step S16. If it is not determined that an operation to perform event recording has been performed (step S11: No), the process proceeds to step S12.

In step S12, the vehicular recording device 100 determines whether a voice command to perform event recording has been received. Specifically, the voice recognition unit 216 determines whether the voice acquired by the microphone 110 includes an utterance indicating a voice command to perform event recording. If it is determined that a voice command to perform event recording has been received (step S12: Yes), the process proceeds to step S16. If it is not determined that a voice command to perform event recording has been received (step S12: No), the process proceeds to step S14.

In step S13, the vehicle recording device 100 determines whether an event has been detected. Specifically, the event detection unit 218 determines whether an event has been detected based on the acceleration detected by the acceleration sensor 112. If it is determined that an event has been detected (step S13: Yes), the process proceeds to step S16. If it is not determined that an event has been detected (step S13: No), the process proceeds to step S14.

In step S14, the vehicle recording device 100 determines whether the emotion of the vehicle occupant is negative. Specifically, the emotion detection unit 224 determines whether the emotion of the vehicle occupant is negative based on the facial expression or speech of the vehicle occupant. Occupants who become negative while driving a vehicle, that is, occupants who feel anger, fear, or surprise, are often the vehicle driver. This is because they are in a position where they are more likely to recognize dangers while driving a vehicle. Also, voice commands to perform event recording are often uttered by the vehicle driver. Therefore, determining whether the emotion of the vehicle occupant is negative can also be rephrased as determining whether the emotion of the vehicle driver is negative. However, vehicle occupants also include occupants other than the vehicle driver. The emotion detection unit 224 may determine whether the emotion of the vehicle occupant has been negative for a predetermined period of time or longer.

If it is determined in step S14 that the emotion of the vehicle occupant is a negative emotion (step S14: Yes), proceed to step S15. If it is not determined that the emotion of the vehicle occupant is a negative emotion (step S14: No), proceed to step S17.

In step S15, the vehicular recording device 100 uses the output unit 108 to output information indicating the voice command to the vehicle occupants. One example of step S15 is to display information such as text indicating the voice command, such as "Start recording" or "Start recording," on the display unit 108a operating as the output unit 108. Another example of step S15 is to cause the audio output unit 108b operating as the output unit 108 to output audio indicating the voice command, such as "Start recording."

Steps S11, S12, S13, and S14 are started simultaneously and in parallel with the start of step S10, and are executed until the processing of Figure 2 is completed.

In step S16, the vehicle recording device 100 saves the event data. Specifically, the recording control unit 208 saves the event data based on the first captured data as overwrite-protected. If it is determined in step S11 that an operation to record an event has been performed, step S16 saves the first captured data, such as for 15 seconds before and after the operation to record the event, as event data. If it is determined in step S12 that a voice command to record an event has been received, step S16 saves the first captured data, such as for 15 seconds before and after the voice command to record the event was recognized, as event data. If it is determined in step S13 that an event has been detected, step S16 saves the first captured data, such as for 15 seconds before and after the event occurred, as event data.

If step S14 returns No, then after step S15 and step S16, the vehicle recording device 100 determines whether the recording process has ended (step S17). Specifically, the vehicle recording device 100 determines whether the process started in step S10 has ended. End of the process started in step S10 means, for example, that the operation of the vehicle recording device 100 has ended when the engine or power supply of the vehicle, which is a moving object, is turned off. If it is determined that the process has ended (step S17: Yes), the process of Figure 2 ends. If it is not determined that the process has ended (step S17: No), proceed to step S11.

When a vehicle driver or other passenger wants to record as event data the captured image data of an event such as an accident in which the vehicle was not involved, they may want to save the event data using a voice command. They may also want to save the event data using a voice command when being tailgated by a nearby vehicle. In such cases, the event will not be detected by the processing in step S13. It may also be unsafe for the driver to perform event recording operations using the operation unit 114 while driving the vehicle. Even for passengers other than the driver, passengers who cannot reach the operation unit 114 of the vehicular recording device 100 may find it difficult to perform event recording operations using the operation unit 113.

Even when using a voice command to instruct event recording, there are cases where it is not possible to immediately recall the voice command to instruct event recording. For example, this may occur when voice commands are not used frequently to instruct event recording, or when a user uses multiple vehicles and the voice commands to instruct event recording on the vehicle recording device installed in each vehicle are different.

In this way, if an accident or other incident occurs near the vehicle and a vehicle driver or other occupant experiences negative emotions such as fear or surprise, or if they are provoked by another vehicle and experience negative emotions such as anger or fear, there is a high need to record an event. In such cases, the occupant may not be able to immediately remember the correct voice command or may utter an incorrect voice command. In such cases, the process shown in Figure 2 outputs a voice command indicating event recording, allowing the vehicle occupant to utter the appropriate voice command, minimizing delay in the timing of event recording and enabling appropriate event recording.

Next, a second embodiment of the present invention will be described with reference to FIG. 3. The configuration of the vehicle recording device 100A according to the second embodiment differs from the vehicle recording device 100 according to the first embodiment in that the control unit 200 is replaced by a control unit 200A. The control unit 200A further includes a gaze detection unit 214 in addition to the control unit 200. Descriptions of the configuration of the vehicle recording device 100A according to the second embodiment that are the same as the configuration of the vehicle recording device 100 according to the first embodiment will be omitted.

The gaze detection unit 214 detects the gaze direction of the vehicle occupants from the second imaging data acquired by the imaging data acquisition unit 202. If the second camera 104 is positioned to capture an image of the vehicle driver, the gaze detection unit 214 detects the gaze direction of the vehicle driver captured in the second imaging data. If the second camera 104 is positioned so that it can capture an image of the entire interior of the vehicle, the gaze detection unit 214 detects the gaze direction of the vehicle driver and, if there are occupants other than the driver, in addition to detecting the gaze direction of the vehicle driver.

The gaze detection unit 214 detects a person's face from the second captured data and detects the position of the eyes on the detected face. The gaze detection unit 214 detects the face and eyes using any method, such as pattern matching. The gaze detection unit 214 detects the gaze direction based on the positional relationship between the inner corner of the eye and the iris detected from the eye image, or the positional relationship between the corneal reflex and the pupil. The gaze detection method is not limited to the one described above, and any method can be applied.

The gaze detection unit 214 detects whether the gaze of a vehicle occupant tends to be fixed on a spot outside the vehicle. The gaze detection unit 214 detects whether the gaze of a vehicle occupant tends to be fixed on a spot outside the vehicle based on pre-stored data indicating the positional relationship between the second camera 104 and the vehicle's windshield, side windows, etc. Outside the vehicle also includes the direction ahead of the vehicle. This is because, for example, if an accident occurs near the vehicle, the vehicle occupant tends to focus on the accident.

The gaze detection unit 214 may also determine that a vehicle occupant is gazing outside the vehicle when it detects that the occupant's gaze tends to be fixed on the vehicle's inner mirror or a display device with inner mirror functionality. The gaze detection unit 214 detects that the occupant's gaze tends to be fixed on the vehicle's outside based on pre-stored data indicating the positional relationship between the second camera 104 and the vehicle's inner mirror or a display device with inner mirror functionality. For example, when being tailgated by a vehicle behind, the occupant of the vehicle tends to use the inner mirror or the like to gaze behind the vehicle, i.e., outside the vehicle.

Gazing here means, for example, that the vehicle occupant's line of sight is directed outside the vehicle for a predetermined period of time, such as five seconds or more. The gaze detection unit 214 may determine that the vehicle occupant is tending to gaze outside the vehicle when the occupant's line of sight is directed outside the vehicle and there is little change in the gaze direction. Furthermore, when gazing outside the vehicle through the windshield or side window of the vehicle, the condition for the gaze direction may be that there is little change in the absolute gaze direction relative to the vehicle, or that there is little change in the relative gaze direction relative to the vehicle when the vehicle's positional relationship with the accident site fluctuates as the vehicle moves.

The gaze detection unit 214 may determine that a vehicle occupant has a tendency to gaze outside the vehicle if their gaze is directed in the same direction outside the vehicle multiple times consecutively. "Multiple times consecutively" means, for example, if the vehicle occupant's gaze is directed in the same direction outside the vehicle two or more times within three seconds. If there are multiple vehicle occupants, the gaze detection unit 214 may detect that the same occupant's gaze is directed in the same direction outside the vehicle multiple times consecutively.

Next, the flow of the recording process performed by the vehicle recording device 100A will be explained using Figure 4. The recording process performed by the vehicle recording device 100A is executed based on a program by the control unit 200A, which is a computer operating as a vehicle recording control device.

The processes of steps S20 to S24, step S26, and steps S27 to S28 shown in Figure 4 are the same as the processes of steps S10 to S14, step S15, and steps S16 to S17 shown in Figure 2, so their explanations are omitted.

If it is determined in step S24 that the emotion of the vehicle occupant is negative (step S24: Yes), the process proceeds to step S25. In step S25, the vehicle recording device 100 detects whether the vehicle occupant tends to gaze outside the vehicle, that is, whether the vehicle occupant's line of sight tends to gaze outside the vehicle. If it is determined that the vehicle occupant tends to gaze outside the vehicle (step S25: Yes), the process proceeds to step S26. If it is not determined that the vehicle occupant tends to gaze outside the vehicle (step S25: No), the process proceeds to step S28.

In this way, if a vehicle driver or other occupant experiences negative emotions such as fear or surprise after an accident or other incident occurs near the vehicle, or if they experience negative emotions such as anger or fear after being tailgated by another vehicle and tend to focus their attention on the target of the accident or tailgating, there is a high need to record an event. In such cases, the occupant may not be able to immediately remember the correct voice command or may utter an incorrect voice command. In such cases, the process shown in Figure 4 outputs a voice command indicating event recording, allowing the occupant to utter the appropriate voice command, minimizing delay in the timing of event recording and enabling appropriate event recording.

The present invention is not limited to the above-described embodiment, and modifications can be made as appropriate without departing from the spirit of the invention. Furthermore, programs for causing a computer to execute the above-described processes can be stored on various types of non-transitory computer-readable media and supplied to a computer operating as a vehicle recording control device.

100, 100A Vehicle recording device 102 First camera 104 Second camera 106 Recording unit 108 Output unit 108a Display unit 108b Audio output unit 110 Microphone 112 Acceleration sensor 114 Operation unit 116 GNSS receiving unit 200, 200A Control unit (vehicle recording control device)
202 Shooting data acquisition unit 204 Buffer memory 206 Shooting data processing unit 208 Recording control unit 210 Playback control unit 212 Output control unit 212a Display control unit 212b Audio output control unit 214 Gaze detection unit 216 Voice recognition unit 218 Event detection unit 220 Operation control unit 222 Position information acquisition unit 224 Emotion detection unit

Claims (5)

an image data acquisition unit that acquires first image data captured by a first camera that captures an image of the periphery of the vehicle;
an emotion detection unit for detecting an emotion of an occupant of the vehicle;
a voice recognition unit that recognizes a voice command for instructing event recording;
an output control unit that outputs information indicating the voice command to the passenger when the emotion detection unit detects that the passenger's emotion is a negative emotion;
a recording control unit that stores the first captured data as event data when the voice recognition unit recognizes the voice command;
A recording control device for a vehicle comprising: a gaze detection unit that detects the direction of gaze of a passenger in the vehicle;
When the gaze detection unit detects that the gaze of the passenger tends to gaze outside the vehicle and that the passenger's emotion is negative, the output control unit outputs information indicating the voice command to the passenger.
The recording control device for a vehicle according to claim 1. the emotion detection unit detects the emotion of the vehicle occupant from a facial expression of the vehicle occupant,
3. A recording control device for a vehicle according to claim 1 or 2. The emotion detection unit detects the emotion of the occupant of the vehicle from an utterance of the occupant of the vehicle.
3. The recording control device for a vehicle according to claim 1 or 2. an image data acquisition unit step of acquiring first image data captured by a first camera that captures an image of the periphery of the vehicle;
an emotion detection step of detecting an emotion of an occupant of the vehicle;
a voice recognition step for recognizing a voice command instructing event recording;
an output control step of outputting information indicating the voice command to the passenger when the emotion of the passenger is detected to be a negative emotion in the emotion detection step;
a recording control step of storing the first captured data as event data when the voice command is recognized in the voice recognition step;
A recording method executed by a vehicle recording control device.