JP7776982B2 - Information processing device, information processing method, and information processing program - Google Patents

Description

The present invention relates to an information processing device, an information processing method, and an information processing program.

Conventionally, systems have been proposed for monitoring specific individuals in vehicles (e.g., children, elderly people, etc.) via systems installed in the vehicle.

JP 2018-169942 A

However, the above-mentioned conventional technologies do not necessarily prevent situations in which driving safety is compromised due to changes in the state of the occupants.

For example, in the above-mentioned conventional technology, video footage of the person being watched inside the car is recorded, and information extracted from this video is notified to the person watching over the person being watched.

As such, the above-mentioned conventional technologies merely assist a watcher outside the vehicle in properly monitoring those being watched over around the vehicle, and do not take into consideration, for example, ensuring driving safety in terms of changes in the state of the occupants.

Therefore, the above-mentioned conventional technologies may not necessarily be able to prevent situations in which driving safety is compromised due to changes in the state of the occupants.

The present invention was made in consideration of the above, and aims to realize an information processing device, information processing method, and information processing program that can prevent situations in which driving safety is compromised due to changes in the state of the occupants.

The information processing device described in claim 1 is characterized by having a first detection unit that detects the occurrence of a predetermined state change in a vehicle occupant based on state information indicating the state of the occupant, a second detection unit that detects a situation related to the vehicle, and a storage unit that stores linked data that links content information indicating the content of the predetermined state change with situation information indicating the situation related to the vehicle when the predetermined state change occurs.

The information processing method described in claim 18 is an information processing method executed by an information processing device, and is characterized by including: a first detection step of detecting the occurrence of a predetermined state change in a vehicle occupant based on state information indicating the state of the occupant; a second detection step of detecting a situation related to the vehicle; and a storage step of storing linked data linking content information indicating the content of the predetermined state change with situation information indicating the situation related to the vehicle when the predetermined state change occurred.

The information processing program described in claim 19 is an information processing program for causing an information processing device to execute a first detection procedure for detecting the occurrence of a predetermined state change in a vehicle occupant based on state information indicating the state of the occupant, a second detection procedure for detecting a situation related to the vehicle, and a storage procedure for storing linked data that links content information indicating the content of the predetermined state change with situation information indicating the situation related to the vehicle when the predetermined state change occurred.

FIG. 1 is a diagram illustrating an example of an information processing system according to an embodiment. FIG. 2 is an explanatory diagram illustrating an overall image of information processing according to the embodiment. FIG. 3 is a diagram illustrating an example of the configuration of a server device according to the embodiment. FIG. 4 is a diagram illustrating an example of a sensor information database according to the embodiment. FIG. 5 is a diagram illustrating an example of the association information database according to the embodiment. FIG. 6 is a diagram illustrating an example of the configuration of a server device according to the embodiment. FIG. 7 is a diagram illustrating an example of a question message information database according to the embodiment. FIG. 8 is a diagram showing an example of a dialogue realized between the server device and a user. FIG. 9 is a flowchart showing a detection process procedure for detecting the vehicle status. FIG. 10 is a flowchart showing the procedure of the linking process for linking content information with situation information. FIG. 11 is a hardware configuration diagram illustrating an example of a computer that realizes the functions of the server device.

Below, an example of a form for implementing an information processing device, an information processing method, and an information processing program (hereinafter referred to as an "embodiment") will be described in detail with reference to the drawings. Note that the information processing device, information processing method, and information processing program are not limited to this embodiment. Furthermore, identical components in the following embodiments will be assigned the same reference numerals, and duplicate explanations will be omitted.

1. System Configuration
First, the configuration of an information processing system according to an embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram illustrating an example of an information processing system according to an embodiment. Fig. 1 illustrates an information processing system 1 as an example of an information processing system according to an embodiment.

As shown in FIG. 1, the information processing system 1 may include an in-vehicle device 10, a user device 60, a server device 100, and a server device 200. The in-vehicle device 10, the user device 60, the server device 100, and the server device 200 are connected to each other via a network N so as to be able to communicate with each other via a wired or wireless connection. The information processing system 1 shown in FIG. 1 may also include any number of in-vehicle devices 10, any number of user devices 60, any number of server devices 100, and any number of server devices 200.

The in-vehicle device 10 may be a dedicated navigation device built into or externally attached to the vehicle VEx, or it may be a recording device (dashcam) installed in the vehicle VEx for crime prevention or to combat aggressive driving.

The in-vehicle device 10 may also be configured with a navigation device and a recording device. As one example, the in-vehicle device 10 may be a composite device in which a navigation device and a recording device, which are independent of each other, are connected to each other so that they can communicate with each other. As another example, the in-vehicle device 10 may be a single device that has both navigation and recording functions.

In addition, users can install a specific application on a portable terminal device (e.g., a smartphone, tablet, notebook PC, desktop PC, PDA, etc.) that they use on a daily basis and use it as an in-vehicle device 10. For example, a portable terminal device with a specific navigation app or a specific recording app installed can be understood as the in-vehicle device 10 referred to here. When a portable terminal device is used as an in-vehicle device 10, it is installed, for example, on the dashboard of the vehicle VEx while driving.

The in-vehicle device 10 may also be equipped with various sensors. For example, the in-vehicle device 10 may be equipped with various sensors such as a camera, an acceleration sensor, a gyro sensor, a GPS sensor, and an air pressure sensor. The server device 100, described below, may perform various information processing based on sensor information detected by these sensors. The server device 100 may also use sensor information detected by sensors provided in the vehicle VEx itself as a safe driving system, in addition to the sensors provided in the in-vehicle device 10.

User device 60 is a portable terminal device owned by a user. As described above, such a portable terminal device may be a smartphone, tablet device, notebook PC, desktop PC, PDA, etc. In this embodiment, an application AP (hereinafter abbreviated as "app AP") that enables the sending and receiving of information between user device 60 and server device 100 and server device 200 is installed on user device 60. Via app AP, the user can register information with server device 100 and interact with the interactive functions of server device 200.

Server device 100 realizes the information processing according to the embodiment by working in cooperation with server device 200. For example, server device 100 detects the occurrence of a predetermined state change for a vehicle occupant based on state information indicating the state of the occupant. Server device 100 may also detect the situation related to the vehicle. As a result, server device 100 accumulates linking data that links content information indicating the content of the predetermined state change with situation information indicating the situation related to the vehicle when the predetermined state change occurred.

The server device 200 generates user information for a specific user related to the passenger based on the linking data accumulated by the server device 100. Furthermore, when the user to whom the user information has been provided responds to the user information, the server device 200 estimates the cause of the specific state change based on the content of the response. The server device 200 then controls the output of information corresponding to the estimation result to the user.

The information processing performed between server devices 100 and 200 provides, for example, a vehicle abnormality monitoring service (hereinafter sometimes referred to as "monitoring service SA") to the user. The monitoring service SA allows the user to, for example, look back on the time their child started crying in the car, find out the cause, and take measures to prevent future crying in the car. The user can also receive the monitoring service SA via the application AP. For this reason, the application AP can be considered an agent (e.g., a conversational agent) that provides the monitoring service SA.

Here, if the in-vehicle device 10 is an edge computer that performs edge processing near the user, the server devices 100 and 200 may be, for example, cloud computers that perform processing on the cloud side.

Furthermore, without being limited to the example of FIG. 1, the server device 100 and the server device 200 may be integrated, and the integrated single server device corresponds to the information processing device according to the embodiment. Furthermore, the following embodiment shows an example in which information processing according to the embodiment is realized by transmitting and receiving information between the in-vehicle device 10 and this information processing device. However, the information processing according to the embodiment may also be realized only on the edge side, i.e., by the in-vehicle device 10. In this case, the in-vehicle device 10 may be configured to behave like an information processing device, for example, by an information processing program according to the embodiment.

[2. Overview of information processing]
From here, an overview of information processing according to the embodiment will be described with reference to Fig. 2. Fig. 2 is an explanatory diagram illustrating an overview of information processing according to the embodiment.

Figure 2 shows person U11 and person B12 as passengers of vehicle VE1 (an example of vehicle VEx). More specifically, in the example of Figure 2, one of the passengers, person U11, is driving vehicle VE1, and the other passenger, person B12, is sitting in the back seat. Also, in the example of Figure 2, person U11 and person B12 are in a parent-child relationship, with person U11 being the mother and person B12 being the child. In this regard, person U11 and person B12 can also be said to be in a caregiver-caregiver relationship.

Here, person U11 is unable to look after his child, person B12, while driving, and if an incident occurs involving person B12, he may be unable to respond appropriately to the incident, which could compromise driving safety. Such incidents include crying, fussiness, strange vocalizations, or emotional changes. Therefore, when such incidents occur, person U11 uses monitoring service SA to investigate the cause and prevent future incidents from occurring.

In the example of Figure 2, person U11 has registered various pieces of information with server device 100 in advance in order to use monitoring service SA. For example, person U11 has registered not only his profile and information for identifying his own vehicle, but also information about the person who is the detection target (monitoring target) whose status change is to be detected. In the example of Figure 2, person U11 has determined that the person to be detected is person B12, and has registered, for example, a facial image of person B12's face and information about person B12's voice.

With this registration complete, Figure 2 shows a scene in which person U11 is driving vehicle VE1 with person B12 in the back seat on the morning of October 10, 2021.

For example, when driving of the vehicle VE1 begins, the in-vehicle device 10 may transmit sensor information detected by various sensors provided in the vehicle VE1 to the server device 100 in real time. As described above, some sensors may be provided in the in-vehicle device 10 and some may be provided in the vehicle VE1 itself. Therefore, the in-vehicle device 10 can transmit sensor information detected by sensors inside the vehicle and sensor information detected by sensors outside the vehicle to the server device 100 in real time, for example.

As a result, as shown in FIG. 2, the server device 100 acquires sensor information from the in-vehicle device 10 (step S11). For example, the server device 100 acquires sensor information detected by sensors inside the vehicle and sensor information detected by sensors outside the vehicle.

Furthermore, each time the server device 100 acquires sensor information, it may detect the current situation regarding the vehicle VE1 (vehicle situation) based on the acquired sensor information (step S12). For example, the server device 100 may detect, as the situation regarding the vehicle VE1, the position of the vehicle VE1, the time at this position, the driving time that the vehicle VE1 has been driving continuously up to now, the interior environment of the vehicle VE1, or the situation of surrounding vehicles relative to the vehicle VE1. Of course, the information detected by the server device 100 is not limited to the above example.

The in-vehicle environment of vehicle VE1 includes the state of the interior of the vehicle obtained by analyzing captured images, sounds produced by vocalization and speech, output sounds output from speakers, vibration conditions, vibration sounds produced by vibration, engine sounds, in-vehicle temperature, and in-vehicle humidity. The conditions of surrounding vehicles relative to vehicle VE1 include output sounds leaking from the speakers of surrounding vehicles, engine sounds of surrounding vehicles, horns and sirens of surrounding vehicles, etc.

In this way, when the server device 100 acquires sensor information and constantly detects the vehicle status of the vehicle VE1 based on the acquired sensor information, it can also determine whether the person to be detected, B12, is in the vehicle VE1 based on the detection results. For example, the server device 100 may determine whether the person to be detected, B12, is in the vehicle VE1 by comparing the appearance inside the vehicle (e.g., a person's face) or the sound produced by utterances or speech inside the vehicle with registered information.

Incidentally, as in the above example, even if information about the person to be detected (for example, facial image or voice information) has not been registered, the server device 100 may determine whether a person (young person) of an age group prone to state changes such as crying, fussing, making strange noises, or emotional changes is riding in the vehicle VE1, based on facial images and voice information detected from sensor information. Then, if the server device 100 determines that such a young person is riding in the vehicle, it may dynamically designate this young person as the detection target.

In the example of Figure 2, the server device 100 can determine that the person to be detected, B12, is riding in the vehicle VE1, and can perform subsequent processing in response to this determination.

For example, if the sensor information includes status information, the server device 100 detects the occurrence of a predetermined status change in person B12 based on that status information (step S13). For example, if the server device 100 detects vocalization information indicating the vocalization of person B12 from the sensor information, or speech information indicating the speech of person B12, the server device 100 executes processing to detect whether person B12 has started crying, become fussy, make strange noises, or have a change in emotion based on the detected information (audio information).

If the server device 100 detects the occurrence of a specified state change, it generates linking data that links content information indicating the content of the detected state change with status information indicating the vehicle status of the vehicle VE1 when the specified state change occurred, and stores the generated linking data (step S14).

For example, if the server device 100 detects that person B12 has started crying based on the status information (audio information) corresponding to person B12, it generates linking data that links content information indicating that person B12 has started crying with status information of vehicle VE1 at the time the crying occurred.

For example, based on the timing at which person B12 begins to cry, the server device 100 may generate linking data that links the content situation to situation information indicating any of the following: the position of vehicle VE1 within a predetermined time range before and after the time at which person B12 begins to cry; the interior environment of vehicle VE1 within this time range; the status of surrounding vehicles relative to vehicle VE1 within this time range; the elapsed time from when person B12 got into vehicle VE1 until when person B12 began to cry; or the status of driving breaks taken by person B12 between when person B12 got into vehicle VE1 and when he began to cry.

The server device 100 may also store the linking data in the linking information database 122 (Figure 5).

So far, we have explained the information processing according to the embodiment using as an example a scene in which person U11 is driving vehicle VE1 with person B12 in the back seat on the morning of October 10, 2021. From here on, we will explain the information processing according to the embodiment using as an example a scene in which person U11, who has finished driving, operates his or her user device 60 at home to use monitoring service SA to find out why person B12 started crying during the morning drive.

For example, suppose that person U11 launches the application AP and performs an operation to call a dialogue agent. In this case, the user device 60 sends a dialogue request to the server device 200 (step S21). Note that the operation to call a dialogue agent may be, for example, entering a dedicated dialogue room.

When the server device 200 accepts the dialogue request, it accesses the server device 100 and references the linked data corresponding to the requesting user, i.e., person U11 (step S22). For example, the server device 200 identifies the person to be detected that corresponds to person U11 based on the registration information registered by person U11, and references the linked data corresponding to the identified person. In the example of Figure 2, the server device 200 references the linked data corresponding to person B12.

Then, based on the linked data, the server device 200 generates a question for estimating the cause of the predetermined change in state of person B12 (step S23). In the example of FIG. 2, the server device 200 may identify possible causes for person B12's crying based on the linked data, and generate a question that prompts the user to answer either Yes or No to the identified possible causes.

Then, the server device 200 starts a dialogue with person B12 using the generated question (step S24). For example, the server device 200 may cause the user device 60 to display a chat screen C1 that virtually shows a dialogue room, and may also display the question as if an agent is speaking to the person.

Now, let us assume that person U11 inputs an answer to the question provided by the server device 200 (e.g., selects either Yes or No). In this case, the user device 60 transmits answer information indicating the input answer to the server device 200 (step S25). An example of a dialogue using a question will be described in detail in Figure 8.

Next, the server device 200 estimates the cause of the predetermined change in person B12's state based on the response entered by person U11 (step S26). In the example of Figure 2, the server device 200 estimates the underlying cause of person B12's crying based on the response information acquired from the user device 60.

Furthermore, if the server device 200 is able to estimate the cause of the predetermined change in person B12's state, it provides advice information corresponding to the estimation result to person U11 (step S27). In the example of FIG. 2, if the server device 200 is able to estimate the cause of person B12's crying, it searches for countermeasures to prevent person B12 from crying in the car in the future. The server device 200 then outputs advice information indicating the searched countermeasures to the user device 60. For example, the server device 200 may display the advice information on the chat screen C1, or may display the advice information on a predetermined screen corresponding to the app AP.

Up to this point, an example of information processing according to an embodiment has been shown using Figure 2. With this information processing, for example, when a predetermined status change occurs in a person to be detected (person B12) inside a vehicle, the user (person U11) can look back at the time the status change occurred and find the cause of the status change through dialogue with the agent. The user can also appropriately determine what measures to take to prevent future status changes from occurring inside the vehicle. As a result, the user can prevent situations in which driving safety is compromised due to status changes caused by the person to be detected.

3. Server Device Configuration (1)
The server device 100 according to the embodiment will now be described with reference to Fig. 3. Fig. 3 is a diagram showing an example of the configuration of the server device 100 according to the embodiment. As shown in Fig. 3, the server device 100 includes a communication unit 110, a storage unit 120, and a control unit 130.

(Regarding the communication unit 110)
The communication unit 110 is realized by, for example, a network interface card (NIC), etc. The communication unit 110 is connected to the network N by wire or wirelessly, and transmits and receives information to and from, for example, the in-vehicle device 10 and the server device 200.

(Regarding the storage unit 120)
The storage unit 120 is realized by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 120 has a sensor information database 121 and an association information database 122.

(Regarding the sensor information database 121)
The sensor information database 121 stores sensor information detected by a sensor included in the in-vehicle device 10 or a sensor included in the vehicle VEx itself. An example of the sensor information database 121 according to the embodiment is shown in Fig. 4. In the example of Fig. 4, the sensor information database 121 has items such as "vehicle ID,""detection date and time,""sensorinformation," and "situation information."

"Vehicle ID" indicates identification information that identifies the vehicle that sent the "sensor information." "Detection date and time" indicates information regarding the date and time when the "sensor information" was detected. "Sensor information" indicates information detected by a sensor possessed by the vehicle VEx indicated by the "Vehicle ID."

"Status information" is information indicating the status (vehicle status) of the vehicle VEx indicated by the "Vehicle ID." "Status information" may be, for example, information detected based on "sensor information," or it may be the "sensor information" itself.

Furthermore, the "situation information" may include, for example, location information indicating the location of the vehicle VEx indicated by the "vehicle ID", time information indicating the time at this location, time information indicating the driving time for which the vehicle VEx indicated by the "vehicle ID" has been driving continuously, environmental information indicating the in-vehicle environment of the vehicle VEx indicated by the "vehicle ID", and surrounding information indicating the status of surrounding vehicles relative to the vehicle VEx indicated by the "vehicle ID".

Furthermore, environmental information includes facial images of people inside the vehicle obtained by analyzing captured images (sensor information) of the interior of the vehicle VEx indicated by the "vehicle ID," audio information indicating vocalizations or speech by occupants of the vehicle VEx indicated by the "vehicle ID," output information indicating the output sound output from the speakers of the vehicle VEx indicated by the "vehicle ID," vibration information indicating the vibration of the vehicle VEx indicated by the "vehicle ID," vibration sound information indicating the vibration sound generated by this vibration, engine sound information indicating the engine sound of the vehicle VEx indicated by the "vehicle ID," temperature information indicating the interior temperature of the vehicle VEx indicated by the "vehicle ID," humidity information indicating the interior humidity of the vehicle VEx indicated by the "vehicle ID," etc.

Further examples of surrounding information include output information indicating the output sound leaking from the speakers of surrounding vehicles relative to the vehicle VEx indicated by the "vehicle ID," engine sound information indicating the engine sound of surrounding vehicles relative to the vehicle VEx indicated by the "vehicle ID," and horn information of surrounding vehicles relative to the vehicle VEx indicated by the "vehicle ID."

Figure 4 shows an example in which vehicle ID "VE1," detection date and time "TM11," sensor information "SC11," and situation information "ST11" are associated. This example shows an example in which a sensor possessed by vehicle VE1 (for example, a sensor possessed by an in-vehicle device 10 provided on vehicle VE1, or a sensor possessed by vehicle VE1 itself) detected sensor information SC11 at date and time TM11. This example also shows an example in which situation information ST11 was detected based on sensor information SC11.

(Regarding the linking information database 122)
The linking information database 122 stores linking data in which content information and status information are linked. Here, Fig. 5 shows an example of the linking information database 122 according to the embodiment. In the example of Fig. 5, the linking information database 122 has items such as "user information,""vehicleID,""target person information,""occurrence date and time,""contentinformation,""statusinformation," and "situation information." Furthermore, the "situation information" includes items such as "vehicle position before and after the occurrence,""vehicle environment before and after the occurrence,""surrounding conditions before and after the occurrence,""elapsed time until the occurrence," and "rest conditions before the occurrence."

"User information" refers to information about a user who has registered their information to receive the monitoring service SA. "User information" may include, for example, identification information, a profile, etc. that identifies the user who has registered their information to receive the monitoring service SA.

"Vehicle ID" is an example of registration information, and indicates, for example, identification information that identifies the vehicle driven by the user indicated by "User Information."

"Target person information" is an example of registration information, and indicates identification information that identifies the person who is the detection target (monitoring target) whose status change is detected. Furthermore, "target person information" may include, for example, a facial image of the detection target person and information about the voice of the detection target person.

"Occurrence date and time" indicates information about the date and time when a specified state change occurred. "Content information" indicates the content of the specified state change that occurred at the "Occurrence date and time." Figure 5 lists examples of the content of specified state changes such as crying, making strange noises, and emotional changes, but other examples include becoming fussy.

"Status information" is the information from which the status change indicated by "content information" is detected among the status information detected from the sensor information, and is information related to the status of the person being detected indicated by "target person information." "Status information" may be, for example, speech information indicating the speech of the person being detected indicated by "target person information," or speech information indicating the content of the speech of the person being detected indicated by "target person information." In other words, "status information" may be the audio information related to the person being detected indicated by "target person information" that is the audio information from which the status change is detected.

As explained above, status information detected from sensor information can also be considered a type of situation information in a broad sense.

Of course, the "status information" is not limited to the above example, and could also be, for example, a facial image showing a crying face or a facial image showing a displeased expression.

"Status information" is status information that indicates the vehicle status of the vehicle VEx when the status change indicated by "content information" occurs.

"Vehicle position before and after occurrence" is information indicating the position of the vehicle VEx within a specified time range before and after the time when the state change indicated by the "content information" occurred.

"Ambient conditions before and after the occurrence" is information indicating the interior environment of the vehicle VEx within a specified time range before and after the time when the state change indicated by the "content information" occurred.

"Time elapsed until occurrence" is information indicating the amount of time that has elapsed since the person to be detected, indicated by the "Target Person Information," got into the vehicle VEx until the state change indicated by the "Content Information" occurred.

"Rest status up to the time of occurrence" is information indicating the driving rest status taken by the person to be detected, indicated by "target person information," from the time they got into the vehicle VEx until the time the status change indicated by "content information" occurred.

(Regarding the control unit 130)
3, the control unit 130 is realized by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or the like executing various programs (e.g., the information processing program according to the embodiment) stored in a storage device inside the server device 100 using RAM as a work area. The control unit 130 is also realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

As shown in FIG. 3, the control unit 130 has an acquisition unit 131, a first detection unit 132, a second detection unit 133, a storage unit 134, a generation unit 135, and a prediction unit 136, and realizes or executes the information processing functions and actions described below. Note that the internal configuration of the control unit 130 is not limited to the configuration shown in FIG. 3, and may be other configurations that perform the information processing described below. Furthermore, the connection relationships between the processing units in the control unit 130 are not limited to the connection relationships shown in FIG. 3, and may be other connection relationships.

(Regarding the Acquisition Unit 131)
The acquisition unit 131 acquires sensor information detected by a sensor of the vehicle VEx (for example, a sensor of the in-vehicle device 10 provided in the vehicle VEx, or a sensor of the vehicle VEx itself), and stores the acquired sensor information in the sensor information database 121.

(Regarding the first detection unit 132)
The first detection unit 132 detects the occurrence of a predetermined state change in the occupant of the vehicle VEx based on state information indicating the state of the occupant. For example, if the sensor information acquired by the acquisition unit 131 includes information about a person (e.g., vocal information, speech information, or a facial image), the first detection unit 132 acquires this information as state information indicating the state of the occupant. Then, the first detection unit 132 performs processing to detect the occurrence of a predetermined state change in the occupant based on the acquired state information. For example, the first detection unit 132 detects, based on the state information, the predetermined state change as the occupant starting to cry, uttering a strange voice, or a change in the occupant's emotions.

(Regarding the second detection unit 133)
The second detection unit 133 detects a situation related to the vehicle VEx. For example, the second detection unit 133 may detect the position of the vehicle VEx, the time at this position, the continuous traveling time of the vehicle VEx, the interior environment of the vehicle VEx, or the situation of surrounding vehicles relative to the vehicle VEx, based on the sensor information acquired by the acquisition unit 131. Furthermore, the second detection unit 133 stores situation information indicating the detected vehicle situation in the sensor information database 121.

(Regarding the storage unit 134)
The storage unit 134 generates linking data that links content information indicating the content of the predetermined state change with situation information indicating the situation regarding the vehicle VEx when the predetermined state change occurs. Then, the storage unit 134 stores the generated linking data in the linking information database 122.

For example, based on the timing at which a predetermined state change occurs, the accumulation unit 134 may generate linked data in which situation information indicating any of the following, among the situations detected by the second detection unit 133, is linked to content information: the position of the vehicle VEx within a predetermined time range relative to the time at which the predetermined state change occurs; the in-vehicle environment of the vehicle VEx within this time range; the status of surrounding vehicles relative to the vehicle VEx within this time range; the elapsed time elapsed since the passenger boarded the vehicle VEx until the predetermined state change occurred; or the status of the passenger taking a driving break between the time at which the passenger boarded the vehicle VEx and the time at which the predetermined state change occurred.

Furthermore, the storage unit 134 may store, among the status information detected from the sensor information, status information detected when a predetermined status change occurs, in a state in which the status information is further associated with the linking data. As a result, a linking information database 122 such as that shown in FIG. 5 is obtained.

(Regarding the generation unit 135)
The generation unit 135 generates a predictive model that predicts whether a predetermined state change will occur for a passenger when situation information indicating the situation regarding the vehicle VEx currently in which the passenger is riding is input, by having the model learn the relationship between the content information contained in the linked data and the situation information.

(Regarding the prediction unit 136)
The prediction unit 136 predicts whether a predetermined state change will occur in the passenger to be processed, based on a trained model that has learned the relationship between the content information and the state information included in the linked data, and the state information that indicates the state of the vehicle that the passenger to be processed is currently riding in. For example, the prediction unit 136 predicts whether a predetermined state change will occur in the passenger to be processed, using the prediction model generated by the generation unit 135 as the trained model.

For example, let us say that the passenger to be processed (the passenger for whom situation information indicating the situation regarding the vehicle VE1 in which the person U11 is currently riding is input) is person B12 in Figure 2. In this case, the prediction unit 136 can predict whether person B12 will start crying by inputting situation information detected based on sensor information acquired in real time while person U11 is driving the vehicle VE1 into a prediction model. Furthermore, if the prediction unit 136 predicts that person B12 will start crying, it may cause the in-vehicle device 10 to output, for example, that there is a high possibility that person B12 will start crying if the driving situation remains the same.

Furthermore, the prediction unit 136 may predict that person B12 is likely to start crying and may also predict the cause of this.

4. Server Device Configuration (2)
Next, the server device 200 according to the embodiment will be described with reference to Fig. 6. Fig. 6 is a diagram showing an example of the configuration of the server device 200 according to the embodiment. As shown in Fig. 6, the server device 200 includes a communication unit 210, a storage unit 220, and a control unit 230.

(Regarding the communication unit 860)
The communication unit 210 is realized by, for example, a NIC, etc. The communication unit 210 is connected to the network N by wire or wirelessly, and transmits and receives information to and from, for example, the user device 60 and the server device 100.

(Regarding the storage unit 120)
The storage unit 120 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk, an optical disk, etc. The storage unit 120 has a question sentence information database 221 and a conversation information database 222.

(Question Information Database 221)
The question message information database 221 stores information about questions used in dialogue with users. An example of the question message information database 221 according to the embodiment is shown in FIG. 7 . In the example of FIG. 7 , the question message information database 221 has items such as "Question ID,""Question Message Candidate," and "Target Age." The "Target Age" includes items such as "Under 1 Year Old (Toddlers),""1 to 3 Years Old (Preschoolers)," and "3 to 6 Years Old (Kindergarteners)."

"Question ID" indicates identification information that identifies the question candidate.

"Question candidate" is a candidate question provided to a user who has requested a dialogue. The question indicated by the "question candidate" may, for example, include content intended to obtain from the requesting user an answer to estimate the cause of a specified change in the state of the person being detected. The question indicated by the "question candidate" may also include content that causes the requesting user to reflect on the time when a specified change in state occurred in the person being detected.

"Target age" is condition information that determines which question should be selected from the "candidate questions" to be provided to the user, based on the maturity level of the person being detected.

For example, in the example in Figure 7, a "Yes" is entered between "Under 1 year old (toddler)" and the question "Is it because he wanted milk?". On the other hand, a "Yes" is not entered between "1 to 3 years old (preschooler)" and the question "Is it because he wanted milk?". Also, a "Yes" is not entered between "3 to 6 years old (kindergartener)" and the question "Is it because he wanted milk?".

In this example, if the age of the person to be detected (person B12 in the example of Figure 2) corresponding to the requesting user (person U11 in the example of Figure 2) is between "1 and 3 years old", the question "Is it because you wanted milk?" is provided to that user, but if the age of the person to be detected is between "1 and 3 years old" or "3 and 6 years old", the question "Is it because you wanted milk?" is not provided to that user.

In another example of Figure 7, a "Yes" is entered between "1 to 3 years old (preschooler)" and the question "Is it because he's bored of the drive?". A "Yes" is also entered between "3 to 6 years old (kindergartener)" and the question "Is it because he's bored of the drive?". On the other hand, a "Yes" is not entered between "Under 1 year old (toddler)" and the question "Is it because he's bored of the drive?".

In this example, if the age of the person to be detected corresponding to the requesting user is between "1 and 3 years old" or "3 and 6 years old", the question "Are you bored of driving?" is provided to that user, but if the age of the person to be detected is between "under 1 year old", the question "Are you bored of driving?" is not provided to that user.

In this way, "target age" can be thought of as condition information that conditions which questions, among those indicated by the "candidate questions," are more suitable to be provided to the user from the perspective of the maturity level of the person being detected.

(Regarding the dialogue information database 222)
The interaction information database 222 stores information about interactions between the requesting user and the requesting user. For example, the interaction information database 222 may store history information about interactions between the requesting user and the requesting user.

(Regarding the control unit 230)
6, the control unit 230 is realized by a CPU, an MPU, or the like executing various programs (e.g., the information processing program according to the embodiment) stored in a storage device inside the server device 200 using RAM as a work area. The control unit 230 is also realized by an integrated circuit such as an ASIC or FPGA.

As shown in FIG. 6, the control unit 230 has a request receiving unit 231, a generating unit 232, an estimating unit 233, and an output control unit 234, and realizes or executes the information processing functions and actions described below. Note that the internal configuration of the control unit 230 is not limited to the configuration shown in FIG. 6, and may be other configurations as long as they perform the information processing described below. Furthermore, the connection relationships between the processing units in the control unit 230 are not limited to the connection relationships shown in FIG. 6, and may be other connection relationships.

(Regarding the request receiving unit 231)
The request receiving unit 231 receives, via the dialogue agent, a dialogue request from the user, which is a request to dialogue with the server device 200. When the dialogue request is received, the request receiving unit 231 may generate a chat screen as a dedicated dialogue room for realizing the dialogue, and may even perform processing to display the generated chat screen on the user device 60.

(Regarding the generation unit 232)
The generation unit 232 generates user information for a predetermined user related to the passenger based on the linked data. For example, the generation unit 232 generates a question sentence including potential causes as content for obtaining an answer from the user used to estimate the cause of the occurrence of a predetermined state change.

Furthermore, for example, the generation unit 232 generates a question based on the linked data corresponding to a passenger (e.g., a person to be detected) belonging to the requesting user who requested the provision of user information, the question including content that causes the requesting user to look back on the time when a specified state change occurred.

More specifically, the generation unit 232 identifies possible causes of a specified state change based on the linked data, and generates a question asking whether the identified causes are true or false.

The generation unit 232 may also generate questions with content that corresponds to the passenger's maturity level.

(Regarding the estimation unit 233)
When the requesting user who has been provided with the user information responds to the user information, the estimation unit 233 estimates the cause of the predetermined state change based on the content of the response. Specifically, when the requesting user inputs an answer to a question, the estimation unit 233 estimates the cause of the predetermined state change in the passenger belonging to the requesting user based on the answer.

(Regarding the output control unit 234)
The output control unit 234 controls so that information according to the estimation result by the estimation unit 233 is output to the user. Specifically, the output control unit 234 controls so that information according to the estimation result by the estimation unit 233 is output to the requesting user.

For example, the output control unit 234 controls the output of advice information regarding a specified state change as information corresponding to the estimation result. For example, the output control unit 234 controls the output of advice information regarding countermeasures to prevent the specified state change from occurring.

[5. Example of a dialogue using a question]
Next, an example of a process in which a question is generated by the generation unit 232 and an example of a dialogue between the requesting user and the server device 200 using the generated question will be described with reference to Fig. 8. Fig. 8 is a diagram showing an example of a dialogue between the server device 200 and a user. While an example in which a chat screen is displayed is shown below, the dialogue may be conducted using only voice.

Furthermore, Figure 8 illustrates an example of a dialogue between the server device 200 and person U11, using the example of steps S21 to S27 described in Figure 2. Therefore, in the example of Figure 8, person U11 corresponds to the requesting user. Also, Figure 8 shows that in response to a dialogue request from person U11, a chat screen C1 virtually showing a dialogue room is displayed on person U11's user device 60.

In this state, the generation unit 232 accesses the linking information database 122 of the server device 100 and references the linking data corresponding to the requesting user, i.e., person U11. For example, the generation unit 232 identifies the person to be detected corresponding to person U11 based on the registration information registered by person U11, and references the linking data corresponding to the identified person. According to the examples above, the generation unit 232 identifies person B12 as the person to be detected corresponding to person U11, and references the linking data corresponding to the identified person.

In the example of Figure 5, the generation unit 232 can determine person B12 as the person to be detected corresponding to person U11 based on the "target person information" associated with user information "U11."

Here, the generation unit 232 controls the question text depending on the maturity level of person B12 and the status change that has occurred to person B12. Therefore, below, three patterns are prepared for the combination of person B12's age and the status change that has occurred to person B12, and each pattern is explained using Figures 8(a), 8(b), and 8(c).

First, let us consider Figure 8(a). In Figure 8(a), person B12's age is assumed to be "less than one year old," and therefore person B12's maturity level corresponds to "infant." Also, in Figure 8(a), it is assumed that a state change of "started crying" has occurred in person B12 while inside vehicle VE1 driven by person U11.

In this example, the generation unit 232 identifies possible causes for person B12 starting to cry based on the linked data corresponding to person B12 among the linked data stored in the linked information database 122, specifically the "status information" and "situation information" corresponding to person B12. For example, the generation unit 232 may perform processing to identify possible causes for person B12 starting to cry based on the "status information," and if no possible causes can be identified, may perform processing to identify possible causes for person B12 starting to cry based on the "situation information."

In the example of Figure 8(a), if the "status information" is audio information indicating the utterance of person B12, "Milk!", the generation unit 232 can identify "request for milk" as a possible cause of person B12's crying.

The generation unit 232 also extracts question sentences that satisfy the maturity level of "infant" from among the candidate question sentences registered in the question sentence information database 221. According to the question sentence information database 221 shown in FIG. 7, the generation unit 232 can extract questions Q1, Q2, Q3, Q4, Q6, and Q9 as question sentences that satisfy the maturity level of "infant."

Then, the generation unit 232 selects, from among questions Q1, Q2, Q3, Q4, Q6, and Q9, question Q1, which asks whether the reason person B12 started crying was because he was "requesting milk," as the question to be provided.

Then, based on the selected question Q1, the generation unit 232 generates a question Q11 in a Yes/No format, such as "Did you start crying while driving because you wanted milk? Yes/No." In this way, question Q11 is a question for obtaining an answer from person U11 that will serve as evidence for the reason why person B12 started crying, and also has the effect of making person U11 look back on the time when person B12 started crying.

Furthermore, the generation unit 232 starts a dialogue with person U11 by displaying question Q11 on the chat screen C1, as shown in FIG. 8(a). Here, as shown in FIG. 8(a), it is assumed that person U11 answers "No!" to question Q11.

In such a case, the generation unit 232 may further generate an additional question to dig deeper into the reason why person B12 started crying, depending on person U11's answer "No!". For example, the generation unit 232 may select a predetermined question from the remaining questions that satisfy the maturity status "infant," i.e., questions Q2, Q3, Q4, Q6, and Q9. For example, the generation unit 232 may select a question from questions Q2, Q3, Q4, Q6, and Q9 that is closest to the reason why person B12 started crying, based on the "status information" and "situation information" corresponding to person B12. In the example of FIG. 8(a), it is assumed that the generation unit 232 selected question Q2.

In this case, the generation unit 232 generates a question Q21 in a Yes/No format, such as "So, is it because the diaper was soiled? Yes/No," based on the selected question Q2.

Then, the generation unit 232 continues the dialogue by displaying question Q21 on the chat screen C1, as shown in FIG. 8(a). Here, as shown in FIG. 8(a), it is assumed that person U11 answers "No!" to question Q21.

In such a case, the generation unit 232 may generate an additional question to further dig into the reason why person B12 started crying, in response to person U11's answer "No!". For example, the generation unit 232 may select a predetermined question from the remaining questions that satisfy the maturity status "infant," i.e., questions Q3, Q4, Q6, and Q9. For example, the generation unit 232 may select a question from questions Q3, Q4, Q6, and Q9 that is closer to the reason why person B12 started crying, based on the "status information" and "situation information" corresponding to person B12. In the example of FIG. 8(a), it is assumed that the generation unit 232 selected question Q3.

In this case, the generation unit 232 generates a question Q31 in a Yes/No format, such as "So, was it because you were surprised by the sudden braking? Yes/No," based on the selected question Q3.

Then, the generation unit 232 continues the dialogue by displaying question Q31 on the chat screen C1, as shown in FIG. 8(a). Here, as shown in FIG. 8(a), it is assumed that person U11 answers "Yes!" to question Q31.

In such a case, the estimation unit 233 can estimate that the reason why person B12 started crying was "surprise at the sudden braking." Furthermore, the output control unit 234 searches for countermeasures to prevent person B12 from starting to cry in the car in the future, based on the reason why person B12 started crying, "surprise at the sudden braking." The output control unit 234 then controls the output so that advice information according to the search results is output to the user device 60.

As a result, for example, when person B12 is driving the vehicle, the output control unit 234 can warn person U11 not to brake suddenly and provide advice information to person U11 showing specific ways to avoid braking suddenly.

Next, we will explain Figure 8(b). In Figure 8(b), person B12's age is assumed to be "1 to 3 years old," and therefore person B12's maturity level corresponds to "kindergartener." Also, in Figure 8(b), it is assumed that a state change of "starting to make noise" has occurred in person B12 while inside vehicle VE1 driven by person U11.

In this example, the generation unit 232 identifies possible causes for person B12 starting to make a fuss based on the linked data corresponding to person B12 among the linked data stored in the linked information database 122, specifically the "status information" and "situation information" corresponding to person B12. For example, the generation unit 232 may perform a process to identify possible causes for person B12 starting to make a fuss based on the "status information," and if no possible causes can be identified, may perform a process to identify possible causes for person B12 starting to make a fuss based on the "situation information."

In the example of Figure 8 (b), if the "status information" is speech information indicating the speech content of person B12, "I'm bored!!", the generation unit 232 can identify "long ride" as a possible cause of person B12's fuss.

The generation unit 232 also extracts question sentences that satisfy the maturity level of "kindergartener" from among the candidate question sentences registered in the question sentence information database 221. According to the question sentence information database 221 shown in FIG. 7, the generation unit 232 can extract questions Q3, Q4, Q5, Q6, Q7, Q8, and Q9 as question sentences that satisfy the maturity level of "kindergartener."

Then, the generation unit 232 selects question Q5, which asks whether the cause of person B12's fussing was the "long ride," as the question to be provided from among questions Q3, Q4, Q5, Q6, Q7, Q8, and Q9.

Then, based on the selected question Q5, the generation unit 232 generates a question Q51 in a Yes/No format, such as "Did you start making noise while driving because you were bored? Yes/No." In this way, question Q51 is a question intended to obtain from person U11 an answer that will serve as evidence for the reason why person B12 started making noise, and also has the effect of making person U11 look back on the time when person B12 started making noise.

Furthermore, the generation unit 232 starts a dialogue with person U11 by displaying question Q51 on the chat screen C1, as shown in FIG. 8(b). Here, as shown in FIG. 8(b), it is assumed that person U11 responds to question Q51 with "No!".

In such a case, the generation unit 232 may further generate an additional question to dig deeper into the reason why person B12 started making a fuss, depending on person U11's answer "No!". For example, the generation unit 232 may select a predetermined question from the remaining questions that satisfy the maturity status "kindergartener," i.e., questions Q3, Q4, Q6, Q7, Q8, and Q9. For example, the generation unit 232 may select a question from questions Q3, Q4, Q6, Q7, Q8, and Q9 that is closer to the reason why person B12 started making a fuss, based on the "status information" and "situation information" corresponding to person B12. In the example of FIG. 8(b), it is assumed that the generation unit 232 selected question Q7.

In this case, the generation unit 232 generates a question Q71 in a Yes/No format, such as "So, was it because you wanted to go to the toilet? Yes/No," based on the selected question Q7.

Then, the generation unit 232 continues the dialogue by displaying question Q71 on the chat screen C1, as shown in FIG. 8(b). Here, as shown in FIG. 8(b), it is assumed that person U11 responds "No!" to question Q71.

In such a case, the generation unit 232 may generate an additional question to further dig into the reason why person B12 started making a fuss, in response to person U11's answer "No!". For example, the generation unit 232 may select a predetermined question from the remaining questions that satisfy the maturity status "kindergartener," i.e., questions Q3, Q4, Q6, Q8, and Q9. For example, the generation unit 232 may select a question from questions Q3, Q4, Q6, Q8, and Q9 that is closer to the reason why person B12 started making a fuss, based on the "status information" and "situation information" corresponding to person B12. In the example of FIG. 8(b), it is assumed that the generation unit 232 selected question Q8.

In this case, the generation unit 232 generates a question Q81 in a Yes/No format, such as "So, is it because you're hungry? Yes/No," based on the selected question Q8.

Then, the generation unit 232 continues the dialogue by displaying question Q81 on the chat screen C1, as shown in FIG. 8(b). Here, as shown in FIG. 8(b), it is assumed that person U11 answers "Yes!" to question Q81.

In such a case, the estimation unit 233 can estimate that "hunger" is the reason why person B12 started making a fuss. Furthermore, the output control unit 234 searches for countermeasures to prevent person B12 from making a fuss in the future while in the car, based on the reason "hunger" that person B12 started making a fuss. The output control unit 234 then controls the output so that advice information according to the search results is output to the user device 60.

As a result, the output control unit 234 can provide advice information to person U11, for example, when driving with person B12 in the car, suggesting that person U11 take a break from time to time and give snacks.

Next, we will explain Figure 8(c). In Figure 8(c), person B12's age is assumed to be "less than one year old," and therefore person B12's maturity level corresponds to "infant." Also, in Figure 8(c), it is assumed that a state change of "started crying" has occurred in person B12 while inside vehicle VE1 driven by person U11.

In this example, the generation unit 232 identifies possible causes for person B12 starting to cry based on the linked data corresponding to person B12 among the linked data stored in the linked information database 122, specifically the "status information" and "situation information" corresponding to person B12. For example, the generation unit 232 may perform processing to identify possible causes for person B12 starting to cry based on the "status information," and if no possible causes can be identified, may perform processing to identify possible causes for person B12 starting to cry based on the "situation information."

In the example of Figure 8 (c), if the "situation information" is peripheral information indicating the "beep-beep" siren sound of person B12, the generation unit 232 can identify "surprise at the siren of a nearby vehicle" as a possible cause of person B12 starting to cry.

The generation unit 232 also extracts question sentences that satisfy the maturity level of "young child" from among the candidate question sentences registered in the question sentence information database 221. According to the question sentence information database 221 shown in FIG. 7, the generation unit 232 can extract questions Q1, Q2, Q3, Q4, Q6, and Q9 as question sentences that satisfy the maturity level of "kindergartener."

Then, the generation unit 232 selects, from among questions Q1, Q2, Q3, Q4, Q6, and Q9, question Q6, which asks whether the reason person B12 started crying was because he was "surprised by the siren of a nearby vehicle," as the question to be provided.

Then, based on the selected question Q6, the generation unit 232 generates a question Q61 in a Yes/No format, such as "Did you start crying while driving because you were startled by the siren? Yes/No." In this way, question Q61 is a question intended to obtain from person U11 an answer that will serve as evidence for the reason why person B12 started crying, and also has the effect of prompting person U11 to look back on the time when person B12 started crying.

Furthermore, the generation unit 232 starts a dialogue with person U11 by displaying question Q61 on the chat screen C1, as shown in FIG. 8(c). Here, as shown in FIG. 8(c), it is assumed that person U11 responds to question Q61 with "No!".

In such a case, the generation unit 232 may further generate an additional question to dig deeper into the reason why person B12 started crying, depending on person U11's answer "No!". For example, the generation unit 232 may select a predetermined question from the remaining questions that satisfy the maturity status "infant," i.e., questions Q1, Q2, Q3, Q4, and Q9. For example, the generation unit 232 may select a question from questions Q1, Q2, Q3, Q4, and Q9 that is closest to the reason why person B12 started crying, based on the "status information" and "situation information" corresponding to person B12. In the example of FIG. 8(c), it is assumed that the generation unit 232 selected question Q3.

In this case, the generation unit 232 generates a question Q31 in a Yes/No format, such as "So, was it because you were surprised by the sudden braking? Yes/No," based on the selected question Q3.

Then, the generation unit 232 continues the dialogue by displaying question Q31 on the chat screen C1, as shown in FIG. 8(c). Here, as shown in FIG. 8(c), it is assumed that person U11 answers "No!" to question Q31.

In such a case, the generation unit 232 may generate an additional question to further dig into the reason why person B12 started crying, in response to person U11's answer "No!". For example, the generation unit 232 may select a predetermined question from the remaining questions that satisfy the maturity status "infant," i.e., questions Q1, Q2, Q4, and Q9. For example, the generation unit 232 may select a question from questions Q1, Q2, Q4, and Q9 that is closer to the reason why person B12 started crying, based on the "status information" and "situation information" corresponding to person B12. In the example of FIG. 8(c), it is assumed that the generation unit 232 selected question Q4.

In this case, the generation unit 232 generates a question Q41 in a Yes/No format, such as "So, was it because it was hot? Yes/No," based on the selected question Q4.

Then, the generation unit 232 continues the dialogue by displaying question Q41 on the chat screen C1, as shown in FIG. 8(c). Here, as shown in FIG. 8(c), it is assumed that person U11 answers "Yes!" to question Q41.

In such a case, the estimation unit 233 can estimate that "interior temperature rise" is the cause of person B12's crying. Furthermore, the output control unit 234 searches for countermeasures to prevent person B12 from making a fuss in the car in the future, based on the cause of person B12's crying, "interior temperature rise." The output control unit 234 then controls the output so that advice information according to the search results is output to the user device 60.

As a result, for example, when person B12 gets into the vehicle when the temperature is high, the output control unit 234 can provide person U11 with advice information that warns the person to be careful about the temperature inside the vehicle and advises the person not to leave person B12 inside the vehicle.

6. Processing Procedure
Next, the procedure of information processing realized by the server device 100 will be described with reference to Figures 9 and 10. The information processing procedure can be broadly divided into two steps: a detection step for detecting the vehicle status based on sensor information, and a step for linking, when a status change is detected based on status information included in the sensor information, content information indicating the status change with the vehicle status information at that time. Therefore, the procedure of the vehicle status detection step will be described with reference to Figure 9, and the procedure of the linking step will be described with reference to Figure 10.

In Figures 9 and 10, information processing by the server device 100 is explained using an example of a scene in which person U11, the mother of person B11, is driving a vehicle VE1, assuming that information about person B12, the person to be detected, is registered in the server device 100.

6-1. Processing Procedure (1)
First, the procedure of information processing performed in the vehicle condition detection step will be described with reference to Fig. 9. Fig. 9 is a flowchart showing the procedure of the detection process for detecting the vehicle condition.

In the example of FIG. 9, first, the acquisition unit 131 determines whether or not driving of the vehicle VE1 has started (step S901). While the acquisition unit 131 determines that driving of the vehicle VE1 has not started (step S901; No), it waits until it can determine that driving of the vehicle VE1 has started.

On the other hand, if the acquisition unit 131 determines that driving of the vehicle VE1 has started (step S901; Yes), it acquires sensor information sent in real time from the in-vehicle device 10 in accordance with the driving of the vehicle VE1 (step S902). Furthermore, the acquisition unit 131 may store the acquired sensor information in the sensor information database 121.

Each time sensor information is acquired by the acquisition unit 131, the second detection unit 133 detects the vehicle situation regarding the vehicle VEx based on the acquired sensor information (step S903). For example, the second detection unit 133 may detect the position of the vehicle VE1, the time at this position, the continuous driving time of the vehicle VE1, the in-vehicle environment of the vehicle VE1, or the situation of surrounding vehicles relative to the vehicle VE1. The second detection unit 133 may also store situation information indicating the detected vehicle situation in the sensor information database 121.

Next, the second detection unit 133 determines whether driving of the vehicle VE1 has ended (step S904). For example, the second detection unit 133 may determine whether driving of the vehicle VE1 has ended based on whether sensor information has been received from the acquisition unit 131. For example, the second detection unit 133 determines that driving of the vehicle VE1 is continuing while it is able to receive sensor information from the acquisition unit 131 (step S904; No) and executes the processing again from step S903. On the other hand, if the second detection unit 133 is no longer able to receive sensor information from the acquisition unit 131, it determines that driving of the vehicle VE1 has ended (step S904; Yes) and ends the processing.

[6-2. Processing Procedure (2)]
Next, the procedure of information processing performed in the linking step will be described with reference to Fig. 10. Fig. 10 is a flowchart showing the procedure of the linking process for linking content information with situation information.

First, the first detection unit 132 determines whether or not the person B12 to be detected is present among the occupants of the vehicle VE1 based on the sensor information acquired by the acquisition unit 131 (step S1001). For example, if the sensor information includes information about a person (e.g., vocal information, speech information, facial image, etc.), the first detection unit 132 may determine whether or not the person B12 to be detected is present among the occupants of the vehicle VE1 by comparing the information about this person with registered information.

If the first detection unit 132 determines that the person to be detected, B12, is not present among the occupants of the vehicle VE1 (step S1001; No), it may terminate the linking process.

On the other hand, if the first detection unit 132 determines that the person to be detected, B12, is present among the occupants of the vehicle VE1 (step S1001; Yes), it determines whether the sensor information acquired from time to time by the acquisition unit 131 includes information about the person (step S10002). For example, the first detection unit 132 may determine whether the information about the person includes speech information indicating speech made by the occupant of the vehicle VE1, speech information indicating the content of speech made by the occupant of the vehicle VE1, or a facial image of the occupant of the vehicle VE1.

If the first detection unit 132 determines that the sensor information does not include information about a person (step S1002; No), it waits until sensor information including information about a person is acquired.

On the other hand, if the first detection unit 132 determines that the sensor information includes information about a person (step S1002; Yes), it determines whether the information about that person indicates person B12, the person to be detected (step S1003).

If the first detection unit 132 determines that the information about the person does not indicate the person B12 to be detected (step S1003; No), it returns to step S1002.

On the other hand, if the first detection unit 132 determines that the information about the person indicates the person B12 to be detected (step S1003; Yes), it acquires this information as status information indicating the status of the person B12 to be detected (step S1004).

Then, the first detection unit 132 performs processing to detect the occurrence of a predetermined state change in the person B12 to be detected based on the acquired state information (step S1005). For example, the first detection unit 132 detects, based on the state information, whether the person to be detected has started crying, become fussy, make a fuss, or make strange noises, or whether they have changed their emotions (for example, become displeased).

Next, the accumulation unit 134 determines whether the first detection unit 132 has detected the occurrence of a predetermined state change (step S1006). If the accumulation unit 134 determines that the occurrence of a predetermined state change has not been detected (step S1006; No), the process returns to step S1002.

On the other hand, if the accumulation unit 134 determines that the occurrence of a predetermined state change has been detected (step S1006; Yes), it generates linking data that links content information indicating the content of the predetermined state change with situation information regarding the vehicle VE1 when the predetermined state change occurred (step S1007).

For example, based on the timing at which a predetermined state change occurs, the accumulation unit 134 may generate linked data in which situation information indicating any of the following, among the vehicle conditions detected by the second detection unit 133, the position of vehicle VE1 within a predetermined time range relative to the time at which the predetermined state change occurred, the in-vehicle environment of vehicle VE1 within this time range, the status of surrounding vehicles relative to vehicle VE1 within this time range, the elapsed time since person B12 boarded vehicle VE1 until the predetermined state change occurred, or the status of driving breaks taken by person B12 between boarding vehicle VE1 and the time at which the predetermined state change occurred, is linked to content information.

The storage unit 134 also stores the generated linking data in the linking information database 122 (step S1008).

7. Modifications
The information processing apparatuses (server apparatus 100, server apparatus 200) according to the above-described embodiments may be implemented in various different forms other than the above-described embodiments. Therefore, other embodiments of the information processing apparatus according to the embodiments will be described below.

In the above embodiment, an example was shown in which the estimation unit 233 estimates the cause of a predetermined status change from the user's response to user information generated based on the linking data. In this way, when the cause of a status change is estimated, the accumulation unit 134 may not only link content information indicating the content of the status change with status information indicating the status of the vehicle when the status change occurred, but may also generate linking data by linking cause information indicating the cause of the occurrence (e.g., lack of milk, dirty diaper, long ride, etc.).

Furthermore, the generation unit 135 may generate a prediction model that not only predicts whether a predetermined state change will occur based on the above-mentioned linked data, but also predicts the possible causes of this predetermined state change.

The prediction unit 136 can then use the prediction model to predict, for example, the timing at which person B12 will start crying and the cause of the crying.

Furthermore, when the prediction unit 136 predicts that the cause of crying is "lack of milk" using a prediction model, the output control unit 234 may provide information suggestion such as "The baby is about to cry, so stop the vehicle and encourage the baby to drink milk."Furthermore, when the prediction unit 136 predicts that the cause of crying is "diaper soiling due to excrement" using a prediction model, the output control unit 234 may provide information suggestion such as "The baby is about to cry, so stop the vehicle and encourage the baby to be changed."

As another example, by using a prediction model, the prediction unit 136 can predict, for example, the timing at which person B12 will start making a fuss and the reason why he or she will start making a fuss.

Furthermore, if the prediction unit 136 predicts that a "long ride" is the cause of fussiness using a prediction model, the output control unit 234 may provide information suggestions such as "encouraging the driver to take a break in advance," "encouraging the driver to watch a video to watch in advance," or "encouraging the driver to engage in recreational activities that can be practiced in the car (for example, word games, simple games, etc.)."

With this modified information processing device, measures to prevent an incident involving a passenger (e.g., person B12) can be presented to relevant parties (e.g., person U11) and encouraged to be implemented, thereby effectively preventing incidents from occurring inside the vehicle.

8. Hardware Configuration
The server devices 100 and 200 according to the above-described embodiments are realized, for example, by a computer 1000 configured as shown in Fig. 11. The following description will be given using the server device 100 as an example. Fig. 11 is a hardware configuration diagram showing an example of a computer that realizes the functions of the server device 100. The computer 1000 has a CPU 1100, a RAM 1200, a ROM 1300, a HDD 1400, a communication interface (I/F) 1500, an input/output interface (I/F) 1600, and a media interface (I/F) 1700.

The CPU 1100 operates based on programs stored in the ROM 1300 or HDD 1400, and controls each component. The ROM 1300 stores a boot program executed by the CPU 1100 when the computer 1000 starts up, as well as programs that depend on the computer 1000's hardware.

HDD 1400 stores programs executed by CPU 1100 and data used by such programs. Communication interface 1500 receives data from other devices via a specified communication network and sends it to CPU 1100, and transmits data generated by CPU 1100 to other devices via a specified communication network.

The CPU 1100 controls output devices such as displays and printers, and input devices such as keyboards and mice, via the input/output interface 1600. The CPU 1100 acquires data from input devices via the input/output interface 1600. The CPU 1100 also outputs generated data to output devices via the input/output interface 1600.

Media interface 1700 reads programs or data stored on recording medium 1800 and provides them to CPU 1100 via RAM 1200. CPU 1100 loads the programs from recording medium 1800 onto RAM 1200 via media interface 1700 and executes the loaded programs. Recording medium 1800 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase Change Rewritable Disc), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

For example, when the computer 1000 functions as the server device 100 according to the embodiment, the CPU 1100 of the computer 1000 executes programs loaded onto the RAM 1200 to realize the functions of the control unit 130. The CPU 1100 of the computer 1000 reads and executes these programs from the recording medium 1800, but as another example, the CPU 1100 may obtain these programs from another device via a specified communications network.

Furthermore, when the computer 1000 functions as the server device 200 according to the embodiment, the CPU 1100 of the computer 1000 executes programs loaded onto the RAM 1200, thereby realizing the functions of the control unit 230. The CPU 1100 of the computer 1000 reads and executes these programs from the recording medium 1800, but as another example, the CPU 1100 may obtain these programs from another device via a specified communications network.

[9. Other]
Furthermore, among the processes described in each of the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using known methods. In addition, the information including the processing procedures, specific names, various data, and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified. For example, the various information shown in each drawing is not limited to the information shown in the drawings.

Furthermore, the components of each device shown in the figure are functional concepts and do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of each device is not limited to that shown, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc.

Furthermore, the above embodiments can be combined as appropriate to the extent that the processing content is not contradictory.

The above describes in detail some of the embodiments of the present application based on the drawings, but these are merely examples, and the present invention can be implemented in other forms that incorporate various modifications and improvements based on the knowledge of those skilled in the art, including the aspects described in the Disclosure of the Invention section.

REFERENCE SIGNS LIST 1 Information processing system 10 In-vehicle device 60 User device 100 Server device 120 Storage unit 121 Sensor information database 122 Linking information database 130 Control unit 131 Acquisition unit 132 First detection unit 133 Second detection unit 134 Accumulation unit 135 Generation unit 136 Prediction unit 200 Server device 220 Storage unit 221 Question sentence information database 222 Dialogue information database 230 Control unit 231 Request reception unit 232 Generation unit 233 Estimation unit 234 Output control unit

Claims (9)

a first detection unit that detects occurrence of a predetermined state change in a person to be detected who is a passenger of a vehicle, based on state information indicating the state of the person to be detected;
a second detection unit that detects a situation related to the vehicle;
a generation unit that generates question information for a user of the vehicle, the user being different from the person to be detected, based on content information indicating the content of the predetermined state change, situation information indicating the situation related to the vehicle when the predetermined state change occurred, and information regarding the maturity state of the person to be detected;
an estimation unit that estimates a cause of the occurrence of the predetermined state change in the detection target person based on a response by the user to the question information;
an output control unit that controls information according to the estimation result by the estimation unit to be output to the user;
An information processing device comprising: a storage unit that stores linked data that links content information indicating the content of the predetermined state change with situation information that indicates the situation related to the vehicle when the predetermined state change occurs,
The generation unit generates the question information based on the linked data and information on the maturity status of the detection subject.
2. The information processing apparatus according to claim 1, wherein: The information processing device according to claim 1 or 2, characterized in that the generation unit generates, as the question information, a question sentence including the candidate causes as content for obtaining an answer used to estimate a cause of the occurrence of the predetermined state change. The information processing device according to claim 1 or 2 , wherein the output control unit controls so that advice information regarding the predetermined state change is output as information according to the estimation result. The information processing apparatus according to claim 4 , wherein the output control unit controls the output of the advice information so as to provide a countermeasure for preventing the predetermined state change from occurring. The information processing device according to claim 2 , further comprising a prediction unit that predicts whether the predetermined state change will occur in the passenger being processed based on the linked data. The information processing device according to claim 6, characterized in that the prediction unit predicts whether the specified state change will occur in the passenger to be processed based on a trained model that has learned the relationship between the content information and the situation information contained in the linked data, and situation information that indicates the situation regarding the vehicle in which the passenger to be processed is currently riding. An information processing method executed by an information processing device,
a first detection step of detecting occurrence of a predetermined state change in a person to be detected who is a passenger of a vehicle, based on state information indicating the state of the person to be detected;
a second detecting step of detecting a situation related to the vehicle;
a generation step of generating question information for a user of the vehicle, who is different from the person to be detected, based on content information indicating the content of the predetermined state change, situation information indicating the situation related to the vehicle when the predetermined state change occurred, and information regarding the maturity state of the person to be detected;
an estimation step of estimating a cause of the occurrence of the predetermined state change in the detection target person based on a response by the user to the question information;
an output control step of controlling so that information corresponding to the estimation result in the estimation step is output to the user;
An information processing method comprising: a first detection step of detecting occurrence of a predetermined state change in a person to be detected, based on state information indicating a state of the person to be detected who is a passenger of a vehicle;
a second detection step for detecting a situation related to the vehicle;
a generation step of generating question information for a user of the vehicle, who is different from the person to be detected, based on content information indicating the content of the predetermined state change, situation information indicating the situation related to the vehicle when the predetermined state change occurs, and information regarding the maturity state of the person to be detected;
an estimation step of estimating a cause of the occurrence of the predetermined state change in the detection target person based on a response by the user to the question information;
an output control procedure for controlling information corresponding to an estimation result obtained by the estimation procedure to be output to the user;
An information processing program for causing an information processing device to execute the above.