JP7415901B2 - Remote support system, remote support device, and remote support program - Google Patents

Description

The present disclosure relates to a remote support system, a remote support device, and a remote support program.

There is technology to manage remote control of self-driving cars.

For example, there is a technology that calculates the cost of work (tasks) related to remote control of a plurality of self-driving cars and allocates the tasks to operators (supervisors) in the order that results in the lowest cost (see Patent Document 1). This technology calculates the predicted time of task occurrence (time targeted for remote control), work time, and priority, determines the processing order based on the work time and priority, and assigns the remote control work to a supervisor. That's what I do.

Japanese Patent Application Publication No. 2019-185279

In the technique of Patent Document 1, a supervisor is assigned to a remote support task required for the movement of a vehicle, and task assignment is optimized. For these remote support tasks, the start time of the task is determined as the vehicle moves, but for tasks that do not move close to the vehicle, the task start time can be adjusted. In the case of tasks where it is possible to adjust (particularly bring forward) the work start time of the observer, it is thought that adjusting the work start time will allow passengers in the automated driving vehicle to use the vehicle more comfortably. For example, when a vehicle such as a self-driving bus carries a large number of passengers, it is expected that stress will increase in the event of trouble, so it is necessary to ensure greater comfort. In the conventional technology, improvements other than task assignment for remote support tasks and optimization for passengers are not considered, so a mechanism to ensure passenger comfort is considered necessary.

In addition, consideration must be given to how to allocate multiple tasks to multiple observers in a system designed to ensure comfort. It is also necessary to consider the case where the work time of a task changes, such as when the work time of the task the supervisor is working on extends longer than the scheduled work time. It is also necessary to consider changes in priority depending on the status of tasks. In addition, it is not always possible to predict the occurrence of tasks in advance, and there may be a change in the number of tasks that the supervisor must handle, such as when an additional task occurs after the task processing order has been determined. It is also necessary to consider

The present disclosure has been made in view of the above circumstances, and aims to provide a remote support system, a remote support device, and a remote support program that can adjust the time related to tasks in consideration of waiting time. .

A remote support system according to the present disclosure includes a control device including an adjustment unit and each monitoring device corresponding to each vehicle to be monitored, wherein the adjustment unit is configured to monitor the occurrence of an abnormal event. Regarding the remote support task that is performed for each vehicle as a trigger and is handled by a supervisor, when a plurality of pieces of abnormality occurrence information indicating the occurrence status of an abnormal event are obtained and each of the plurality of tasks is processed, Adjusting the duration of the predetermined guidance and the start time of each of the tasks based on a cost calculated for a combination of task processing orders that includes moving the start time of the task earlier than the scheduled start time. However, each of the monitoring devices updates the predetermined guidance duration time and the task start time according to the adjustment result by the adjustment unit.

The remote support device according to the present disclosure is capable of obtaining a plurality of pieces of abnormality occurrence information indicating the occurrence status of an abnormal event, with respect to a remote support task handled by a supervisor, which is performed for each vehicle in response to the occurrence of an abnormal event. When processing each of the tasks, a predetermined guidance is provided based on a cost calculated for a combination of task processing orders that includes moving the start time of the task earlier than the scheduled start time for the task. and an adjustment unit that adjusts a duration and a start time of each of the tasks.

The remote support program according to the present disclosure obtains a plurality of pieces of abnormality occurrence information indicating the occurrence status of the abnormal event, with respect to a remote support task handled by a supervisor, which is performed for each vehicle in response to the occurrence of an abnormal event. When processing each of the tasks, a predetermined guidance is provided based on a cost calculated for a combination of task processing orders that includes moving the start time of the task earlier than the scheduled start time for the task. A computer is caused to perform a process of adjusting the duration and start time of each of the tasks.

According to the remote support system, remote support device, and remote support program of the present disclosure, it is possible to adjust the time related to a task in consideration of waiting time.

FIG. 3 is a diagram illustrating an example of a case where the remote support system detects that an abnormal event has occurred in a plurality of vehicles. FIG. 3 is a diagram illustrating an example of the flow of processing in the remote support system when an abnormal event occurs in a specific vehicle. FIG. 3 is a diagram illustrating an example of remote support when tasks are assigned using a conventional method. FIG. 3 is a diagram illustrating an example of remote support when tasks are assigned using the method of the present embodiment. FIG. 1 is a block diagram showing the configuration of a remote support system according to each embodiment of the present disclosure. FIG. 2 is a block diagram showing the hardware configuration of a control device and a monitoring device. FIG. 3 is a diagram showing a list of types of abnormal events, alerts, and call notices. FIG. 3 is a diagram illustrating an example of cost calculation for each combination of multiple tasks. 3 is a flowchart showing a supervisory control processing routine of a supervisory control unit. 3 is a flowchart showing an update processing routine of a supervisory control unit. 3 is a flowchart showing a guidance processing routine of a guidance section. 3 is a flowchart showing an adjustment processing routine of an adjustment section. It is a figure showing an example of a display of vehicle control information. FIG. 7 is a diagram illustrating an example of a case where tasks are concentrated on the supervisor 1 due to task assignment in the second embodiment. FIG. 7 is a diagram illustrating an example of a case where tasks are distributed to a monitor 1 and a monitor 2 in task assignment in the second embodiment. It is a figure showing an example of cost calculation for each combination of a plurality of supervisors and a plurality of tasks. FIG. 7 is a diagram showing an example of a case where recalculation is not performed when remote support is extended in the third embodiment. FIG. 9 is a diagram showing an example of recalculation when remote support is extended in the third embodiment. 12 is a flowchart showing a recalculation processing routine of an adjustment unit in the third embodiment.

Hereinafter, each embodiment of the present disclosure (hereinafter referred to as the present embodiment) will be described using the drawings.

First, an overview of the remote support system of this embodiment will be explained. A remote support system is a system that remotely monitors the interior of a vehicle during normal times and provides remote support when an abnormal event occurs. Remote monitoring refers to a supervisor monitoring the situation inside the vehicle. Additionally, depending on the type and severity of abnormal events that occur inside the vehicle, some abnormal events may require the intervention of a supervisor, so remote support will be provided if such action is required. Remote support refers to assisting the supervisor in resolving abnormal events occurring in the vehicle by calling out to passengers inside the vehicle or having a dialogue with the passengers. It goes without saying that remote support refers to measures taken to resolve abnormal events that have occurred, and is not limited to the measures described here. The monitor performs remote monitoring or remote support from a remote location via the control device. Further, the tasks described below refer to remote support work assigned to the monitor. It is assumed that the required work time for each task is determined in advance depending on the type of abnormal event, for example.

It is assumed that a remote support system will be in charge of multiple vehicles, the number of which is greater than the number of observers. This is because as the automation of vehicles, vehicle interior equipment, etc. progresses, the frequency of responding to abnormalities inside the vehicle will decrease, so it is expected that one supervisor will be in charge of multiple vehicles. In a remote support system, when a control device receives support requests requesting support from more vehicles than the number of observers at the same time, vehicles waiting for response occur. Here, the case where support requests are received at the same time includes a case where a support request for another task occurs while a specific task is being handled. Assuming such a case, the remote support system manages the task schedule while predicting the time when the support request will occur. Vehicles may be operated manually by a driver or automatically operated; however, unless otherwise specified, examples of automatically operated vehicles will be described below. . Note that automatic driving refers to a driving automation level equivalent to level 3 or higher in Non-patent Document 1, and manual driving refers to a driving automation level equivalent to level 2 or lower in Non-patent Document 1. Hereinafter, the vehicle assumed in this embodiment is a self-driving bus, but the present invention is applicable to any vehicle managed by a remote support system.

[Non-patent document 1]
Advanced Information and Communication Network Society Promotion Strategy Headquarters, Public-Private Data Utilization Promotion Strategy Council, “Public-Private ITS Concept Roadmap 2020”, Cabinet Office, July 15, 2020, p. 23

As described above, for a single person monitoring multiple vehicles, multiple vehicles may accidentally require remote support at the same time. As remote support requests concentrate on a specific supervisor, the load increases, the time it takes for the supervisor to respond to an abnormal event becomes longer, and the passengers inside the vehicle may continue to stay in the vehicle where the abnormal event continues. Therefore, there was a problem of inducing a feeling of anxiety. In particular, in the case of an abnormal event that involves physical injury to passengers, a delay in responding can put lives at risk. Furthermore, in the case of an abnormal event that impairs safety inside the vehicle, it may lead to an accident inside the vehicle. Therefore, when remote support requests are received from multiple vehicles, it is desirable to respond in an appropriate order and timing depending on the type and degree of impact of the abnormal event, and if possible, move the start time of the task forward.

In view of the above, in the remote support system of this embodiment, when an abnormal event occurs, assignment of tasks to monitors is adjusted in advance based on the priority of the task. Furthermore, in accordance with the adjustment results, the start time of the task is updated, and the duration of the in-vehicle guidance performed before remote support is updated. With the above, it is possible to smooth the load on the lifeguard and improve the sense of security of the passengers.

FIG. 1 is a diagram showing an example of a case where a remote support system detects that an abnormal event has occurred in a plurality of vehicles. In the example shown in Figure 1, each of a plurality of vehicles has an on-board monitoring device, and when the on-board monitoring device detects the occurrence of an abnormal event and it becomes difficult to take measures inside the vehicle, the abnormal event information is sent to the vehicle. Send to control equipment. When the control device receives abnormality information for multiple vehicles, it creates a task for each vehicle and assigns the task to a supervisor. The control device may notify the monitor to whom the task is assigned based on the scheduled call time. The notification based on the scheduled call time is information indicating the remaining time until the monitor for each task is called, such as "Call in 60 seconds" or "Call in 20 seconds", for example.

FIG. 2 is a diagram showing an example of the flow of processing in the remote support system when an abnormal event occurs in a specific vehicle. As shown in FIG. 2, (1) When an abnormal event that leads to an accident inside the vehicle occurs due to movement inside the vehicle, the occurrence of the abnormal event is detected and the control device is notified. (2) As an initial response when an abnormal event is detected, announcements will be made to alert passengers. The alert is an announcement made for a predetermined duration as described later. In addition, the alert level is set according to the elapsed time. For example, the caution level may be set to "weak" or "strong", and the announcement urging passengers to be careful is changed depending on the caution level. Note that there is no need to provide stages of attention levels. (3) If the abnormal event is not resolved even after alerting for a predetermined period of time, a supervisor is called (request for assistance). (4) Provide remote support by monitors. The control device uses the support request as a trigger to start remote support by the monitor. The supervisor checks vehicle information and camera images, calls passengers via videophone or voice telephone, and sends control commands to the vehicle's on-board monitoring device to resolve the abnormal event. It goes without saying that remote support refers to measures taken to resolve abnormal events that have occurred, and is not limited to the measures described here.

FIG. 3 shows an example of remote support when tasks are assigned using a conventional method. The example shown in FIG. 3 shows a case where a plurality of pieces of abnormality occurrence information are obtained and each of a plurality of tasks is processed. In the task assignment shown in Figure 3, while the supervisor is providing remote support for vehicle A, the warning guidance for vehicles B and C ends within a predetermined time, resulting in waiting time. Put it away. The waiting time is the time from when an announcement is made to call a supervisor through a call notice until when the supervisor starts providing remote support. If the waiting time from the time the lifeguard calls and announces the lifeguard until they begin responding to an abnormal event becomes long, a situation may arise in which passengers feel uneasy.

Generally speaking, as a method to resolve an abnormal event, after a predetermined number of seconds has elapsed to call attention to the set value, a request for assistance is automatically made to the air traffic control device, and remote assistance is provided. Call the lifeguard. However, from the perspective of both reducing the risk of accidents and reducing the burden on the supervisor, it is desirable for the supervisor to respond promptly after detecting an abnormal event if the supervisor is not available.

Therefore, the remote support system of this embodiment is characterized in that the transmission time of the support request is predicted based on notifications from a plurality of vehicles of the occurrence of an abnormal event inside the vehicle, and the start time of the task is adjusted. When adjusting the start time of a task, the time from the detection of an abnormal event to the determination of the necessity of remote support is adjusted to shorten the time required for connection, depending on the response status of the remote monitor. Additionally, in order to reflect the adjusted task start time in actual operations, it is also necessary to adjust the time at which support requests are sent from the on-vehicle monitoring device to the control device. Therefore, the in-vehicle monitoring device adjusts the time for transmitting the support request based on the start time (scheduled call time) of the adjustment result. This prevents duplication of calls from multiple vehicles at the same time. It will also create a system that smoothes the load on lifeguards and ensures passenger comfort.

FIG. 4 shows an example of remote support when tasks are assigned using the method of this embodiment. In the assignment of tasks as shown in FIG. 4, tasks are brought forward according to the work status of the lifeguard so as not to make passengers feel uneasy. In addition, due to the advance schedule, the number of warnings will be shortened, and the number of warnings will be extended to prevent waiting times from occurring. In the example shown in FIG. 4, for vehicle A, the call notice time is brought forward and the warning time is shortened. As for vehicle B, the waiting time has been eliminated due to vehicle A being brought forward. Regarding Vehicle C, even if Vehicle A is brought forward and the waiting time for Vehicle B is eliminated, there may still be a waiting time, so measures are being taken to alleviate passengers' anxiety by extending the warning time. In this way, for vehicles whose start time is earlier than the scheduled start time, the task is moved forward, and for vehicles whose start time is later than the scheduled start time, the task is moved back.

The above is an overview of the method of this embodiment. The configuration and operation of this embodiment will be explained below.

(First embodiment)
[Remote support system configuration]
FIG. 5 is a block diagram showing the configuration of the remote support system 100 according to the first embodiment of the present disclosure. As shown in FIG. 5, in the remote support system 100, a control device 110, a monitoring device 120 mounted on a plurality of vehicles, and a monitoring terminal 115 operated by a monitoring personnel are connected via a network N. . The monitoring terminal 115 only needs to be connected to the control device 110. The remote support system 100 provides remote support for a vehicle equipped with a monitoring device 120. The control device 110 is an example of a remote support device according to the technology of the present disclosure.

FIG. 6 is a block diagram showing the hardware configuration of the control device 110 and the monitoring device 120. As shown in FIG. 6, the control device 110 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a storage 14, an input unit 15, and a display interface (I/F). 16 and a communication interface (I/F) 17. Each configuration is communicably connected to each other via a bus 19. Further, the monitoring device 120 can also be configured in a similar manner. The monitoring device 120 includes a CPU 21 , a ROM 22 , a RAM 23 , a storage 24 , an input section 25 , a display interface 26 , and a communication interface 27 . Each configuration is communicably connected to each other via a bus 29. Each part of the control device 110 will be described below as an example.

The CPU 11 is a central processing unit that executes various programs including a driving support program and controls various parts. That is, the CPU 11 reads a program from the ROM 12 or the storage 14 and executes the program using the RAM 13 as a work area. The CPU 11 controls each of the above components and performs various arithmetic operations according to programs stored in the ROM 12 or the storage 14. In this embodiment, the ROM 12 or the storage 14 stores a control program (remote support program).

The ROM 12 stores various programs and data. The RAM 13 temporarily stores programs or data as a work area. The storage 14 is constituted by a storage device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive), and stores various programs including an operating system and various data.

The input unit 15 includes a pointing device such as a mouse and a keyboard, and is used to perform various inputs.

The display interface 16 is, for example, a liquid crystal display, and displays various information. The display interface 16 may employ a touch panel method and function as the input section 15.

The communication interface 17 is an interface for communicating with other devices such as terminals, and supports, for example, standards such as Ethernet (registered trademark), FDDI, Wi-Fi (registered trademark), Bluetooth (registered trademark), and cellular communication systems. is used. The above is a description of an example of the hardware configuration of the control device 110 and the monitoring device 120.

The functional configuration will be explained below.

The control device 110 includes an information storage section 111, a communication section 112, an adjustment section 113, and a display section 114. The monitoring device 120 includes a sensor 121 , a communication section 122 , a recognition section 123 , a monitoring control section 124 , and a guide section 125 .

First, the functional configuration of each part of the control device 110 will be explained. The information storage unit 111 stores vehicle control information, which is various information for providing remote support. The vehicle control information includes vehicle information, in-vehicle information, abnormality occurrence information, supervisor information, task information, task schedule, etc. for each vehicle. Vehicle information, in-vehicle information, and abnormality occurrence information are updated by receiving them from the monitoring device 120. Vehicle information includes, for example, vehicle ID, driving state, speed, and driving position. The in-vehicle information includes, for example, the number of passengers, the abnormal event that has occurred, and the contents of the guidance currently being provided. The abnormality occurrence information includes the vehicle ID, the abnormality occurrence time, and the type of abnormal event. It is assumed that the information storage unit 111 stores in advance a table in which times such as the scheduled call time and the duration of the alert are recorded according to the type of abnormal event. Note that if the scheduled calling time is individually managed on the monitoring device 120 side, the scheduled calling time may be included in the abnormality occurrence information. The supervisor information, task information, and task schedule are updated by the adjustment unit 113. The supervisor information includes the vehicle ID of the supervisor and the work status of the supervisor. The work status includes information indicating whether the supervisor is working on a task, the progress of the work, and the scheduled start time of the task. The task information includes, for example, the corresponding start time of the task, the elapsed time, the scheduled end time, the remaining time until the scheduled end time of the task, the contents of the call, and the priority. The task schedule is the result of adjusting task assignments by the adjustment unit 113, and is information indicating how tasks are assigned to monitors for each unit of time. The cost for each combination of tasks calculated by the adjustment unit 113 is also recorded in the task schedule.

The communication unit 112 periodically receives vehicle information and in-vehicle information from the monitoring device 120. Further, when an abnormality occurs in the vehicle, abnormality occurrence information is received from the monitoring device 120. The communication unit 112 also transmits the task adjustment result by the adjustment unit 113 to the monitoring device 120.

When a plurality of pieces of abnormality occurrence information indicating the occurrence status of an abnormal event are obtained, the adjustment unit 113 calculates a cost for each combination of task processing orders, and adjusts task assignment based on the cost. Combinations of task processing orders include combinations in which the start time of a task is moved earlier than the scheduled start time (reference time) of the task. The scheduled call time set as a default value is used as the reference time. Depending on the cost, some vehicle tasks have start times earlier than the reference time, while other vehicle tasks have start times later than the reference time. The adjustment unit 113 adjusts the start time of a task to be moved forward or backward relative to the reference time. By adjusting the assignment of tasks, the start time (scheduled call time) of each task is adjusted.

In addition, we will organize the expressions "scheduled call time" and "task start time" that correspond to the point in time when remote support starts. A support request is transmitted from the monitoring device 120 according to the "scheduled call time." Then, the time when the control device 110 that received the support request starts remote support by the monitor becomes the "task start time." In other words, in chronological order, the "scheduled call time" is followed by the "task start time", and by adjusting the "scheduled call time", the "task start time" is also adjusted.

The cost calculation method will be explained below. The adjustment unit 113 transmits the adjustment result to the monitoring device 120 of the vehicle to be adjusted. The adjustment result includes the adjusted scheduled call time. Further, the adjustment result may include an instruction to shorten or extend the warning period.

The cost function of equation (1) below is used to calculate the cost. The adjustment unit 113 calculates the task cost for each task i using a cost function for each task processing order, and calculates the sum of the task costs as the cost of the entire assignment of tasks in the task processing order.

...(1)

Here, time h _i is the time required to start work on task i. The time h _i until work starts for task i is an example of the task start time according to the technology of the present disclosure. For task i, the task cost is calculated using the time h _i and the priority according to the task type. The priority may be determined according to the task type and in-vehicle information.

A negative value is added if the response is brought forward, a 0 value is added if the call is on time, and a positive value is added if the response is delayed. For example, when responding to task i by moving forward the scheduled call time for a supervisor's work request by X seconds, a value equivalent to X seconds is subtracted from time h _i . If it is in seconds, it is calculated as "time h _i = h _i -X". When corresponding to the scheduled call time for a work request from a supervisor from a vehicle, the time h _i of the task is set to 0. When responding to a supervisor's work request with a delay of Y seconds from the scheduled call time, a value equivalent to Y seconds is added from time h _i to calculate “time h _i =h _i +Y”. In Patent Document 1, addition of only the delay time was considered, but in this embodiment, subtraction in the case of bringing forward the schedule is also considered, so that the merits of bringing the schedule forward are reflected in the cost calculation. In this way, in order to reduce the cost value only when the task start time is brought forward, if an abnormal event occurs and the supervisor is not working, the supervisor is actively involved in the event inside the vehicle. This makes it possible to prevent accidents inside the car. In this way, the scheduled calling time (task start time) is adjusted earlier or later depending on the work status of the supervisor, and the cost is calculated. As described above, the adjustment unit 113 adjusts the assignment of tasks by determining the task processing order that has the minimum cost value among the costs of each task processing order calculated using the cost function. Adjustment unit 113 transmits the adjustment result obtained in this way to monitoring device 120.

The display unit 114 displays predetermined vehicle control information on the monitoring terminal 115 operated by the monitoring staff in response to the monitoring staff's operation. The displayed vehicle control information includes the task start time adjusted by the adjustment unit 113. A specific example of the display will be described later.

Next, the functional configuration of each part of the monitoring device 120 will be explained. As the sensor 121, for example, an in-vehicle camera capable of capturing images inside the vehicle, and an in-vehicle sensor for detecting vehicle information can be used. A camera image (or camera video) captured by an on-vehicle camera is output to the recognition unit 123 as sensor information. In addition to the in-vehicle camera, any sensor that can detect the situation inside the vehicle may be used, such as a distance sensor, millimeter wave sensor, infrared sensor, or ultrasonic sensor. . On-vehicle sensors detect vehicle information indicating driving behavior. Driving behavior includes the vehicle speed, acceleration, steering, accelerator, brake pedal, etc., as well as the state indicating whether or not automatic driving is in progress.

The communication unit 122 periodically transmits in-vehicle information, which is a recognition result by the recognition unit 123, and vehicle information, which is determined by the sensor 121, to the control device 110. Furthermore, when the recognition unit 123 detects the occurrence of an abnormal event inside the vehicle, the communication unit 122 transmits abnormality occurrence information to the control device 110. The communication unit 122 also receives the task adjustment results from the control device 110.

The recognition unit 123 detects the occurrence of an abnormal event inside the vehicle based on sensor information. Abnormal events may be detected using a recognition model that has learned sensor information patterns for each abnormal event in advance through machine learning or the like.

For the type of abnormal event detected by the recognition unit 123, an announcement to be made to passengers inside the vehicle is determined. FIG. 7 is a diagram showing a list of types of abnormal events, alerts, and call notices. Note that although the example in FIG. 7 shows only one example of an announcement to call for attention, even if the caution level is to be set in stages, one announcement can be used to raise the volume or raise the volume as the caution level changes. , a warning sound may be used. As shown in FIG. 7, the types of abnormal events include, for example, "rise-up occurrence", "illegal act", "nuisance act", and "sitting-in occurrence". Announcements are prescribed for alerts and call notices. The alert for a "stand-up occurrence" can be "A stand-up has been detected. It is dangerous to drive, so please take a seat.", and the call notice can be "Please take a seat. If you do not take a seat, we will connect you to the operator." . The alert for "illegal activity" can be "An illegal activity has been detected. Please stop.", and the call notice can be "Please stop the illegal activity. If you do not stop, we will contact the operator." The alert for "nuisance behavior" can be "A nuisance behavior has been detected. Please stop.", and the call notice can be "Please stop the nuisance behavior. If you do not stop, we will contact the operator." The alert for "nuisance behavior" is "A sit-in has been detected. If you are not feeling well, please use the call button.", and the alert alert is "A sit-in has been detected. You will be connected to an operator for safety confirmation." ”. Depending on the abnormal event, it is also possible to prioritize tasks (eg, set weights depending on the abnormal event) or change costs over time. Further, a label may be defined for the type of abnormal event. For example, it is possible to distinguish between abnormal events by labeling "standing up," "illegal act," or "nuisance" as a "safety-obstructing event," and "sitting down" as a "physical abnormality event." You can. In addition, in the case of "standing up", the warning level is different depending on the warning level: "Weak" alert means "Please sit down while driving" and "Strong" warning means "Standing up while driving is dangerous. Please sit down." An announcement may also be made.

The monitoring control unit 124 updates the setting value of the remote support start time (scheduled call time) when receiving the adjustment result from the control device 110. Furthermore, the monitoring control unit 124 shortens or extends the warning time depending on the adjustment result. If the start time is brought forward as a result of the adjustment, the time required to call attention will be shortened. If the start time is pushed back as a result of the adjustment, the warning time will be extended. Further, the monitoring control unit 124 transmits a request for remote support to the control device 110 when the scheduled call time has arrived based on the set value.

The guide unit 125 makes an announcement to alert the user when the recognition unit 123 detects an abnormal event. Further, the guide unit 125 executes a call advance notice when the scheduled call time comes. The alert and call notice announcement have the contents shown in FIG.

Here, it is assumed that default values are set for the duration of the alert and the scheduled call time. The alert starts after the abnormal event is detected, and the guidance starts with the alert being "weak." As an example, the duration of the alert "weak" is set as 60 seconds, and the announcement of the alert "weak" is repeated for 60 seconds. Furthermore, after the "weak" warning ends, the "strong" warning begins. As an example, assume that the duration of the "strong" alert is set to 60 seconds. In this case, the scheduled call time is set to 120 seconds later. Using the set value of the scheduled call time as a trigger, a support request from the supervisory control unit 124 and a call notice from the guide unit 125 are simultaneously executed. The monitoring control unit 124 updates the set value by changing this default value.

Here, a specific example of remote support processing by the control device 110 and the monitoring device 120 will be described.

FIG. 8 is a diagram illustrating an example of cost calculation for each combination of multiple tasks. FIG. 8 shows a comparison example of cost calculation using the conventional method and the present method when there are tasks α, β, and γ. The time-series line table for each task in the upper row represents a time-series change in priority every 10 seconds from the reference time, such as 10 seconds, 20 seconds, 30 seconds, and so on. In the time-series change in priority, each task is assigned a priority of 1 to 3. Note that, as an example of cost calculation using this method, FIG. 8 shows a case where the start time of the task that corresponds first among task α, task β, and task γ is brought forward by 10 seconds. Not as long. An example of calculating the cost for each combination of tasks will be described below. Note that the priority is expressed as P.

A case of a conventional combination of tasks αβγ in the first row will be described. Task α is at the standard time (scheduled call time set as the default value), so h _α = 0, task β is delayed by 10 seconds, so h _β = 1, and task γ is delayed by 40 seconds, so h It is expressed as _γ =4. Regarding the time series priorities, task α is expressed as P _α =1, task β is expressed as P _β =1, and task γ is expressed as P _γ =3. Therefore, according to the cost function of equation (1) above, the cost of the combination of tasks αβγ is calculated as “0×1+1×1+4×3=13”.

Next, cost calculation in the case of this method will be explained. The case of the combination of tasks αγβ in the second line of this method will be explained. In addition, in the case where the task is moved forward, only the first task is subtracted, and the second and subsequent tasks are calculated to be on time. Note that depending on the design of the cost function, the second and subsequent values may also be subtracted, and the subtraction target may be determined as appropriate depending on the design. Task α is brought forward by 10 seconds from the standard time, so h _α = (0-1), task γ is the second task brought forward from the standard time, so h _γ = 0, and task β is delayed by 40 seconds. Therefore, it is expressed as h _γ =4. Regarding the time series priorities, task α is expressed as P _α =1, task γ is expressed as P _γ =3, and task β is expressed as P _β =2. Therefore, according to the cost function of equation (1) above, the cost of the combination of tasks αβγ is calculated as “(0−1)×1+0×3+4×2=7”. Note that the calculation is just an example, and for example, the value before moving forward and the value equivalent to moving forward may be calculated separately. For the above task α, the calculation is “(0×1)+(−1×1)=(0−1)”.

[Operation of remote support system]
Next, regarding the operation of the remote support system 100 according to the first embodiment of the present disclosure, the respective processes of the control device 110 and the monitoring device 120 will be described using a flowchart.

Processing related to the monitoring device 120 is executed by the CPU 21 functioning as each part of the monitoring device 120. Note that the recognition processing by the recognition unit 123 and the communication processing by the communication unit 122 are assumed to be executed each time data is transmitted and received, and a description thereof will be omitted.

FIG. 9 is a flowchart showing the supervisory control processing routine of the supervisory control unit 124. The supervisory control processing routine is repeated as a periodic operation.

In step S100, the CPU 21 determines whether or not the recognition result has been received from the recognition unit 123. If it is determined that the information has been received, the process moves to step S102, and if it is determined that the information has not been received, this step is repeated.

In step S102, the CPU 21 identifies whether or not there is an abnormality in the recognition result.

In step S104, the CPU 21 determines whether an abnormal event has occurred based on the identification result. If it is determined that an abnormal event has occurred, the process moves to step S112, and if it is determined that no abnormal event has occurred, the process moves to step S106.

In step S106, the CPU 21 determines whether the abnormal event that was occurring has been resolved. If the problem has been resolved, the process moves to step S108; if the problem has not been solved, the process returns to step S100 and repeats the process.

In step S108, the CPU 21 transmits an abnormality resolution notification to the control device 110. Note that the abnormality resolution notification is information for notifying that the abnormal event has been resolved. Upon receiving the abnormality resolution notification, the notification control device 110 treats the task associated with the abnormal event as completed in task assignment.

In step S110, the CPU 21 instructs the guide unit 125 to make a normality confirmation announcement.

In step S112, the CPU 21 determines whether the scheduled calling time according to the set value has elapsed. If it is determined that the scheduled calling time has not elapsed, the process moves to step S114, and if it is determined that the scheduled calling time has passed, the process moves to step S118.

In step S114, the CPU 21 transmits abnormality occurrence information to the control device 110 via the communication unit 122.

In step S116, the CPU 21 instructs the guide unit 125 to make an announcement to alert the user. When the guide unit 125 receives an instruction to make a warning announcement, the guide unit 125 repeats the caution announcement at regular intervals. If the duration of the warning "weak" is set to 60 seconds, the announcement of the warning "weak" is repeated at regular intervals within the 60 seconds. After 60 seconds of warning "weak" has elapsed, the warning shifts to "strong". If the duration of the "strong" alert is set to 60 seconds, the announcement of the "strong" alert is repeated at regular intervals within the 60 seconds. Naturally, the intervals between announcements may not be constant. For example, more effective announcements can be made by making the announcement interval for the latter 30 seconds shorter than the announcement interval for the first half of the 60 seconds of "weak" alerting.

In step S118, the CPU 21 transmits a request for remote support to the control device 110.

In step S120, the CPU 21 instructs the guide unit 125 to announce a call notice. When the guide unit 125 receives an instruction to announce a call notice, the guide unit 125 repeats the alert announcement at regular intervals. The above is an explanation of the supervisory control processing routine.

Next, the setting value update processing routine will be explained. FIG. 10 is a flowchart showing the update processing routine of the monitoring control unit 124.

In step S140, the CPU 21 determines whether or not the adjustment result has been received from the control device 110. If the adjustment result has been received, the process moves to step S142, and if the adjustment result has not been received, this step is repeated.

In step S142, the CPU 21 updates the set value of the scheduled call time according to the adjustment result.

In step S144, the CPU 21 updates the set value of the alert duration time to be shortened or extended in accordance with the updated scheduled call time. Note that the shortening or extension may be executed as an arbitrary process, and does not need to be updated according to the settings of the monitoring device 120.

In step S146, the CPU 21 instructs the guide unit 125 to call attention to the updated duration.

FIG. 11 is a flowchart showing the guidance processing routine of the guidance section 125.

In step S160, the CPU 21 determines whether an instruction has been received. If the instruction has been received, the process moves to step S162, and if the instruction has not been received, this step is repeated.

In step S162, the CPU 21 makes an announcement based on the received instruction. The instructions are as explained above.

The above is an explanation of the processing of the monitoring device 120. Next, the processing of the control device 110 will be explained. Processing related to the control device 110 is executed by the CPU 11 functioning as each part of the control device 110. Note that the communication processing of the communication unit 112 will be omitted since it is executed every time data is transmitted and received. Further, the data received by the communication unit 112 may be stored in the information storage unit 111.

FIG. 12 is a flowchart showing the adjustment processing routine of the adjustment unit 113.

In step S200, the CPU 11 acquires abnormality occurrence information and supervisor information from the information storage unit 111.

In step S202, the CPU 11 obtains the scheduled call time for each task corresponding to the abnormality occurrence time and the type of abnormal event included in the abnormality occurrence information. The scheduled call time obtained here is the scheduled call time (reference time) set as a default value.

In step S204, the CPU 11 determines whether the work times of the tasks overlap based on the reference time. If it is determined that there is overlap, the process moves to step S206, and if it is determined that there is no overlap, the process returns to step S200 and repeats the process.

In step S206, the CPU 11 determines whether the scheduled call time can be adjusted based on the monitor information. If it is determined that adjustment is possible, the process moves to step S208, and if it is determined that adjustment is not possible, the process returns to step S200 and repeats the process. In addition, adjustment is possible if there is no task in progress in the work situation of the lifeguard, or if no other task is assigned before the lifeguard's scheduled task. And it is sufficient.

In step S208, the CPU 11 calculates the cost according to the cost function of the above equation (1) so as to make adjustments depending on the time before and after the reference time. In advance, the task cost is calculated by adding a negative value, and the sum of the task costs is calculated as the overall cost.

In step S210, the CPU 11 adjusts the task schedule based on the cost calculated in step S208. The adjustment result includes the adjusted scheduled call time for each task.

In step S212, the CPU 11 transmits the adjustment result via the communication unit 112 to the monitoring device 120 corresponding to the automatic driving vehicle whose scheduled call time has been brought forward or delayed. The above is a description of the processing of the control device 110.

Remote support is performed by the above processing of the remote support system 100. Here, an example of a screen of vehicle control information displayed on the monitoring terminal 115 when a monitoring person performs remote monitoring will be described. FIG. 13 is a diagram showing an example of a display of vehicle control information. In the example screen shown in FIG. 13, a list of vehicle control information regarding the tasks that the observer is in charge of is displayed. The displayed vehicle control information is task information, in-vehicle information as operation information, vehicle information, and task schedule. Naturally, the vehicle control information items shown here are just examples, and it goes without saying that one or more of them may be selected and displayed. The task information is the task information that the monitor is currently working on. The in-vehicle information and the vehicle information are vehicle operation information corresponding to the task in progress. The task schedule is a schedule showing tasks (displayed as blocks indicating vehicle IDs and priorities) scheduled for the observer in chronological order. The task schedule also includes the scheduled progress of the task. Note that the monitoring terminal 115 may be notified from the control device 110 based on the scheduled call time regarding the task assigned to the monitor who has the monitoring terminal 115. The timing of the notification may be determined as appropriate, such as when the task is assigned or at regular intervals.

As explained above, according to the remote support system according to the first embodiment of the present disclosure, it is possible to adjust the time related to a task in consideration of waiting time.

(Second embodiment)
The second embodiment assumes a case where tasks are distributed to other monitors when tasks are concentrated on a specific monitor. Note that the configuration and operation of the second embodiment are similar to those of the first embodiment, so only the different parts will be described.

FIG. 14 is a diagram showing an example of a case where tasks are concentrated on the supervisor 1 due to task assignment. In the example shown in FIG. 14, tasks occur during overlapping time periods for vehicles A, B, and C, which are in charge of supervisor 1. Therefore, the tasks are assigned such that vehicle B and vehicle C have a waiting time. On the other hand, since there is no task for the vehicle that the supervisor 2 is in charge of, the supervisor 2 is in a task ready state.

Therefore, in the second embodiment, the task is assigned to two or more monitors, the cost is calculated, and the task is assigned. Here, a case will be described in which a task for a vehicle other than the one in charge of the supervisor is added to the task assignment target. It is assumed that the costs for newly assigned tasks to the monitor will be calculated taking into consideration the preparation time required for response preparation. It goes without saying that the present invention is effective even when cost calculations are performed without considering the preparation time for response preparation. The reason for assuming the preparation time for response preparation is that it is an irregular response for the monitor who has to deal with the task of a vehicle that is not the vehicle in charge. For example, if you are responsible for a number of predetermined vehicles as vehicles to be monitored, and if you suddenly want to respond to a task for a vehicle that you are not responsible for during normal times, you can normally do this before the start of monitoring work. Confirmation work such as understanding the driving area and route of the vehicle will be carried out in a hurry, and it is expected that it will take more time to prepare for the response compared to the task response of the vehicle in charge. Therefore, the preparation time for response preparation assumes the time required to respond to the task of a vehicle other than the above-mentioned vehicle in charge. Naturally, the preparation time is set as appropriate depending on the operation mode, and is not necessarily required.

FIG. 15 is a diagram showing an example of a case where tasks are distributed to the supervisor 1 and the supervisor 2 in task assignment. In the example shown in FIG. 15, the task of the vehicle A of the observer 1 is brought forward, and the warning time is shortened. The tasks for vehicle B are assigned in consideration of the preparation time of observer 2, and the alert is extended during the preparation time. Also, the task of the vehicle C of the observer 1 is brought forward. As described above, task assignments are adjusted to avoid waiting time.

When a supervisor is added, the adjustment unit 113 calculates the cost according to the cost function of equation (2) below, including the preparation time of the added supervisor in the combination of task processing orders.

...(2)

When the total number of tasks is N _t and the total number of monitors is N _op , there may be monitors to whom no tasks are assigned, and the number of cases in which tasks are assigned to each monitor is also considered. , N _t +(N _op −1) permutations. At this time, the number in the case of allocating all tasks N _t and all monitors N _op is {N _t + (N _op −1)}! There is a street. Here, the number of tasks assigned to the n-th observer is expressed as (N _tn ) below. N _tn can be expressed by a pattern of assignment of task t to supervisor n, such as assigning tasks 1 and 2 to supervisor 1 and tasks 1, 2, and 3 to supervisor 2, for example.

The adjustment unit 113 uses the above equation (2) to calculate {N _t + (N _op −1)}! The cost function is calculated and the task allocation is adjusted so that the task allocation pattern has the minimum cost function.

FIG. 16 is a diagram illustrating an example of cost calculation for each combination of multiple monitors and multiple tasks. For convenience of explanation, the order of combinations is simplified. In contrast to the conventional method, this method moves forward and calculates costs by taking into account the value equivalent to the front-loading. In the conventional method, the αβγ task order has the minimum cost value. On the other hand, in this method, the αβγ or αγβ task order has the minimum cost value, and it is possible to recommend working in either task order. In the case of the example shown in Figure 16, it is possible to prioritize task γ, which has a higher priority than task β. can be realized.

(Third embodiment)
In the third embodiment, recalculation is considered when the progress of the remote support work of the supervisor is delayed and it is predicted that the task will be extended. Note that the configuration of the third embodiment is similar to that of the first embodiment, and since the operation for recalculation is different, only the different parts will be described below.

FIG. 17 is a diagram showing an example of a case where recalculation is not performed when remote support is extended. As shown in FIG. 17, as the remote support for vehicle A is extended, waiting time that was not originally scheduled to occur will occur for the following vehicles B and C.

FIG. 18 is a diagram showing an example of recalculation when remote support is extended. As shown in FIG. 18, after the recalculation, the task of vehicle B is reassigned to extend the alert. Vehicle C is reassigned to observer 2. In addition, even if there is only one observer, the period will be extended by reassignment based on recalculation. As described above, even if the remote support work time is extended, the waiting time is suppressed by recalculation.

FIG. 19 is a flowchart showing the recalculation processing routine of the adjustment unit 113.

In step S300, the CPU 11 acquires the work status of the supervisor and the task schedule from the information storage unit 111.

In step S302, the CPU 11 determines whether the work of the supervisor is progressing as planned by comparing the work status of the supervisor with the task schedule. If it is determined that the progress is progressing as planned, the process returns to step S300 and the process is repeated. If it is determined that the progress is not progressing as planned, the process moves to step S304. Progress is determined by comparing the work progress in the work status of the monitor and the planned progress of the task for which the monitor is providing remote support, and determining whether the work progress is behind the planned progress. If the progress of the work is not behind the scheduled progress, it is determined that the work is not progressing as planned, and if the progress of the work is not behind the scheduled progress, it is determined that the work is progressing as planned. If it is determined that the task is not progressing as planned, recalculation is performed assuming that a prediction that the task will be extended is obtained. Furthermore, in addition to the progress, the occurrence of a task extension may be predicted by considering the skill of the observer, the difficulty level of the task, the type of task, the situation inside the vehicle, etc.

In step S304, the CPU 11 recalculates the assignment of tasks to the monitors. Note that when there is one observer, the cost function of equation (1) above may be used, and when a plurality of observers are assumed, the cost function of equation (2) above may be used.

In step S306, the CPU 11 readjusts the task assignment based on the recalculation and creates an adjustment result. The adjustment result based on the recalculation is created for the monitoring device 120 corresponding to the vehicle whose scheduled call time has been changed (moved forward or delayed) as a result of the recalculation. Note that due to a change in the scheduled call time, the monitoring device 120 extends the alert if necessary (step S144), but an instruction to extend the alert is included in the adjustment result based on the recalculation. Good too.

In step S308, the CPU 11 transmits the adjustment result based on the recalculation to the target monitoring device 120.

The monitoring device 120 that has received the adjustment result based on the recalculation may execute a monitoring control processing routine or a guidance processing routine.

Note that the present disclosure is not limited to the embodiments described above, and various modifications and applications can be made without departing from the gist of the present invention.

Further, although the present specification has been described as an embodiment in which a program is installed in advance, it is also possible to provide the program by storing it in a computer-readable recording medium.

100 Remote support system 110 Control device 111 Information storage section 112 Communication section 113 Adjustment section 114 Display section 115 Monitoring terminal 120 Monitoring device 121 Sensor 122 Communication section 123 Recognition section 124 Monitoring control section 125 Guide section

Claims (14)

A remote support system including a control device including an adjustment unit and each monitoring device corresponding to each vehicle to be monitored,
The adjustment unit obtains a plurality of pieces of abnormality occurrence information indicating the occurrence status of an abnormal event with respect to a remote support task to be performed for each of the vehicles in response to the occurrence of an abnormal event and is handled by a supervisor, a start time of each of said tasks based on a cost calculated for a combination of task processing orders that includes moving the start time of said task earlier than the scheduled start time of said task when processing each of said tasks; Adjust the
Each of the monitoring devices updates a predetermined guidance duration time and a start time of the task according to the adjustment result by the adjustment unit.
Remote support system. The adjustment unit is configured to move the start time of the task forward with respect to the scheduled start time for a vehicle task whose start time is earlier than the scheduled start time according to the cost. Adjust,
The remote support system according to claim 1, wherein the monitoring device corresponding to a vehicle whose start time is brought forward in the adjustment result shortens the duration of the guidance and starts the task after the shortened guidance ends. The adjustment unit is configured to postpone the start time of the task with respect to the scheduled start time for a vehicle task whose start time is later than the scheduled start time according to the cost. Adjust,
The monitoring device corresponding to a vehicle whose start time is delayed in the adjustment result extends the duration of the guidance, and starts the task after the extension of the guidance ends. Remote support system. For each of the tasks in the task processing order, the adjustment unit adjusts the tasks for each of the tasks using the start time of the task, which is moved forward or backward according to the work status of the supervisor, and the priority according to the task type. 4. The remote support system according to claim 1, wherein a cost is calculated, a sum of the task costs is calculated as the cost, and the task processing order is determined. 5. The remote support system according to claim 4, wherein the task cost is calculated using a value equivalent to bringing the task forward when the task is brought forward. 6. The remote support system according to claim 4, wherein the priority value is set at each point in time in accordance with the type of the task. The adjustment unit calculates the cost for each combination of the task processing order in which each of the tasks is assigned to two or more of the monitors in chronological order. remote support system. 8. The remote support system according to claim 7, wherein when the supervisor is added, the adjustment unit calculates the cost by including preparation time for the added supervisor in the combination of the task processing order. The remote controller according to any one of claims 1 to 8, wherein the adjustment unit recalculates the cost when it is predicted that the task for which the supervisor is providing remote support will be extended. support system. Based on the progress of the remote support work of the supervisor and the scheduled progress of the task for which the supervisor is providing remote support, determine whether the work of the supervisor is progressing as planned. The remote support system according to claim 9, wherein the remote support system predicts that the task will be extended if it is determined that the work of the supervisor is not progressing as planned. The control device further includes a display section,
The remote support system according to claim 1, wherein the display unit displays vehicle control information including the adjusted start time of the task. The monitoring device further includes a monitoring control section,
The adjustment unit transmits an adjustment result including a start time of the task according to the adjustment to the monitoring device,
Any one of claims 1 to 11, wherein the supervisory control unit updates a setting value indicating the calling time of the lifeguard according to the adjustment result, and starts calling the lifeguard in accordance with the updated setting value. The remote support system according to item 1. Regarding the remote support task that is performed for each vehicle in response to the occurrence of an abnormal event, and which is handled by a supervisor, multiple pieces of abnormal occurrence information indicating the occurrence status of the abnormal event are obtained, and when processing each of the plurality of tasks, , a predetermined guidance duration, and a predetermined guidance duration for each of the tasks based on a cost calculated for a combination of task processing orders that includes moving the start time of the task earlier than the scheduled start time for the task. An adjustment section that adjusts the start time;
remote support equipment including; Regarding the remote support task that is performed for each vehicle in response to the occurrence of an abnormal event, and which is handled by a supervisor, when multiple pieces of abnormal occurrence information indicating the occurrence status of the abnormal event are obtained and each of the plurality of tasks is processed, , a predetermined guidance duration, and a predetermined guidance duration for each of the tasks based on a cost calculated for a combination of task processing orders that includes moving the start time of the task earlier than the scheduled start time for the task. adjust the start time,
A remote support program that causes a computer to perform a process.