JP7838546B2 - Information processing device - Google Patents

Description

This disclosure relates to an information processing device.

Conventionally, vehicles that operate autonomously via remote control are known (Patent Document 1).

Special table 2017-538619 publication

The inventors of this application conceived of acquiring type information indicating the body type of a moving object such as a vehicle using at least one of a machine learning model and management information. The machine learning model is a trained model that outputs type information when sensor information is input. The management information is information that indicates the travel order of multiple moving objects traveling within the detection range of a sensor that acquires sensor information representing a moving object, in association with its body type. When the acquired type information is used to determine the travel behavior of a moving object, deficiencies may occur in at least one of the machine learning model and the management information.

This disclosure can be implemented in the following forms:

(1) According to one embodiment of the present disclosure, an information processing device is provided. The information processing device used for driving a mobile body that can be moved by unmanned operation includes a storage unit, an information acquisition unit that acquires sensor information representing the mobile body detected using a sensor, a first type acquisition unit that acquires type information by inputting the sensor information to a machine learning model that outputs type information indicating the body type of the mobile body represented by the sensor information, which is a body type of the mobile body classified according to the external shape of the mobile body when the sensor information is input, and a second type acquisition unit that acquires type information by identifying the body type of the mobile body represented by the sensor information using management information that indicates the driving order of a plurality of mobile bodies that travel within the detection range of the sensor in association with the body type The information processing device includes: a unit; a first determination unit that compares first type information obtained by the first type acquisition unit as the type information and second type information obtained by the second type acquisition unit as the type information to determine whether there is a defect in at least one of the machine learning model and the management information; an execution unit that performs at least one of the following processes if the first determination unit determines that there is a defect: a first process that retrains the machine learning model; a second process that notifies the user of error information indicating that there may be a defect; a third process that stores an error code indicating that a defect has been determined in the storage unit; and a fourth process that stops the mobile object. According to this configuration, the information processing device can determine whether there is a defect in at least one of the machine learning model and the management information by comparing first type information and second type information. If the information processing device determines that there is a defect in at least one of the machine learning model and the management information, it can perform at least one of the first process, the second process, the third process, and the fourth process. This allows for addressing deficiencies if there are flaws in at least one of the machine learning model or the management information.
(2) In the above configuration, the execution unit generates a stop signal for stopping the mobile body in the fourth process, and the information processing device further includes a transmission unit for transmitting the stop signal to the mobile body. In this configuration, if the information processing device determines that there is a defect in at least one of the machine learning model and the management information, it can stop the mobile body by generating a stop signal in the fourth process and transmitting the stop signal to the mobile body.
(3) In the above configuration, the first determination unit may compare the first type information and the second type information for each of the plurality of mobile bodies, and determine that no defect has occurred if the ratio of the number of mobile bodies for which the first type information and the second type information match to the total number of the plurality of mobile bodies compared is equal to or greater than a reference value, and determine that a defect has occurred if it is less than the reference value. According to this configuration, the information processing device can determine whether or not a defect has occurred in at least one of the machine learning model and the management information by comparing the agreement rate with a reference value.
(4) In the above configuration, the reference value may be 100 percent. In this configuration, the information processing device can determine that there is a defect in at least one of the machine learning model and the management information when the reference value is less than 100 percent.
(5) According to another embodiment of the present disclosure, an information processing device is provided. The information processing device used for driving a mobile body that can be moved by unmanned operation includes: an information acquisition unit that acquires sensor information representing the mobile body detected using a sensor; a second determination unit that determines whether or not there is a defect in the management information; and a first type acquisition unit that, when the second determination unit determines that there is a defect in the management information, acquires type information by inputting the sensor information into a machine learning model, wherein the type information is a body type of the mobile body classified according to the external shape of the mobile body, and is information indicating the body type of the mobile body represented by the sensor information; the management information is information indicating the driving order of a plurality of mobile bodies driving within the detection range of the sensor in association with the body type; and the machine learning model outputs the type information when the sensor information is input. According to this embodiment, the information processing device can determine whether or not there is a defect in the management information. And, when the information processing device determines that there is a defect in the management information, it can acquire type information using a machine learning model.
This disclosure can be implemented in various forms other than the information processing apparatus described above. For example, it can be implemented in the form of an information processing method, a method for manufacturing an information processing apparatus, a method for controlling an information processing apparatus, a computer program that implements the control method, a non-temporary recording medium on which the computer program is stored, and so on.

This is a diagram showing the configuration of the information processing system in the first embodiment. This figure shows the configuration of the information processing device in the first embodiment. This is a flowchart showing the processing flow in the first embodiment. This figure shows the configuration of the information processing device in the second embodiment. This is a flowchart showing the processing flow in the second embodiment. This figure shows the configuration of the information processing device in the third embodiment. This is the first flowchart showing the processing flow in the third embodiment. This is a second flowchart showing the processing flow in the third embodiment. This figure shows the configuration of the information processing system in the fourth embodiment. This figure shows the configuration of the information processing device in the fourth embodiment. This is a flowchart showing the processing flow in the fourth embodiment.

A. First embodiment:
Figure 1 shows the configuration of the information processing system 1 in the first embodiment. The information processing system 1 identifies the body type of the moving object represented by the sensor information. The sensor information is information acquired by detecting the target moving object from the outside using a sensor. Therefore, the sensor information includes information representing the moving object. The sensor information consists of at least one of raw data acquired by the sensor and processed data obtained by processing the raw data. The body type is a classification used when classifying the moving object according to its external shape. The information processing system 1 comprises one or more moving objects, a sensor, and an information processing device 6.

"Mobile object" refers to an object capable of movement, such as a vehicle 10 or an electric vertical take-off and landing aircraft (a so-called flying car). In this embodiment, the mobile object is a vehicle 10. Vehicle 10 may be a vehicle that runs on wheels or a vehicle that runs on tracks, and examples include passenger cars, trucks, buses, motorcycles, cars, tanks, construction vehicles, etc. Vehicle 10 includes electric vehicles (BEVs: Battery Electric Vehicles), gasoline-powered vehicles, hybrid vehicles, and fuel cell vehicles. If the mobile object is something other than a vehicle 10, the terms "vehicle" and "car" in this disclosure may be replaced with "mobile object" as appropriate, and the term "driving" may be replaced with "moving" as appropriate.

In this embodiment, vehicle 10 is a four-wheeled automobile. In this embodiment, four-wheeled automobiles are classified into a single body type based on their vehicle class (also called "body size"), which is determined by their overall length, width, and height, and their shape. In this embodiment, body types include, for example, "SUV," "sedan," "station wagon," "minivan," "one-box," "compact car," and "kei car." In other words, a body type is a group of vehicle types that are classified in a way that groups together multiple types of four-wheeled automobiles according to their external shape. The type of four-wheeled automobile is, for example, a type of four-wheeled automobile classified by the vehicle name assigned to four-wheeled automobiles with the same or common external shape. In this case, for example, the vehicle class of a four-wheeled automobile is uniquely determined for each vehicle name.

Vehicle 10 comprises a drive unit 11, a steering unit 12, a braking unit 13, a communication device 14, and a vehicle control device 15. The drive unit 11 accelerates the vehicle 10. The steering unit 12 changes the direction of travel of the vehicle 10. The braking device 13 decelerates the vehicle 10. The communication device 14 communicates with external devices using wireless communication or the like. External devices include other devices such as an information processing device 6, and other vehicles 10. The communication device 14 is, for example, a wireless communication device. The vehicle control device 15 controls the operation of the vehicle 10. The vehicle control device 15 comprises a CPU 150, a storage unit 157, and an input/output interface 159. The input/output interface 159 is used to communicate with various devices mounted on the vehicle 10. The storage unit 157 of the vehicle control device 15 stores various information, including various programs that control the operation of the vehicle control device 15. The CPU 150 of the vehicle control device 15 functions as an operation control unit 154 by deploying various programs stored in the memory unit 157. The operation control unit 154 performs driving control of the vehicle 10 by controlling the operation of actuators that change the acceleration, deceleration, and steering angle of the vehicle 10. "Driving control" refers to various controls for driving actuators that perform the three functions of the vehicle 10: "driving," "turning," and "stopping," such as adjusting the acceleration, speed, and steering angle of the vehicle 10. In this embodiment, the actuators include the actuators of the drive unit 11, the actuators of the steering unit 12, and the actuators of the braking unit 13.

Vehicle 10 is capable of unmanned operation. "Unmanned operation" means operation without the driver's input. Driver's input refers to operations related to at least one of the following: "going," "turning," or "stopping" of vehicle 10. Unmanned operation is achieved by automatic or manual remote control using a device installed outside vehicle 10, or by autonomous control of vehicle 10. Even if vehicle 10 is operating unmanned, there may be a passenger on board who does not perform driver's input. Passengers who do not perform driver's input include, for example, a person simply sitting in the seat of vehicle 10, or a person performing tasks other than driver's input, such as assembly, inspection, or operating switches, while on board vehicle 10. Note that operation by a passenger is sometimes called "manned operation." In this embodiment, vehicle 10 travels unmanned within the factory by remote control by the information processing device 6. When vehicle 10 is operating by remote control, vehicle 10 travels by receiving a driving control signal that defines the vehicle's driving operation. The configuration of vehicle 10 is not limited to the above.

In this embodiment, the sensor is an external sensor 9, specifically a camera 90 (hereinafter referred to as the external camera 90), located at a location separate from the vehicle 10. The external camera 90 acquires images by capturing a detection range RG, including the vehicle 10, from outside the vehicle 10. Therefore, in this embodiment, the sensor information is the image acquired by the external camera 90. To capture the entire road 2 using one or more external cameras 90, the installation location and number of external cameras 90 are determined considering the detection range RG (angle of view) of the external camera 90, etc. The external camera 90 transmits the captured images to the information processing device 6.

Figure 2 shows the configuration of the information processing device 6 in the first embodiment. The information processing device 6 is used to control the operation of a vehicle 10 capable of autonomous driving. In this embodiment, the information processing device 6 acquires type information indicating the body type of the vehicle 10 represented by the captured image. Furthermore, the information processing device 6 calculates the position of the vehicle 10 using the type information and the captured image. In addition, the information processing device 6 generates a driving control signal that defines the driving operation of the vehicle 10 using the vehicle 10's position information, etc., and transmits it to the vehicle 10, thereby remotely controlling the vehicle 10's driving operation. The information processing device 6 comprises a communication unit 61, a CPU 62, a storage unit 63, and a notification unit 65. In the information processing device 6, the communication unit 61, CPU 62, storage unit 63, and notification unit 65 are each connected to one another, for example, via an internal bus.

The communication unit 61 of the information processing device 6 connects the information processing device 6 to other devices other than the information processing device 6 in a communication-enabled manner. The communication unit 61 of the information processing device 6 is, for example, a wireless communication device.

The storage unit 63 of the information processing device 6 stores various information, including various programs that control the operation of the information processing device 6, management information In, the ideal path Ip, the specific model Md, and the detection model Mf. Details of the ideal path Ip and the detection model Mf will be described later.

Management information In is information indicating the travel order of multiple vehicles 10 traveling within the detection range RG of the external camera 90, associated with their body types. Management information In is created, for example, using the position and orientation of the vehicles 10, the transmission history of travel control signals to the vehicles 10, and the installation location of the external camera 90.

The specific model Md is a pre-trained machine learning model that, when an image is input, outputs type information for each vehicle 10 represented by the image. The specific model Md learns features corresponding to the body type in order to identify the body type of the vehicle 10. These body type-specific features include, for example, the external shape and vehicle class of the vehicle 10. The specific model Md uses, for example, a convolutional neural network (CNN). The specific model Md is trained using supervised learning with a training dataset (hereinafter, the first training dataset) for training the specific model Md. The first training dataset is, for example, a dataset in which a ground truth label (hereinafter, type ground truth label) indicating the body type of each vehicle 10 represented by multiple images is associated. During CNN training, for example, the CNN parameters are updated to reduce the error between the output of the specific model Md and the type ground truth label by backpropagation. Note that the configuration of the specific model Md is not limited to the above. The specific model Md may, for example, be a pre-trained model that uses a method other than a neural network.

The CPU 62 of the information processing device 6 functions as an information acquisition unit 621, a first type acquisition unit 622, a second type acquisition unit 623, and a first judgment unit 624 by expanding various programs stored in the storage unit 63 of the information processing device 6. Furthermore, the CPU 62 of the information processing device 6 functions as an execution unit 625, a calculation unit 626, a signal generation unit 627, and a transmission unit 628 by expanding various programs stored in the storage unit 63 of the information processing device 6. The information acquisition unit 621 acquires captured images from the external camera 90.

The first type acquisition unit 622 acquires type information by inputting the captured image to a specific model Md. Hereinafter, the type information acquired by the first type acquisition unit 622 will also be referred to as the first type information. In this embodiment, the first type acquisition unit 622 acquires multiple sets of first type information by acquiring first type information for each of the multiple vehicles 10.

The second type acquisition unit 623 acquires type information by identifying the body type of the vehicle 10 represented by the captured image using management information In. Hereinafter, the type information acquired by the second type acquisition unit 623 will also be referred to as second type information. In this embodiment, the second type acquisition unit 623 acquires multiple sets of second type information by acquiring second type information for each of the multiple vehicles 10.

The first determination unit 624 determines whether there is a deficiency in at least one of the specific model Md and the management information In. In this embodiment, the first determination unit 624 compares the first type information and the second type information for each of the multiple vehicles 10. If the agreement rate is equal to or greater than a predetermined standard value, the first determination unit 624 determines that there is no deficiency in either the specific model Md or the management information In. The agreement rate is the ratio of the number of vehicles 10 in which the first type information and the second type information match, relative to the total number of multiple vehicles 10 compared. On the other hand, if the agreement rate is less than the standard value, the first determination unit 624 determines that there is a deficiency in at least one of the specific model Md and the management information In.

The execution unit 625 executes at least one of the following processes: the first process, the second process, the third process, and the fourth process, when the first determination unit 624 determines that there is a defect in at least one of the specific model Md and the management information In. The first process is to retrain the specific model Md. The second process is to notify the user of error information indicating that there may be a defect in at least one of the specific model Md and the management information In. The error information may be, for example, text information or audio information. In the second process, the execution unit 625 notifies the user of the error information via the notification unit 65. The third process is to store an error code indicating that there has been a defect in at least one of the specific model Md and the management information In in the storage unit 63 of the information processing device 6. The fourth process is to stop the vehicle 10. In this embodiment, in the fourth process, the execution unit 625 generates a stop signal to stop the vehicle 10. The stop signal includes, for example, the acceleration of the vehicle 10 as a parameter. The execution unit 625 calculates the vehicle's speed from the change in the vehicle's position, and determines the acceleration so that the vehicle 10 decelerates from the calculated speed. Furthermore, to stop the vehicle 10 at a desired location, the stop signal may also include the steering angle of the vehicle 10 as a parameter.

The calculation unit 626 calculates the position of the vehicle 10. In this embodiment, the calculation unit 626 calculates the position of the vehicle 10 using the captured image acquired by the external camera 90. For example, the calculation unit 626 calculates the coordinates of the vehicle 10's positioning point in the image coordinate system using the external shape of the vehicle 10 detected from the captured image. Then, according to the body type of the vehicle 10 identified by the type information, the calculation unit 626 converts the calculated coordinates in the image coordinate system to coordinates in the global coordinate system to calculate the position of the vehicle 10. In this embodiment, the position information of the vehicle 10 includes the X, Y, and Z coordinates in the factory's global coordinate system.

The external shape of the vehicle 10 included in the captured image can be detected, for example, by inputting the captured image into a detection model Mf utilizing artificial intelligence. The detection model Mf can be, for example, a pre-trained machine learning model trained to implement either semantic segmentation or instance segmentation. As the detection model Mf, for example, a CNN trained by supervised learning using a training dataset for training the detection model Mf (hereinafter referred to as the second training dataset) can be used. The second training dataset includes, for example, multiple training images including the vehicle 10, and ground truth labels (hereinafter referred to as region ground truth labels) indicating whether each region in the training image represents the vehicle 10 or something other than the vehicle 10. During CNN training, it is preferable to update the CNN parameters using backpropagation to reduce the error between the output result of the detection model Mf and the region ground truth labels.

Furthermore, the calculation unit 626 may also calculate the orientation of the vehicle 10. The orientation of the vehicle 10 can be estimated, for example, based on the direction of the vehicle's movement vector calculated from the positional changes of the vehicle 10's feature points between frames of the captured image, using the optical flow method. The orientation of the vehicle 10 may also be calculated, for example, using the output results of a yaw rate sensor mounted on the vehicle 10.

The signal generation unit 627 first determines the target location to which the vehicle 10 should next go in order to generate a driving control signal. In this embodiment, the target location is represented by the X, Y, and Z coordinates in the factory's global coordinate system. The signal generation unit 627 uses the vehicle 10's position information and an ideal path Ip pre-stored in the storage unit 73 of the remote control device 7 to determine the target location to which the vehicle 10 should next go. The ideal path Ip is the path that the vehicle 10 should travel. The path is represented by a node indicating the starting point, a node indicating a waypoint, a node indicating the destination, and links connecting each node. The signal generation unit 627 determines the target location on the ideal path Ip beyond the vehicle 10's current location. The signal generation unit 627 generates a driving control signal to drive the vehicle 10 toward the determined target location. In this embodiment, the driving control signal includes the vehicle 10's acceleration and steering angle as parameters. The signal generation unit 627 calculates the vehicle's speed from the change in the vehicle's position and compares the calculated speed with a predetermined target speed for the vehicle. If the vehicle speed is lower than the target speed, the signal generation unit 627 determines the acceleration to accelerate the vehicle. If the vehicle speed is higher than the target speed, the signal generation unit 627 determines the acceleration to decelerate the vehicle. If the vehicle is located on the ideal path Ip, the signal generation unit 627 determines the steering angle to prevent the vehicle from deviating from the ideal path Ip. If the vehicle is not located on the ideal path Ip, in other words, if the vehicle has deviated from the ideal path Ip, the signal generation unit 627 determines the steering angle to return the vehicle to the ideal path Ip. In this way, the signal generation unit 627 generates a driving control signal that includes the vehicle's acceleration and steering angle as parameters.

The transmitting unit 628 transmits various information to the vehicle 10. In this embodiment, the transmitting unit 628 transmits a stop signal and a drive control signal to the vehicle 10 to be controlled.

The notification unit 65 notifies the user of error information. The notification unit 65 is, for example, a display that shows error information as text. The notification unit 65 may also be a speaker that plays error information as audio. The notification unit 65 may be configured separately from the information processing device 6. Note that the configuration of the information processing device 6 is not limited to the above. Each part of the information processing device 6 may be configured separately by, for example, multiple devices. Furthermore, each part of the information processing device 6 may be implemented by, for example, cloud computing configured by one or more computers. Also, at least some of the functions of the information processing device 6 may be a function of the vehicle control device 15, or a function of the external sensor 9.

Figure 3 is a flowchart showing the processing flow in the first embodiment. Each step shown in Figure 3 is performed, for example, while the controlled vehicle 10 is being driven by remote control.

In step 101, the external camera 90 acquires multiple captured images. Each of the multiple captured images includes a different vehicle 10. An image identifier is associated with each captured image to distinguish it from the other captured images. In step 102, the external camera 90 transmits the multiple captured images to the information processing device 6.

In step 103, the information acquisition unit 621 of the information processing device 6 acquires multiple captured images from the external camera 90. In step 104, the first type acquisition unit 622 inputs each of the multiple captured images to a specific model Md. As a result, the first type acquisition unit 622 acquires multiple first type information by acquiring first type information for each vehicle 10. In step 105, the second type acquisition unit 623 uses management information In to identify the body type of each vehicle 10 represented by the multiple captured images. As a result, the second type acquisition unit 623 acquires multiple second type information by acquiring second type information for each vehicle 10. In step 106, the first determination unit 624 compares the first type information and the second type information for each of the multiple vehicles 10 to confirm whether the first type information and the second type information for each vehicle 10 match. In step 107, the first determination unit 624 calculates the match rate.

If the agreement rate is above the standard value (step 108: Yes), in step 109, the first determination unit 624 determines that there are no defects in either the specific model Md or the management information In. If the first determination unit 624 determines that there are no defects in either the specific model Md or the management information In, the calculation unit 626 executes step 110. In step 110, the calculation unit 626 calculates the position of the vehicle 10 using the captured image of the vehicle 10 to be controlled and the type information of the vehicle 10 to be controlled. In step 111, the signal generation unit 627 determines the next target position that the vehicle 10 should head to using the position information of the vehicle 10 and the ideal path Ip. In step 112, the signal generation unit 627 generates a driving control signal to drive the vehicle 10 toward the determined target position. In step 113, the transmission unit 628 transmits the driving control signal to the vehicle 10. In step 114, the operation control unit 154 of the vehicle control device 15 mounted on the vehicle 10 controls the actuator using the received driving control signal, causing the vehicle 10 to move at the acceleration and steering angle indicated in the driving control signal. If a predetermined time has elapsed since step 109 was determined (step 115: Yes), the process returns to step 103, and each step from step 103 onward is executed again. This causes the operation control unit 154 of the vehicle control device 15 to repeatedly receive the driving control signal and control the actuator at predetermined intervals.

On the other hand, if the agreement rate is below the standard value (Step 108: No), in Step 116, the first determination unit 624 determines that there is a deficiency in at least one of the specific model Md and the management information In. In Step 117, the execution unit 625 executes at least one of the following processes: the first process, the second process, the third process, and the fourth process.

According to the first embodiment described above, the information processing device 6 can acquire captured images and obtain first type information and second type information to determine whether or not there is a defect in at least one of the specific model Md and the management information In. If the information processing device 6 determines that there is a defect in at least one of the specific model Md and the management information In, it can perform the first process to retrain the specific model Md. In this way, the information processing device 6 can improve the accuracy of the specific model Md. As a result, the information processing device 6 can correct the defect in the specific model Md when one exists.

Furthermore, according to the first embodiment described above, the information processing device 6 can notify the user of the error information by executing the second process when it determines that a defect has occurred in at least one of the specific model Md and the management information In. In this way, the information processing device 6 can quickly inform the user that a defect has been determined in at least one of the specific model Md and the management information In. This allows the user to promptly perform the process to correct the defect in at least one of the specific model Md and the management information In.

Furthermore, according to the first embodiment described above, when the information processing device 6 determines that a defect has occurred in at least one of the specific model Md and the management information In, it can execute a third process to store an error code in the storage unit 63 of the information processing device 6. In this way, the user can, for example, recognize that a defect has been determined to have occurred in at least one of the specific model Md and the management information In by reading the error code. This allows the user to perform a process to correct the defect in at least one of the specific model Md and the management information In.

Furthermore, according to the first embodiment described above, the information processing device 6 can stop the vehicle 10 by executing the fourth process when it determines that a defect has occurred in at least one of the specific model Md and the management information In. Specifically, when the information processing device 6 determines that a defect has occurred in at least one of the specific model Md and the management information In, in the fourth process, it can stop the vehicle 10 by generating a stop signal and transmitting the stop signal to the vehicle 10.

Furthermore, according to the first embodiment described above, the acquired type information can be used to determine the driving operation of the vehicle 10. In this case, if the information processing device 6 determines that there is a deficiency in at least one of the specific model Md and the management information In, it can execute at least one of the first, second, third, and fourth processes. This allows the system to address deficiencies when they occur in at least one of the specific model Md and the management information In.

Furthermore, according to the first embodiment described above, the information processing device 6 can acquire first type information and second type information for each of the multiple vehicles 10. The information processing device 6 can then calculate the agreement rate by comparing the first type information and second type information for each of the multiple vehicles 10. By comparing the agreement rate with a reference value, the information processing device 6 can determine whether or not there is a deficiency in at least one of the specific model Md and the management information In.

Furthermore, according to the first embodiment described above, the information processing device 6 can acquire type information indicating the body type of the vehicle 10 represented by the captured image by inputting the captured image to a specific model Md. In other words, the information processing device 6 can identify the body type of the vehicle 10 represented by the captured image by utilizing artificial intelligence.

Furthermore, according to the first embodiment described above, the information processing system 1 can remotely control the movement of the vehicle 10 within the factory. Therefore, the information processing system 1 can move the vehicle 10 within the factory without using transport equipment such as cranes or conveyors. Note that the vehicle 10 may also be driven unmanned outside the factory.

B. Second Embodiment:
Figure 4 shows the configuration of the information processing device 6a in the second embodiment. In this embodiment, when there is a defect in the management information In, the information processing device 6a acquires type information by inputting the captured image as sensor information to a specific model Md. The CPU 62a of the information processing device 6a functions as an information acquisition unit 621, a first type acquisition unit 622a, a second judgment unit 629, a calculation unit 626, a signal generation unit 627, and a transmission unit 628 by deploying various programs stored in the storage unit 63a. The same reference numerals are used for components identical to those in the first embodiment.

The second determination unit 629 determines whether or not there is a deficiency in the management information In. For example, if a user moves some of the vehicles 10 that are traveling along an ideal route Ip in a predetermined order, deviating from the ideal route Ip, the management information In may be modified by the user. In this case, the user deletes the vehicles 10 that were moved deviating from the ideal route Ip from the management information In, for example, via an input device. At this time, the user may mistakenly delete a vehicle 10 different from the vehicle 10 that was moved deviating from the ideal route Ip from the management information In. If the user deletes a vehicle 10 different from the desired vehicle 10 from the management information In, the contents of the management information In become incorrect. Therefore, the second determination unit 629 determines whether or not there is a deficiency in the management information In, for example, using change information that indicates the changes made to the management information In.

Furthermore, for example, when the information processing device 6a acquires management information In from another device, there is a non-zero possibility of communication errors such as bit corruption occurring during the communication process of management information In. Therefore, if a communication error occurs during the communication process of management information In, the content of management information In may become incorrect. Accordingly, the second determination unit 629 uses, for example, result information indicating the result of a parity check on management information In acquired by the information processing device 6a from another device to determine whether or not there is a defect in management information In.

The first type acquisition unit 622a acquires type information by inputting the captured image to a specific model Md when the second determination unit 629 determines that there is a deficiency in the management information In.

Figure 5 is a flowchart showing the processing flow in the second embodiment. Each step shown in Figure 5 is performed, for example, while the controlled vehicle 10 is being driven by remote control.

In step 201, the external camera 90 acquires an image by capturing the detection range RG, which includes the vehicle 10 to be controlled. In step 202, the external camera 90 transmits the captured image to the information processing device 6a.

In step 203, the information acquisition unit 621 of the information processing device 6a acquires an image captured from the external camera 90. In step 204, the second determination unit 629 determines whether or not there is a defect in the management information In. If the second determination unit 629 determines that there is a defect in the management information In (step 204: Yes), in step 205, the first type acquisition unit 622a inputs the captured image to a specific model Md. As a result, the first type acquisition unit 622a acquires type information about the vehicle 10 represented by the captured image. On the other hand, if the second determination unit 629 determines that there is no defect in the management information In (step 204: No), in step 206, the second type acquisition unit 623 acquires type information about the vehicle 10 represented by the captured image using the management information In. In step 207, the calculation unit 626 calculates the position of the vehicle 10 using the captured image and the type information about the vehicle 10 represented by the captured image. In step 208, the signal generation unit 627 uses the vehicle 10's position information and the ideal path Ip to determine the next target location for the vehicle 10. In step 209, the signal generation unit 627 generates a driving control signal to drive the vehicle 10 toward the determined target location. In step 210, the transmission unit 628 transmits the driving control signal to the vehicle 10. In step 211, the operation control unit 154 of the vehicle control device 15 mounted on the vehicle 10 uses the received driving control signal to control the actuators, causing the vehicle 10 to move at the acceleration and steering angle indicated in the driving control signal. The operation control unit 154 of the vehicle control device 15 repeats the reception of the driving control signal and the control of the actuators at predetermined intervals.

According to the second embodiment described above, the information processing device 6a can determine whether or not there is a deficiency in the management information In. If the information processing device 6a determines that there is a deficiency in the management information In, it can acquire type information by inputting the captured image to a specific model Md. In other words, even if there is a deficiency in the management information In, the information processing device 6a can acquire type information by another method. This allows the information processing device 6a to continuously calculate the position of the vehicle 10 and generate a driving control signal. Therefore, the vehicle 10 can continue to drive under remote control.

Furthermore, the information processing device 6a may also include an execution unit that executes at least one of the fifth process and the sixth process if the second determination unit 629 determines that there is a defect in the management information In. The fifth process is to notify the user via the notification unit 65 of error information indicating that there may be a defect in the management information In. The sixth process is to store an error code indicating that a defect has been determined in the management information In in the storage unit 63a of the information processing device 6a. This makes it easier for the user to perform the process of correcting the defect in the management information In.

C. Third Embodiment:
Figure 6 shows the configuration of the information processing device 6b in the third embodiment. In this embodiment, the information processing device 6b determines whether or not there are defects in the specific model Md and the management information In. Then, the information processing device 6b executes different processes according to the determination result. The CPU 62b of the information processing device 6b functions as an information acquisition unit 621, a first type acquisition unit 622, a second type acquisition unit 623, and an execution unit 625b by deploying various programs stored in the storage unit 63b. Furthermore, the CPU 62b of the information processing device 6b functions as a calculation unit 626b, a signal generation unit 627, a transmission unit 628, a reliability calculation unit 630, and a third determination unit 631. Note that the same reference numerals are used for components identical to those in the first embodiment.

The confidence calculation unit 630 calculates the confidence level by comparing the output result of the specific model Md with the type-correct label. The confidence level indicates the accuracy of the specific model Md. The confidence level can be, for example, the accuracy, precision, recall, or F-score for the specific model Md.

The third determination unit 631 determines whether or not there are defects in the specific model Md and the management information In, respectively. In this embodiment, the third determination unit 631 uses the agreement rate calculated by the same method as in the first embodiment and the reliability of the specific model Md to determine whether or not there are defects in the specific model Md and the management information In, respectively. Specifically, the third determination unit 631 determines that there are defects in both the specific model Md and the management information In if the agreement rate is below the standard value and the reliability is below the threshold. The third determination unit 631 determines that there are no defects in either the specific model Md or the management information In if the agreement rate is above the standard value and the reliability is above the threshold. The third determination unit 631 determines that there are no defects in the specific model Md, but there are defects in the management information In, if the agreement rate is below the standard value and the reliability is above the threshold. The third judgment unit 631 determines that there is no defect in the management information In, but there is a defect in the specific model Md, when the agreement rate is above the standard value and the confidence level is below the threshold.

The execution unit 625b performs the following processes according to the determination result of the third determination unit 631. Specifically, if the third determination unit 631 determines that there is a defect in either the specific model Md or the management information In, the execution unit 625b performs at least one of the first process, second process, third process, and fourth process shown in the first embodiment. If the third determination unit 631 determines that there is a defect in the management information In, the execution unit 625b performs at least one of the fifth process and sixth process shown in the second embodiment. If the third determination unit 631 determines that there is a defect in the specific model Md, the execution unit 625b performs at least one of the first process, seventh process, and eighth process. The seventh process is a process that notifies the user via the notification unit 65 of error information indicating that there may be a defect in the specific model Md. The eighth process involves storing an error code indicating that a defect has been determined in a specific model Md in the storage unit 63b of the information processing device 6b.

If the third determination unit 631 determines that there are no defects in either the specific model Md or the management information In, the calculation unit 626b calculates the position of the vehicle 10 as follows. In this case, the calculation unit 626b calculates the position of the vehicle 10 using the captured image of the vehicle 10 to be controlled and any type information from either the first type information or the second type information for the vehicle 10 to be controlled. If the third determination unit 631 determines that there are defects in the management information In, the calculation unit 626b calculates the position of the vehicle 10 using the captured image of the vehicle 10 to be controlled and the first type information for the vehicle 10 to be controlled. If the third determination unit 631 determines that there are defects in the specific model Md, the calculation unit 626b calculates the position of the vehicle 10 using the captured image of the vehicle 10 to be controlled and the second type information for the vehicle 10 to be controlled.

Figure 7 is a first flowchart showing the processing flow in the third embodiment. Figure 8 is a second flowchart showing the processing flow in the third embodiment. Each step shown in Figures 7 and 8 is executed, for example, while the controlled vehicle 10 is being driven by remote control. Note that steps 301 to 305 are the same as steps 101 to 105 shown in Figure 3, and therefore their explanation is omitted.

As shown in Figure 7, after step 305, in step 306, the third determination unit 631 compares the first type information and the second type information for each of the multiple vehicles 10 to determine whether the first type information and the second type information for each vehicle 10 match. In step 307, the third determination unit 631 calculates the matching rate.

If the agreement rate is below the standard value (Step 308: No) and the confidence level is below the threshold (Step 309: No), in Step 310, the third determination unit 631 determines that there is a deficiency in both the specific model Md and the management information In. In Step 311, the execution unit 625b executes at least one of the following processes: the first process, the second process, the third process, and the fourth process.

If the agreement rate is above the standard value (step 308: Yes) and the confidence level is above the threshold (step 309: Yes), in step 312, the third determination unit 631 determines that there are no defects in either the specific model Md or the management information In. As shown in Figure 8, in step 313, the calculation unit 626 calculates the position of the vehicle 10 using the captured image of the vehicle 10 to be controlled and arbitrary type information about the vehicle 10 to be controlled. In step 314, the signal generation unit 627 determines the next target position that the vehicle 10 should head to using the position of the vehicle 10 and the ideal path Ip. In step 315, the signal generation unit 627 generates a driving control signal to drive the vehicle 10 toward the determined target position. In step 316, the transmission unit 628 transmits the driving control signal to the vehicle 10. In step 317, the operation control unit 154 of the vehicle control device 15 mounted on the vehicle 10 uses the received driving control signal to control the actuator, thereby driving the vehicle 10 at the acceleration and steering angle indicated in the driving control signal.

As shown in Figure 7, if the agreement rate is below the standard value (Step 308: No) and the confidence level is above the threshold (Step 309: Yes), in Step 318, the third determination unit 631 determines that there is no defect in the specific model Md, but there is a defect in the management information In. After Step 318, Steps 319 and 320 shown in Figure 8 are executed. In Step 319, the calculation unit 626b calculates the position of the vehicle 10 using the captured image of the vehicle 10 to be controlled and the first type information about the vehicle 10 to be controlled. After Step 320, each of the steps from Step 314 to Step 317 is executed similarly. In Step 320, the execution unit 625b executes at least one of the fifth process and the sixth process.

As shown in Figure 7, if the agreement rate is above the standard value (step 308: Yes) and the confidence level is below the threshold (step 309: No), in step 321, the third determination unit 631 determines that there is no defect in the management information In, but there is a defect in the specific model Md. After step 321, steps 322 and 323 shown in Figure 8 are executed. In step 322, the calculation unit 626b calculates the position of the vehicle 10 using the captured image of the vehicle 10 to be controlled and the second type information about the vehicle 10 to be controlled. After step 323, each of the steps from step 314 to step 317 is executed in the same manner. In step 323, the execution unit 625b executes at least one of the first process, the seventh process, and the eighth process. Furthermore, if the information processing device 6b determines that any of steps 312, 318, or 321 has occurred, and a predetermined time has elapsed since then (step 324: Yes), it returns to step 303 as shown in Figure 7, and each step from step 303 onward is executed again. As a result, the operation control unit 154 of the vehicle control device 15 repeatedly receives the driving control signal and controls the actuator at predetermined intervals.

According to the third embodiment described above, the information processing device 6b can determine whether or not there are defects in both the specific model Md and the management information In. Based on this, the information processing device 6b can address the defects by executing each of the processes from the first to the eighth process, depending on the object in which the defect is located.

Furthermore, according to the third embodiment described above, the information processing device 6b can acquire type information by inputting the captured image to a specific model Md if it determines that there is a defect in the management information In. The information processing device 6b can acquire type information using the management information In if it determines that there is a defect in the specific model Md. In other words, even if there is a defect in either the specific model Md or the management information In, the information processing device 6b can acquire type information using either the specific model Md or the management information In that it determines does not have a defect. As a result, the information processing device 6b can continuously calculate the position of the vehicle 10 and generate a driving control signal. Therefore, the vehicle 10 can continue to drive under remote control.

D. Fourth Embodiment:
Figure 9 shows the configuration of the information processing system 1c in the fourth embodiment. In this embodiment, the information processing system 1c comprises one or more vehicles 10c equipped with a vehicle control device 15c that also has some of the functions of the information processing device 6, an external camera 90 as a sensor, and an information processing device 6c. In this embodiment, the vehicle control device 15c calculates the position of the vehicle 10c, determines the next target position to go to, and generates a driving control signal for driving toward the determined target position. The other configurations of the information processing system 1c are the same as in the first embodiment unless otherwise specified.

The vehicle control device 15c comprises an input/output interface 159, a storage unit 157c, and a CPU 150c. The storage unit 157c of the vehicle control device 15c stores various information, including various programs that control the operation of the vehicle control device 15c, a detection model Mf, and an ideal path Ip. The CPU 150c of the vehicle control device 15c functions as an information acquisition unit 151, a calculation unit 152, a signal generation unit 153, and an operation control unit 154 by deploying the various programs stored in the storage unit 157c. The information acquisition unit 151 acquires various information. In this embodiment, the information acquisition unit 621 acquires captured images from the external camera 90. Furthermore, the information acquisition unit 151 acquires type information from the information processing device 6. The information acquisition unit 151 of the vehicle control device 15c has the same functions as the information acquisition unit 621 of the information processing device 6 described in the first embodiment. The calculation unit 152 calculates the position of the vehicle 10c using the captured images and type information. The calculation unit 152 of the vehicle control device 15c has the same function as the calculation unit 626 of the information processing device 6 described in the first embodiment. The signal generation unit 153 uses the position information of the vehicle 10c and the ideal path Ip to determine the next target position to which the vehicle 10c should go. Then, the signal generation unit 153 generates a driving control signal to drive the vehicle 10c toward the determined target position. The signal generation unit 153 of the vehicle control device 15c has the same function as the signal generation unit 627 of the information processing device 6 described in the first embodiment. The vehicle control device 15c repeats the acquisition of the vehicle 10c's position information, determination of the target position, generation of the driving control signal, and control of the actuator at a predetermined cycle.

Figure 10 shows the configuration of the information processing device 6c in the fourth embodiment. The information processing device 6c comprises a communication unit 61, a storage unit 63c, and a CPU 62c. The storage unit 63c of the information processing device 6c stores various programs for controlling the operation of the information processing device 6c, a specific model Md, and management information In. The CPU 62c of the information processing device 6c functions as an information acquisition unit 621, a first type acquisition unit 622, a second type acquisition unit 623, a first determination unit 624, an execution unit 625, and a transmission unit 628 by deploying the various programs stored in the storage unit 63c. In this embodiment, the transmission unit 628 transmits type information to the vehicle 10c.

Figure 11 is a flowchart showing the processing flow in the fourth embodiment. Each step shown in Figure 11 is performed, for example, while the controlled vehicle 10c is moving under remote control.

In step 401, the external camera 90 acquires multiple captured images. Each of the multiple captured images includes a different vehicle 10c. An image identifier is associated with each captured image to distinguish it from the other captured images. In step 402, the external camera 90 transmits the multiple captured images to the information processing device 6 and the vehicle 10c.

In step 403, the information acquisition unit 621 of the information processing device 6 acquires multiple captured images from the external camera 90. In step 404, the first type acquisition unit 622 inputs each of the multiple captured images to a specific model Md. As a result, the first type acquisition unit 622 acquires multiple first type information by acquiring first type information for each vehicle 10c. In step 405, the second type acquisition unit 623 uses management information In to identify the body type of each vehicle 10c represented by the multiple captured images. As a result, the second type acquisition unit 623 acquires multiple second type information by acquiring second type information for each vehicle 10c. In step 406, the first determination unit 624 compares the first type information and the second type information for each of the multiple vehicles 10c to confirm whether the first type information and the second type information for each vehicle 10c match. In step 407, the first determination unit 624 calculates the match rate.

If the agreement rate is above the standard value (step 408: Yes), in step 409, the first determination unit 624 determines that there are no defects in either the specific model Md or the management information In. If the first determination unit 624 determines that there are no defects in either the specific model Md or the management information In, in step 410, the transmission unit 628 transmits the type information to the vehicle 10c. When the information acquisition unit 151 of the vehicle control device 15c mounted on the vehicle 10c acquires the type information from the information processing device 6, the calculation unit 152 executes step 411. In step 411, the calculation unit 152 calculates the position of the vehicle 10c using the captured image of the vehicle 10c to be controlled and the type information of the vehicle 10c to be controlled. In step 412, the signal generation unit 153 determines the target position that the vehicle 10c should next head to using the position information of the vehicle 10c and the ideal path Ip. In step 413, the signal generation unit 153 generates a driving control signal to move the vehicle 10c toward the determined target position. In step 414, the operation control unit 154 uses the driving control signal to control the actuator, thereby moving the vehicle 10c at the acceleration and steering angle indicated in the driving control signal. If a predetermined time has elapsed since step 409 was determined (step 415: Yes), the process returns to step 403, and each step from step 403 onward is executed again. In this way, the vehicle control device 15c repeatedly performs the acquisition of vehicle 10c position information, determination of the target position, reception of the driving control signal, and control of the actuator at predetermined intervals.

On the other hand, if the agreement rate is below the standard value (step 408: No), in step 416, the first determination unit 624 determines that there is a deficiency in at least one of the specific model Md and the management information In. In step 417, the execution unit 625 executes at least one of the following processes: the first process, the second process, the third process, and the fourth process.

According to the fourth embodiment described above, the information processing system 1c can drive the vehicle 10c autonomously without remotely controlling the vehicle 10c using the information processing device 6c.

In the fourth process, the information processing device 6c may transmit a stop instruction to the vehicle 10c instead of a stop signal. Upon receiving the stop instruction, the vehicle 10c generates a stop signal by determining its acceleration to decelerate below its current speed. The vehicle 10c then stops by controlling its actuators using the stop signal.

E. Other embodiments:
E-1. Other Embodiment 1:
The threshold value used to determine whether or not there are defects in the specific model Md and the management information In, based on a comparison with the agreement rate, may be 100 percent. In this configuration, the information processing devices 6, 6a to 6c can determine that there are defects in at least one of the specific model Md and the management information In if the threshold value is less than 100 percent.

E-2. Another Embodiment 2:
The information processing devices 6, 6a to 6c may determine whether there is a defect in at least one of the specific model Md and the management information In by comparing the first type information and the second type information for a single vehicle 10. In this case, the information processing devices 6, 6a to 6c will determine, for example, that there is a defect in at least one of the specific model Md and the management information In if the first type information and the second type information do not match. Even in this configuration, the information processing devices 6, 6a to 6c can determine whether there is a defect in at least one of the specific model Md and the management information In.

E-3. Other Embodiment 3:
The second determination unit 629 may determine whether or not there is a defect in the management information In by an indirect method, as shown in the third embodiment, in addition to the direct method shown in the second embodiment. The direct method is a method of determining whether or not there is a defect in the management information In using information related to the management information In, such as change information and result information. The indirect method is a method of determining whether or not there is a defect in the management information In using information other than information related to the management information In, in addition to or instead of information related to the management information In. For example, as an indirect method, the second determination unit 629 may determine whether or not there is a defect in the management information In using the agreement rate and the confidence level. In this configuration, the second determination unit 629 can determine whether or not there is a defect in the management information In by at least one of the direct method and the indirect method.

E-4. Other Embodiments 4:
The specific model Md may be a trained model that outputs the type of vehicle 10, 10c as the body type. In this configuration, the information processing devices 6, 6a to 6c can input sensor information into the specific model Md to obtain type information indicating the type of vehicle 10, 10c represented by the sensor information.

E-5. Other Embodiments 5:
The specific model Md may be a pre-trained model learned using either a random forest or a support vector machine (SVM). Even in this configuration, the information processing devices 6, 6a to 6c can input sensor information into the specific model Md to obtain type information about the vehicles 10, 10c represented by the sensor information.

E-6. Other Embodiments 6:
The sensor information used to acquire type information may be information acquired by a sensor of a different type than the external camera 90. The sensor information may be, for example, measurement point cloud data representing the external shape of the vehicles 10 and 10c in three dimensions, acquired by a LiDAR (Light Detection and Ranging) (hereinafter referred to as the external LiDAR) as the external sensor 9. In this configuration, the information processing devices 6, 6a to 6c can acquire type information of the vehicles 10 and 10c using the measurement point cloud information acquired by the external LiDAR.

E-7. Other Embodiments 7:
Vehicles 10 and 10c may further be equipped with one or more on-board sensors. On-board sensors are sensors mounted on vehicles 10 and 10c. On-board sensors are, for example, on-board cameras, on-board radars, and on-board lidars. On-board cameras capture images of the area surrounding vehicles 10 and 10c. On-board radars and on-board lidars detect objects present in the area surrounding vehicles 10 and 10c. If vehicles 10 and 10c are equipped with on-board sensors, the sensor information used to acquire type information may be information acquired by on-board sensors of other vehicles 10 and 10c. In this configuration, information processing devices 6, 6a to 6c can acquire type information of vehicles 10 and 10c using sensor information acquired by on-board sensors.

E-8. Other Embodiments 8:
The calculation units 152 and 626 may calculate the position and orientation of the vehicles 10 and 10c using information acquired by a sensor of a different type than the external camera 90. The calculation units 152 and 626 may also calculate the position and orientation of the vehicles 10 and 10c using, for example, measurement point cloud data acquired by an external lidar. In this case, the calculation units 152 and 626 calculate the position and orientation of the vehicles 10 and 10c by, for example, matching the measurement point cloud data with reference point cloud data. The reference point cloud data is reference data used as a template in matching with the measurement point cloud data. The reference point cloud data is, for example, virtual 3D point cloud data generated based on 3D CAD data representing the external shape of the vehicles 10 and 10c. The matching algorithm used by the calculation units 152,626 may include, for example, ICP (Iterative Closest Point) and NDT (Normal Distributions Transform). Alternatively, the calculation units 152,626 may calculate the position and orientation of the vehicles 10,10c using measurement point cloud data acquired by detecting the target vehicles 10,10c using an on-board lidar mounted on a vehicle different from the target vehicle 10,10c.

E-9. Other Embodiments 9:
In each of the embodiments from the first to the third embodiment described above, the information processing devices 6, 6a, and 6b perform the processing from acquiring the vehicle's position information to generating the driving control signal. Alternatively, the vehicle 10 may perform at least a part of the processing from acquiring the vehicle's position information to generating the driving control signal. For example, the following forms (1) to (3) may also be used.

(1) The information processing devices 6, 6a, and 6b may acquire the location information of the vehicle 10, determine the next target location that the vehicle 10 should head to, and generate a route from the vehicle 10's current location, as shown in the acquired location information, to the target location. The information processing devices 6, 6a, and 6b may generate a route to the target location between the current location and the destination, or they may generate a route to the destination. The information processing devices 6, 6a, and 6b may transmit the generated route to the vehicle 10. The vehicle 10 may generate a driving control signal to travel along the route received from the information processing devices 6, 6a, and 6b, and may use the generated driving control signal to control the actuators.

(2) The information processing devices 6, 6a, and 6b may acquire the location information of the vehicle 10 and transmit the acquired location information to the vehicle 10. The vehicle 10 may determine the next target location to which it should go, generate a route from the vehicle 10's current location to the target location as shown in the received location information, generate a driving control signal so that the vehicle 10 travels along the generated route, and control the actuators using the generated driving control signal.

(3) In the embodiments of (1) and (2) above, the vehicle 10 is equipped with internal sensors, and the detection results output from the internal sensors may be used in at least one of the generation of the route and the generation of the driving control signal. Internal sensors may include, for example, an on-board camera, an on-board lidar, a millimeter-wave radar, an ultrasonic sensor, a GPS sensor, an acceleration sensor, a gyroscope, and the like. For example, in the embodiment of (1) above, the information processing devices 6, 6a, and 6b may acquire the detection results from the internal sensors and reflect them in the route when generating the route. In the embodiment of (1) above, the vehicle 10 may acquire the detection results from the internal sensors and reflect them in the driving control signal when generating the driving control signal. In the embodiment of (2) above, the vehicle 10 may acquire the detection results from the internal sensors and reflect them in the route when generating the route. In the embodiment of (2) above, the vehicle 10 may acquire the detection results from the internal sensors and reflect them in the driving control signal when generating the driving control signal.

(4) In the fourth embodiment described above, the vehicle 10c is equipped with an internal sensor, and the detection result output from the internal sensor may be used in at least one of the process of generating the route and generating the driving control signal. For example, the vehicle 10c may acquire the detection result from the internal sensor and reflect the detection result in the route when generating the route. The vehicle 10c may acquire the detection result from the internal sensor and reflect the detection result in the driving control signal when generating the driving control signal.

(5) In the fourth embodiment described above, the vehicle 10c acquires its position information using the detection results of the external sensor 9. Alternatively, the vehicle 10c may be equipped with an internal sensor. The vehicle 10c may acquire position information using the detection results of the internal sensor, determine the next target location to be reached, generate a route from the vehicle 10c's current location (as shown in the acquired position information) to the target location, generate a driving control signal for traveling along the generated route, and control the actuators using the generated driving control signal. In this case, the vehicle 10c can travel without using the detection results of the external sensor 9 at all. Furthermore, the vehicle 10c may acquire target arrival time and congestion information from outside the vehicle 10c and reflect this information in at least one of the route and the driving control signal. Also, the entire functional configuration of the information processing system 1c may be provided within the vehicle 10c. That is, the processing realized by the information processing system 1c, such as the acquisition of type information as shown in this disclosure, may be realized by the vehicle 10c alone.

(6) In each of the embodiments from the first to the third embodiment described above, the information processing devices 6, 6a, and 6b automatically generate driving control signals to be transmitted to the vehicle 10. In contrast, the information processing devices 6, 6a, and 6b may generate driving control signals to be transmitted to the vehicle 10 according to the operations of an operator located outside the vehicle 10. For example, an operator may operate a control device equipped with a display for displaying images output from an external sensor 9, a steering wheel for remotely controlling the vehicle 10, an accelerator pedal, a brake pedal, and a communication device for communicating with the information processing devices 6, 6a, and 6b via wired or wireless communication. The information processing devices 6, 6a, and 6b may then generate driving control signals in response to the operations performed on the control device.

E-10. Other Embodiments 10:
In each of the above embodiments, the vehicles 10 and 10c only need to be configured to be able to move by unmanned operation, and may, for example, be in the form of a platform having the configuration described below. Specifically, in order for the vehicles 10 and 10c to perform the three functions of "driving,""turning," and "stopping" by unmanned operation, they only need to be equipped with at least a vehicle control device 15 and 15c, a drive device 11, a steering device 12, and a braking device 13. When the vehicles 10 and 10c acquire information from the outside for unmanned operation, they may further be equipped with a communication device 14. That is, the vehicles 10 and 10c that can move by unmanned operation do not need to have at least some of the interior parts such as a driver's seat and dashboard, at least some of the exterior parts such as bumpers and fenders, and do not need to have a body shell. In this case, the remaining parts such as the body shell may be attached to the vehicles 10 and 10c before they are shipped from the factory, or the remaining parts such as the body shell may be attached to the vehicles 10 and 10c after they have been shipped from the factory, while the remaining parts such as the body shell are not attached to the vehicles 10 and 10c. Each part may be attached from any direction, such as the top, bottom, front, rear, right, or left side of the vehicles 10 and 10c, and they may be attached from the same direction or from different directions. The platform configuration may also be positioned in the same way as the vehicles 10 and 10c in the above embodiments.

E-11. Other Embodiments 11:
Vehicles 10 and 10c may be manufactured by combining multiple modules. A module means a unit composed of multiple parts grouped together according to the part or function of the vehicle 10 and 10c. For example, the platform of vehicle 10 and 10c may be manufactured by combining a front module that constitutes the front part of the platform, a central module that constitutes the central part of the platform, and a rear module that constitutes the rear part of the platform. The number of modules that constitute the platform is not limited to three, but may be two or fewer, or four or more. In addition to, or instead of, the parts that constitute the platform may be modularized, as well as parts that constitute parts of vehicle 10 and 10c that are different from the platform. Furthermore, various modules may include any exterior parts such as bumpers and grilles, or any interior parts such as seats and consoles. Moreover, not limited to vehicles 10 and 10c, any form of mobile body may be manufactured by combining multiple modules. Such modules may be manufactured, for example, by joining multiple parts by welding or fasteners, or by integrally molding at least a part of the parts that constitute the module as a single part by casting. A molding technique for integrally molding a single component, especially a relatively large component, is also called gigacast or megacast. For example, the front module, central module, and rear module mentioned above may be manufactured using gigacast.

E-12. Other Embodiments 12:
Transporting vehicles 10,10c using unmanned operation is also called "self-propelled transport." The configuration for realizing self-propelled transport is also called a "vehicle remote control autonomous driving transport system." Furthermore, a production method that uses self-propelled transport to produce vehicles 10,10c is also called "self-propelled production." In self-propelled production, for example, at a factory that manufactures vehicles 10,10c, at least a portion of the transport of vehicles 10,10c is realized by self-propelled transport.

This disclosure is not limited to the embodiments described above, and can be implemented in various configurations without departing from its spirit. For example, the technical features of the embodiments corresponding to the technical features in each form described in the summary of the invention can be replaced or combined as appropriate to solve some or all of the above-described problems, or to achieve some or all of the above-described effects. Furthermore, if a technical feature is not described as essential in this specification, it can be deleted as appropriate.

1, 1c... Information processing system, 2... Track, 6, 6a-6c... Information processing device, 9... External sensor, 10, 10c... Vehicle, 11... Drive system, 12... Steering system, 13... Braking system, 14... Communication device, 15, 15c... Vehicle control device, 61... Communication unit of the information processing device, 62, 62a-62c... CPU of the information processing device, 63, 63a-63c... Memory unit of the information processing device, 65... Notification unit, 90... External camera, 150, 150c... CPU of the vehicle control device, 151, 621... Information acquisition unit, 152 626, 626b…Calculation Unit, 153, 627…Signal Generation Unit, 154…Operation Control Unit, 157, 157c…Vehicle Control Device Storage Unit, 159…Input/Output Interface, 622, 622a…First Type Acquisition Unit, 623…Second Type Acquisition Unit, 624…First Decision Unit, 625, 625b…Execution Unit, 628…Transmission Unit, 629…Second Decision Unit, 630…Reliability Calculation Unit, 631…Third Decision Unit, In…Management Information, Ip…Ideal Path, Md…Specific Model, Mf…Detection Model, RG…Detection Range

Claims (5)

An information processing device used for driving a mobile body that can be moved by unmanned operation,
Memory unit and,
An information acquisition unit that acquires sensor information representing the moving object detected using a sensor,
A first type acquisition unit acquires type information by inputting the sensor information to a machine learning model that outputs type information indicating the body type of the mobile body represented by the sensor information, which is classified according to the external shape of the mobile body when the sensor information is input.
A second type acquisition unit acquires type information by identifying the body type of the moving body represented by the sensor information, using management information that indicates the travel order of a plurality of moving bodies traveling within the detection range of the sensor in association with the body type,
A first determination unit compares the first type information obtained by the first type acquisition unit as the type information and the second type information obtained by the second type acquisition unit as the type information to determine whether or not there is a defect in at least one of the machine learning model and the management information.
An information processing device comprising: an execution unit that performs at least one of the following processes when the first determination unit determines that the aforementioned defect has occurred: a first process of retraining the machine learning model; a second process of notifying the user of error information indicating that the aforementioned defect may have occurred; a third process of storing an error code indicating that the aforementioned defect has been determined to have occurred in the storage unit; and a fourth process of stopping the moving object. An information processing apparatus according to claim 1,
In the fourth process, the execution unit generates a stop signal to stop the moving body.
The information processing device further comprises a transmitting unit that transmits the stop signal to the moving body. An information processing apparatus according to claim 1,
The first judgment unit,
For each of the multiple moving bodies, the first type information and the second type information are compared,
An information processing device that determines that no defect has occurred if the ratio of the number of mobile bodies for which the first type information and the second type information match, relative to the total number of mobile bodies compared, is equal to or greater than a standard value, and determines that a defect has occurred if it is less than the standard value. An information processing apparatus according to claim 3,
The aforementioned reference value is 100 percent, in this information processing device. An information processing device used for driving a mobile body that can be moved by unmanned operation,
An information acquisition unit that acquires sensor information representing the moving object detected using a sensor,
A second judgment unit that determines whether or not there are deficiencies in the management information,
The system includes a first type acquisition unit that, when the second determination unit determines that the management information has the aforementioned deficiency, inputs the sensor information into a machine learning model to acquire type information,
The type information is a body type of the mobile body classified according to the external shape of the mobile body, and is information indicating the body type of the mobile body represented by the sensor information.
The management information is information that indicates the travel order of a plurality of moving bodies traveling within the detection range of the sensor, in association with the body type.
The machine learning model is an information processing device that outputs the type information when the sensor information is input.