JP7329983B2 - Learning data generation device and object detection device - Google Patents

Description

The present invention relates to a learning data generation device and an object detection device.

2. Description of the Related Art Conventionally, a platform, which is a place for getting on and off a vehicle such as a railway, is provided with a platform door for preventing users from entering the vehicle track. The platform door has a door panel that opens and closes, and this door panel is arranged to correspond to the door of the vehicle.

The platform door opens the door panel when the vehicle stops at the platform and a passenger gets on or off the vehicle, and closes the door panel while the vehicle is running or when the vehicle is not parked on the platform.

A platform door is provided with a sensor for detecting an obstacle such as a user or an obstacle in the vicinity of the platform door. Based on the detection result of the sensor, the opening and closing of the door panel or the operation of the vehicle is controlled to ensure the safety of the passenger.

A three-dimensional distance sensor is used as a platform door sensor, and the three-dimensional distance sensor scans the detection area to acquire a three-dimensional distance image, and the presence or absence of obstacles in the detection area is determined. .

A detection area is generated to include the space between the platform door and the vehicle. Here, the boundaries that define the regions within the detection area are set so as to ensure the safety of passengers and prevent excessive detection. For example, the boundaries of the detection area are defined when the passenger puts his hand on the upper part of the platform door or puts his feet on the lower part of the platform door. It is set not to detect such situations.

From the viewpoint of ensuring the safety of users, it is desirable to install platform doors in as many places as possible.

However, the high cost of the three-dimensional distance sensor that detects obstacles within the detection area hinders the promotion of the installation of platform doors.

Therefore, it has been proposed to determine the presence or absence of an obstacle by photographing an image around the door of the vehicle with a camera. For example, a vehicle door monitoring system disclosed in Patent Document 1 captures a camera image around a vehicle door using a camera, detects the door from the camera image, and sets a determination area from the position of the detected door. Then, an image of the determination area is obtained from the camera image, and whether or not a foreign object is caught in the door is determined based on the image of the determination area.

JP 2018-167594 A

Here, the shape or size of the platform door or the distance between the platform door and the vehicle may differ depending on the combination of the platform door and the vehicle. The boundaries may differ from platform door to platform door and vehicle combination.

Also, in order to detect an object within a detection area using an image captured by a camera, it is required to find the position of the three-dimensional object from the image, which is two-dimensional information.

Therefore, in this specification, using an image of the detection area set on the platform door, learning data for a machine learning device that can detect objects in the detection area that may differ depending on the combination of platform door and vehicle. An object of the present invention is to provide a learning data generation device that generates data.

In addition, in this specification, an object is detected using a machine learning device that can detect objects in the detection area, which can differ for each platform door and for each vehicle, using an image of the detection area set on the platform door. An object of the present invention is to provide an object detection device that detects objects.

According to the learning data generation device disclosed in the present specification, a region including a detection area set for the platform door of the platform where the vehicle stops is a plurality of images captured using a camera, and the detection A detection area is detected using a 3D range image sensor when a plurality of images whose presence or absence of an object in an area is detected using a 3D range image sensor is input, and each of the plurality of images is captured. an input unit for inputting the detection result obtained; a label generation unit for generating a label for identifying the presence or absence of an object in the detection area based on the detection result; a learning data generation unit that generates learning data for the machine learning device by associating a label generated based on the detection result at the time of detection; and a storage unit that stores the learning data.

Further, according to the object detection device disclosed in this specification, the relationship between each of a plurality of images and the labels associated with the images is determined using the learning data generated by the learning data generation device described above. It is placed in the same positional relationship as when the image of the learning data was taken with respect to the machine learning machine containing the learning model that learns the sex and the detection area set for the platform door of the platform where the vehicle stops. and a determination unit that inputs an image captured by the camera to a machine learning device and determines the presence or absence of an object in the detection area based on the output of the machine learning device.

According to the above-described learning data generation device disclosed in this specification, images can be used to generate learning data for a machine learner capable of detecting objects in detection areas that may differ from one platform door to another and from one vehicle to another. .

Further, according to the object detection device disclosed in the present specification described above, an image can be used to detect an object within a detection area that may differ from one platform door to another and from one vehicle to another.

1 is a diagram showing the configuration of one embodiment of the machine learning system disclosed in this specification; FIG. It is a perspective view explaining the detection area set to a platform door. It is a top view explaining the detection area set to a platform door. It is a sectional view explaining a detection area set to a platform door. FIG. 11 is a diagram (part 1) showing an example in which an object exists in a detection area; FIG. 12 is a diagram (part 2) showing an example in which an object exists in the detection area; FIG. 13 is a diagram (part 3) showing an example in which an object exists in the detection area; FIG. 12 is a diagram (part 4) showing an example in which an object exists in the detection area; 4 is a flowchart of processing for generating learning data; 6 is a flowchart of processing for generating a learning model; 1 is a diagram showing the configuration of an embodiment of an obstacle detection device disclosed in this specification; FIG.

The learning data generation device will be described below with reference to the drawings. This learning data generation device is a plurality of images of an area including a detection area set for a platform door of a platform where a vehicle stops, photographed using a camera, and the presence or absence of an object in the detection area. A plurality of images are input that are sensed using a 3D range image sensor. Further, the learning data generation device inputs the detection results obtained by detecting the detection area using the three-dimensional distance image sensor when each of the plurality of images is captured. The learning data generation device generates a label for identifying the presence or absence of an object within the detection area based on the detection result. Then, the learning data generation device associates each of the plurality of images with a label generated based on the detection result when the image was captured, and generates learning data for the machine learning device. , is stored in the storage unit.

An example in which the learning data generation device is applied to a machine learning system will be described below. In this example, the learning data generation device generates learning data for the machine learner, and the machine learner uses this learning data to learn the relationship between the image and the label. However, the technical scope of the present invention is not limited to those embodiments, but extends to the invention described in the claims and equivalents thereof.

FIG. 1 is a diagram showing the configuration of one embodiment of the machine learning system disclosed in this specification.

A system 1 of this embodiment includes a learning data generation device 10 , an object detection device 20 , a machine learning device 60 and a camera 50 . The camera 50 captures an image of an area including the detection area 40 set for the platform door 30 installed on the platform 2, as shown in FIGS.

The platform door 30 has two door pocket panels 35 that house the door panels 33 and a protective post 36 that supports the door pocket panels 35 . The obstacle detection device 20 is arranged on the track-side surface of the door pocket panel 35 of the platform door 30 .

The platform door 30 closes the door panel 33 except when the vehicle 6 stops at the platform 2 and the user gets on and off the vehicle, and prevents the user on the platform 2 from entering the vehicle track.

The object detection device 20 includes a communication unit 21 that can communicate with the learning data generation device 10, a three-dimensional range image sensor (hereinafter simply referred to as a sensor) 22 that acquires a three-dimensional range image, and a processing unit that controls them. 23. As shown in FIGS. 2 to 4, the sensor 22 scans the space between the platform door 3 and the vehicle 6 at a predetermined cycle to acquire a three-dimensional range image and output it to the processing unit 23. FIG. For example, the sensor 22 irradiates and scans a pulsed laser beam within the range of the angle of view, measures the time required for the reflected light to return, measures the distance for each pixel, and measures the distance of each pixel. Acquire the original range image.

The processing unit 23 controls the sensor 22 based on the setting file that stores the detection area 40 . Each time a three-dimensional range image is input from the sensor 22, the processing unit 23 determines the presence or absence of an object within the detection area 40, and transmits the detection result indicating the presence or absence of the object along with the detection time via the communication unit 21. and transmits it to the learning data generation device 10 .

As shown in FIGS. 2 to 4, the detection area 40 is a three-dimensional area set in the space between the platform door 30 and the vehicle 6. FIG. As the sensor 22 scans the detection area 40 , the object detection device 20 detects an object within the detection area 40 .

The detection area 40 has a flat shape that corresponds to the shape of the space between the platform door 30 and the vehicle 6 as a whole. It has a second area 42 which is a region on the door 30 side. The detection area 40 has a first area 41 and a second area 42 to ensure the safety of passengers and set boundaries so as to prevent excessive detection.

The first area 41 has a substantially flat rectangular parallelepiped shape, and the second area 42 also has a substantially flat rectangular parallelepiped shape. The dimension of the second area 42 in the height direction (perpendicular to the floor surface of the platform 2) is shorter than that of the first area 41. As shown in FIG. The position of the second area 42 in the height direction is determined when the customer puts his or her hand on the upper portion of the door panel 33 or when the customer puts his/her feet on the lower portion of the door panel 33. is determined so that the patron does not detect his hands or feet.

The shape of the first area 41 on the vehicle track side can be determined so as to correspond to the outline of the vehicle. Similarly, the shape of the platform door 30 side of the second area 42 can be determined so as to correspond to the shape of the door panel 33 and the like. As a result, the degree of matching between the shape of the detection area 40 and the shape of the space between the platform door 30 and the vehicle 6 is increased, so that the obstacle detection accuracy is improved. Note that the detection area 40 may not be divided into the first area and the second area 42 .

The detection area 40 has a boundary closer to the sensor 22 and a boundary farther from the sensor 22 for each pixel of the three-dimensional range image acquired by the sensor 22 . Using the three-dimensional distance image measured by the sensor 22, the processing unit 23 of the object detection device 20 determines that the distance obtained from each pixel exists within the detection area 40, that is, the boundary closer to the sensor 22. , and the boundary farther from the sensor 22 , it is determined that an object exists within the detection area 40 .

FIG. 5 is a diagram showing an example in which passengers 501 and 502, which are examples of objects, exist within the detection area 40. As shown in FIG. FIG. 6 is a diagram showing an example in which a bag 601, which is an example of an object, exists within the detection area 40. As shown in FIG. The bag 601 is caught in the door 603 of the vehicle 602 and protrudes into the detection area 40 . FIG. 7 is a diagram showing an example in which an umbrella 701, which is an example of an object, exists within the detection area 40. As shown in FIG. Umbrella 701 is sandwiched between doors 703 of vehicle 702 and protrudes into detection area 40 . FIG. 8 is a diagram showing an example in which a can 801, which is an example of an object, exists within the detection area 40. As shown in FIG. The can 801 falls onto platform 2 .

The camera 50 is controlled by the learning data generation device 10 and takes an image of the detection area 40 set on the platform door 30 as shown in FIG. The camera 50 is an example of an imaging unit, and includes a two-dimensional detector composed of an array of photoelectric conversion elements sensitive to visible light, such as a CCD or C-MOS, and an object to be photographed on the two-dimensional detector. It has an imaging optical system that forms an image of the area. The optical axis and angle of view of the camera 50 are adjusted so as to photograph the space including the detection area 40 . The camera 50 transmits the captured image to the learning data generation device 10 . In the example shown in FIG. 2, the camera 50 is arranged so as to photograph the detection area 40 obliquely from above. You may make it photograph. The camera 50 transmits the photographed image together with the photographing time to the learning data generation device 10 .

The learning data generation device 10 uses the camera 50 to photograph an area including the detection area 40 set for the platform door 30 of the platform 2 where the vehicle stops. In addition, the learning data generation device 10 inputs a detection result indicating the presence or absence of an object in the detection area 40 each time the camera 50 acquires an image, and assigns a label for identifying the presence or absence of the object in the detection area 40. create. The learning data generation device 10 associates a label with each of a plurality of images to generate and store learning data for a machine learning device.

The learning data generation device 10 has an operation unit 11 , a display unit 12 , a communication unit 13 , a storage unit 14 and a processing unit 15 .

The operation unit 11 is operated by a user to input information necessary for generating learning data. The operation unit 11 has an input device such as a keyboard and a mouse for inputting a user's operation.

The display unit 12 displays information, such as images captured by the camera 50, used for processing for generating learning data. The display unit 12 has, for example, a liquid crystal display.

The communication unit 13 has an interface circuit for connecting the learning data generation device 10 to the object detection device 20 , the camera 50 and the machine learning device 60 . The communication unit 13 is an example of an input unit that inputs images captured by the camera 50 . Also, the communication unit 13 is an example of an input unit that inputs the detection result from the object detection device 20 .

The storage unit 14 has, for example, a volatile semiconductor memory and a nonvolatile semiconductor memory. The storage unit 14 stores the image captured by the camera 50, the detection result of the object detection device 20 when the image was captured, the generated learning data, and the like. For example, an image captured by the camera 50 is stored in the storage unit 14 in association with a detection result having a detection time that matches the capture time.

The processing unit 15 has one or more processors and their peripheral circuits. The processing section 15 may further comprise other arithmetic circuits such as a logical operation unit, a numerical operation unit or a graphics processing unit. Then, the processing unit 15 acquires the image captured by the camera 50 from the storage unit 14 or the like, and identifies the presence or absence of the object in the detection area 40 based on the detection result of the object detection device 20 when the image was captured. A label is generated, the label is associated with the image, and learning data for a machine learning device is generated. Then, the processing unit 15 stores the image associated with the label in the storage unit 14 as learning data.

The processing unit 15 includes an image acquisition unit 15a, a label generation unit 15b, and a learning data generation unit 15c for the process of generating learning data for the machine learning device. Each of these units included in the processing unit 15 is, for example, a functional module implemented by a computer program running on the processor. Alternatively, these units included in the processing unit 15 may be dedicated arithmetic circuits provided in the processing unit 15 . The processing unit 15 has a mode of acquiring original data for learning and a mode of generating data for learning with respect to processing for generating learning data for the machine learning device.

In the mode of acquiring original data for learning, the image acquisition unit 15a uses the camera 50 to acquire a plurality of images of an area including the detection area 40 set for the platform door 30 of the platform 2 where the vehicle 6 stops. to shoot. Then, the image acquiring unit 15 a receives the image of the detection area 40 captured and the capturing time from the camera 50 via the communication unit 13 and stores it in the storage unit 14 . In addition, in the learning data generating mode, the image acquiring unit 15a reads the image and the shooting time from the storage unit 14, notifies the label generating unit 15b of the shooting time, and sends the image and the shooting time to the learning data generating unit. 15c. Also,

In the mode of acquiring the original data for learning, the label generation unit 15b associates the detection result with the detection time and stores the detection result in the storage unit 14 each time it receives the detection result and the detection time from the object detection device 20 . In addition, in the learning data generating mode, the label generation unit 15b reads out from the storage unit 14 the detection result of the detection time that matches the shooting time every time the image acquisition unit 15a notifies the shooting time. , based on this detection result, a label for identifying the presence or absence of an object in the detection area 40 is generated, and this label is notified to the learning data generation unit 15c together with the photographing time. If there is no detection result for a detection time that matches the photographing time, the label generating unit 15b reads out from the storage unit 14 the detection results within a predetermined time range with respect to the photographing time, and performs detection based on the detection result. A label may be generated that identifies the presence or absence of an object within the area 40 . Further, the label generator 15b may generate a label so as to include information for identifying whether or not an object detected within the detection area is an obstacle that hinders safe operation of the vehicle 6 . Objects that are obstacles include, for example, a user, a bag or an umbrella carried by the user, and the like. Non-obstructive objects include, for example, rain, snow, fog, drink containers such as plastic bottles or cans, insects, and sunlight. In this case, the label generation unit 15b reads the image generated by the camera 50 from the storage unit 14 and displays it on the display unit 12. FIG. The user visually confirms the image displayed on the display unit 12, determines whether the object is an obstacle when the object is displayed, and learns the determination result using the operation unit 11. input to the data generation device 10 for use. The label generation unit 15b adds information for identifying whether the object is an obstacle or not to the label associated with the image based on the input judgment result. A machine learning device that has learned using such learning data can identify the presence or absence of an obstacle in the detection area 40 with respect to the image captured by the camera. Further, the label generation unit 15b may generate a label so as to include information for identifying the type of object detected within the detection area. In this case, the label generation unit 15b reads the image generated by the camera 50 from the storage unit 14 and displays it on the display unit 12. FIG. The user looks at the image displayed on the display unit 12, and if an object is displayed, determines the type of the object, and inputs the type of the object to the learning data generation device 10 using the operation unit 11. . The label generator 15b adds information identifying the type of the input object to the label associated with the image. For example, the label generator 15b may generate a label to include information identifying whether an object detected within the detection area is a vehicle. A machine learning device that has learned using such learning data can identify the presence or absence of a vehicle entering the vehicle track at a station in the image captured by the camera. Furthermore, the label generator 15b may generate a label to include information identifying whether the object detected within the detection area is a vehicle door. A machine learner trained using such training data can be used to identify images of vehicle doors. Then, by checking the change over time of the areas of the regions representing the respective doors in the continuously photographed images, it is possible to further identify whether the doors of the vehicle 6 are being opened or closed. As described above, the label generator 15b can generate a label for each image that identifies information such as the presence or absence of an object in the detection area, whether the object is an obstacle, and the type of the object.

The learning data generation unit 15c generates learning data for the machine learning device by associating the label notified from the label generation unit 15b with the image notified from the image acquisition unit 15a using the shooting time. and stored in the storage unit 14.

Next, the process of generating learning data by the machine learning system 1 will be described below with reference to the flowchart shown in FIG.

First, the camera 50 captures a plurality of images of the detection area 40 controlled by the learning data generation device 10, transmits these images to the learning data generation device 10, and the object detection device 20 detects the detection area. 40 is transmitted to the learning data generation device 10 (step S901). In this process, as much as possible situations such as the types of objects that can enter the detection area 40 of the platform doors 30 and the positions of the objects in the detection area 40 are assumed on the actual platform 2, and an image representing each situation is created. It is preferable that the presence/absence of an object within the detection area 40 at the time of photographing is detected along with photographing.

Next, the learning data generation device 10 stores the plurality of images received from the camera 50 in association with the detection results received from the object detection device 20 (step S902).

Next, the learning data generation device 10 generates a label for identifying the image based on the detection result, and based on the type of the object if an object is detected within the detection area (step S903). In this process, if there are multiple images, a label identifying each image may be collectively generated.

Next, the learning data generation device 10 associates each of the plurality of images with a label generated based on the detection result generated when the image was captured, and generates learning data for the machine learning device. is generated (step S904). In this process, if there are multiple images, the process of associating a label with each image may be performed collectively.

Next, the learning data generation device 10 stores the learning data (step S905).

Next, the machine learning device 60 will be explained below.

The machine learning device 60 has one or more processors and their peripheral circuits. The machine learner 60 may further have other arithmetic circuits such as a logical arithmetic unit, a numerical arithmetic unit or a graphics processing unit. The machine learning device 60 has a communication unit (not shown) that communicates with the learning data generation device 10 and a storage unit (not shown) that stores information used for machine learning processing.

The machine learning device 60 has a learning model 61 , an error calculator 62 and a learning model updater 63 . As the learning model 61, a deep neural network (DNN) that learns image features by supervised learning and learns relationships between images and labels can be used. A model other than the DNN may be used as the learning model 61 . A learning model 61 learns the relationship between each of a plurality of images and the label associated with that image.

The operation of the machine learning device 60 to generate a learning model will be described below with reference to the flowchart shown in FIG.

First, the learning model 61 receives an image of learning data from the learning data generation device 10 (step S1001). For example, information for each pixel forming an image is input to the input layer of a deep neural network.

Next, the error calculator 62 inputs the label of the learning data from the learning data generation device 10, and calculates the error based on the output and the label obtained from the learning model 61 (step S1002).

Next, the learning model updating unit 63 determines whether or not the error satisfies a predetermined condition (step S1003). For example, the learning model updating unit 63 determines that a predetermined condition is satisfied if the numerical error is equal to or less than a predetermined threshold value (step S1003-Yes). The processing shown in FIG. 10 is performed using all the learning data images generated by the learning data generation device 10 .

On the other hand, if the error does not satisfy the predetermined condition (step S1003-No), the learning model updating unit 63 updates the learning model 61 based on the error, and the learning model 61 updates the same learning data again. An image is input (step S1001). For example, the learning model updating unit 63 may back-propagate the error to update the weight parameters of the neural network. The learning model 61 thus generated by the machine learner 60 can be used by other machine learners.

Next, an obstacle detection device, which is an example of an object detection device equipped with a machine learning device having the learning model described above, will be described below with reference to FIG. 11 .

FIG. 11 is a diagram showing the configuration of an embodiment of the obstacle detection device disclosed in this specification. The obstacle detection device 70 detects the presence or absence of an obstacle within the detection area 40 of the platform door 30 based on the image captured by the camera 50 . The obstacle detection device 70 detects the presence or absence of an obstacle in the detection area 40 and outputs a detection signal indicating the presence or absence of the obstacle to the platform door 30 . The platform door 30 opens and closes the door panel 33 based on the detection signal.

The platform door 30 has a drive unit 32 that drives the door panel 33 to open and close, a communication unit 34 that communicates with the obstacle detection device 70, and a control unit 31 that controls them. The configuration of the platform door 30 is as shown in FIGS. 2 to 4. FIG. Note that the platform door does not have to be the same as the one used to generate the learning data. The platform door may have the same detection area as that used to generate the learning data.

As shown in FIG. 2, the camera 50 is arranged with respect to the detection area 40 of the platform door 30 in the same positional relationship as when the images for generating the learning data for the machine learning device are taken. That is, the optical characteristics of the camera 50, such as the optical axis and angle of view, and the relative positional relationship with the detection area 40 are the same as when the image for generating the learning data for the machine learning device is captured. preferable. Note that the camera does not have to be the same as the one used to generate the training data. The camera has the same optical characteristics, such as the optical axis and angle of view, as well as the imaging optical system used to generate the training data. It suffices if they are arranged in the same positional relationship as when the image of the learning data was captured.

The obstacle detection device 70 has a communication section 71 , a storage section 72 , a machine learning device 60 and a processing section 73 .

The communication unit 71 has an interface circuit for connecting to the platform door 30 and the camera 50 . Communication unit 71 outputs the image captured by camera 50 to processing unit 73 .

The storage unit 72 has, for example, a volatile semiconductor memory and a nonvolatile semiconductor memory. The storage unit 72 stores various types of information used in the obstacle detection process.

A machine learning device 60 similar to the machine learning system 1 described above can be used. The machine learning device 60 has a learning model 61 that has learned the relationship between images and labels using the learning data generated by the learning data generation device 10 .

The processing unit 73 has one or more processors and their peripheral circuits. The processing section 73 may further comprise other arithmetic circuits such as a logical operation unit, a numerical operation unit or a graphics processing unit. Then, the processing unit 73 inputs the image captured by the camera to the machine learning device 60 and determines whether or not there is an obstacle in the detection area 40 based on the output of the machine learning device 60 . An obstacle is an example of an object.

The processing unit 73 has a determination unit 73a for the process of detecting obstacles. Further, the processing unit 73 has a door opening/closing detection unit 73b for processing for detecting an obstacle, and for processing for detecting opening/closing operation of the door of a vehicle entering the railroad track of a station. Each of these units included in the processing unit 73 is, for example, a functional module implemented by a computer program running on the processor. Alternatively, these units included in the processing unit 73 may be dedicated arithmetic circuits provided in the processing unit 73 .

The determination unit 73 a inputs the image generated by the camera 50 to the machine learning device 60 and determines whether or not there is an obstacle in the detection area 40 based on the output of the machine learning device 60 . The determination unit 73a determines that there is an obstacle in the detection area 40 when the machine learning device 60 identifies an obstacle in the image. The determination unit 73a can detect an obstacle such as a user, a bag, or an umbrella in the detection area 40. FIG. On the other hand, the determination unit 73a does not detect objects such as rain, snow, fog, beverage containers such as PET bottles or cans, insects, and sunlight in the detection area 40 as obstacles. Accordingly, the determination unit 73a can determine whether or not there is an obstacle positioned within a complicated spatial shape such as the detection area 40 based on the image generated by the camera 50. FIG. When determining that there is an obstacle within the detection area 40 , the determination unit 73 a transmits a detection signal requesting opening of the door panel 33 to the platform door 30 via the communication unit 71 . The determination unit 73 a also inputs the image generated by the camera 50 to the machine learning device 60 and determines the type of object based on the output of the machine learning device 60 . For example, when the machine learning device 60 identifies that the object shown in the image is a train, the determination unit 73a can determine that the train is entering the vehicle track of the station. This makes it possible to detect a train using the obstacle detection device 70 without separately providing a sensor for detecting the train. Further, when the machine learning device 60 identifies that the object shown in the image is a train door, the determination unit 73a notifies the door opening/closing detection unit 73b of the image.

The door open/close detector 73b identifies the area corresponding to the train door in the notified image. The door opening/closing detection unit 73b receives, for example, a feature value (for example, Histograms of Oriented Gradients, HOG) calculated from a window set on the image, and the door of the train to be detected is displayed in the window. A pre-trained support vector machine (SVM) may be used to output the confidence. Further, the door opening/closing detection unit 73b may detect the area of the door by performing template matching between a template representing a train door to be detected and an image. The door opening/closing detection unit 73b identifies areas corresponding to the train doors in the images, which are continuous in time series, and increases the areas of these areas. or is decreasing. Then, when the area increases, the door opening/closing detection unit 73b determines that the doors of the train are closing. On the other hand, when the area is decreasing, the door opening/closing detection unit 73b determines that the doors of the train are opening. This makes it possible to detect the open/closed state of the train door using the obstruction detection device 70 without separately providing a sensor for detecting the open/closed state of the train door. For example, when the processing unit 73 detects an obstacle in the detection area 40 and determines that the doors of the train are closing, the processing unit 73 sends a signal requesting the platform door 30 to open the door panel 33 to the communication unit. 71 may be sent. In the image captured while the train is running, the doors of the train are in a closed state. does not determine that the door is being opened or closed.

In addition, when the platform door 30 has a three-dimensional distance sensor, the obstacle detection device 70 uses the camera 50 to acquire the image of the learning data and the detection result, and the learning model of the machine learner 60 61 may be generated or updated.

Since the obstacle detection device 70 can be applied to the platform door having the same detection area as that for which the learning data was generated, if one learning model is generated, the obstacle detection device using this learning model can be applied. 70 can be placed at the same platform door that generated the training data. As a result, the safety of passengers can be further ensured by arranging a plurality of obstacle detection devices 70 at each platform door without using an expensive three-dimensional distance.

In the present invention, the learning data generation device and the object detection device of the above-described embodiments can be modified as appropriate without departing from the gist of the present invention.

Also, the object detection device has a three-dimensional distance image sensor that detects the presence or absence of an object within the detection area, and a machine learning device capable of learning the relationship between the image of the detection area and the type of object in the image. may When an object within the detection area is detected using a 3D range image sensor, the object detection apparatus uses a machine learning device to identify whether or not the object in the image is an obstacle, and detects the object within the detection area. may be detected.

1 system 2 home 6 vehicle 10 learning data generation device 11 operation unit 12 display unit 13 communication unit 14 storage unit 15 processing unit 15a image acquisition unit 15b label generation unit 15c learning data generation unit 20 object detection device 21 communication unit 22 tertiary Original range image sensor 23 processing unit 30 platform door 31 control unit 32 driving unit 33 door panel 34 communication unit 35 door pocket panel 36 protection post 40 detection area 41 first area 42 second area 50 camera 60 machine learning device 61 learning model 62 error calculation Part 63 Learning model update part 70 Obstacle detection device 71 Communication part 72 Storage part 73 Processing part 73a Judgment part 73b Door opening/closing detection part

Claims (4)

inputting a plurality of images of an area including a detection area set for the platform door of the platform where the vehicle is parked and photographed using a camera, and when each of the plurality of images is photographed ; an input unit for inputting a detection result obtained by detecting the presence or absence of an object in the detection area using a dimensional distance image sensor;
a label generation unit that generates a label for identifying the presence or absence of an object in the detection area based on the detection result;
a learning data generation unit that generates learning data for a machine learning device by associating each of the plurality of images with the label generated based on the detection result when the image was captured;
a storage unit that stores the learning data;
has
The learning data generation device , wherein the label includes information identifying whether the object is a vehicle . 2. The learning data generation device according to claim 1, wherein said label further includes information identifying whether said object is an obstacle that hinders safe operation of the vehicle. 2. The learning data generation device according to claim 1 , wherein said label further includes information identifying whether said object is a vehicle door. A learning model that learns the relationship between each of the plurality of images and the labels associated with the images using the learning data generated by the learning data generation device according to claim 1. a machine learner;
a camera that is arranged in the same positional relationship as when the image of the learning data was captured with respect to a detection area set for the platform door of the platform where the vehicle stops, and captures the detection area;
a determination unit that inputs an image captured by the camera to the machine learning device and determines whether an object exists in the detection area based on the output of the machine learning device;
An object detection device having

Images