JP7694897B2 - Object detection device, system, method, and program - Google Patents

Description

The present invention relates to an object detection device, system, method, and program.

In order to manage the safety of workers at work sites, there is a technology that uses 3D (dimension) cameras (3D-LIDAR (Laser Imaging Detection and Ranging), ToF (Time of Flight) cameras, stereo cameras, 3D scanners, etc.) to set up no-entry areas (no-entry spaces) in dangerous areas where people and objects are prohibited from entering, and monitors these no-entry areas to detect the intrusion of objects (see, for example, Patent Documents 1 to 5).

Patent Document 1 discloses an intrusion detection device that includes a portable sensor unit, an information acquisition unit that acquires new sensor observation information detected by the portable sensor unit from the portable sensor unit, an area setting unit that sets a new detection area within an observation area corresponding to the new sensor observation information, and an intrusion detection unit that detects the intrusion of an animal body into the new detection area, and the area setting unit sets the new detection area based on a comparison between the new sensor observation information and past sensor observation information and past detection area information. In the intrusion detection device described in Patent Document 1, a 3D camera is installed so that a scene to be managed for safety is included within a range that can be captured by the 3D camera, the area to be monitored is grasped as a "3D space", and one or more no-entry areas are set. When setting the no-entry areas, the site is photographed with a 3D camera, the angle of view is adjusted so that the scene to be managed for safety is included in the angle of view of the 3D camera, and the no-entry areas are set while referring to the photographed 3D point cloud data.

Patent Document 2 discloses a projection image designation device that is equipped with a projection device capable of projecting a predetermined range, a photographing device capable of photographing the range that the projection device can project, and a figure recognition unit that is connected to the projection device and recognizes a figure drawing operation by a user based on an image photographed by the photographing device, and a projection control unit that instructs the projection device to project a figure based on the figure drawing operation, as a technology for providing a simple means of designating an image to be projected by a projection device. In the projection image designation device described in Patent Document 2, an area to be monitored is set by the user performing a figure drawing operation by placing three or more markers.

Patent Document 3 discloses a construction machine having a laser scanner device, distance data acquisition means for acquiring distance data for each scan position in the surrounding space based on light reflected in the surrounding space from light periodically emitted from the laser scanner device into the surrounding space, distance calculation means for calculating the distance to the worker based on radio waves received from an ID transmission means worn by the worker, object position detection means for detecting the position of an object in the surrounding space based on the distance data, worker position detection means for detecting the position of the worker based on the distance calculated by the distance calculation means and the position detected by the object position detection means, monitoring area setting means for setting a monitoring area, first alarm output means for outputting an alarm based on the presence of an object detected by the object position detection means in the monitoring area, and second alarm output means for outputting an alarm based on the presence of a worker detected by the worker position detection means in the monitoring area. In the construction machine described in Patent Document 3, the monitoring area is dynamically set according to the turning range of the working device.

Patent Document 4 discloses a robot system that includes an object recognition device that includes an imaging unit that acquires three-dimensional and two-dimensional images of a predetermined monitoring area, an object extraction unit that extracts an area having pixel values in a predetermined range in the acquired three-dimensional image, an image search unit that searches for a reference image registered in advance according to the type of object in the two-dimensional image, and a determination unit that determines the type of object based on whether the reference image searched by the image search unit is present in the area extracted by the object extraction unit, a robot that works in a predetermined working area in the monitoring area, and a robot control unit that controls the robot, and when the object recognition device recognizes that the object is a safety monitoring target, it detects the object entering the working area, and when the object recognition device detects the object entering the working area, the robot control unit stops the robot. In the robot system described in Patent Document 4, the monitoring area is set to include the working area of the robot installed at a predetermined position.

Patent document 5 discloses a technology for avoiding contact between humans and industrial robots, which includes a robot, a sensor that detects human intrusion into the area surrounding the robot, and a control device that controls the robot and the sensor, and the control device includes a robot control unit that controls the robot based on an operation program, an operation area prediction unit that successively predicts the operation area of the robot until a predetermined time based on the operation program, and a detection area control unit that sets the sensor detection results to valid in an area of the surrounding area that includes the operation area predicted by the operation area prediction unit and sets the sensor detection results to invalid in other surrounding areas, and the robot control unit restricts the operation of the robot when a human intrusion is detected within the area where the sensor detection results are set to valid. In the robot system described in Patent document 5, the area detectable by the sensor is set to the area surrounding the robot installed at a predetermined position.

JP 2020-197971 A JP 2020-113102 A JP 2020-56169 A JP 2019-193974 A JP 2019-58990 A

The following analysis is provided by the present inventors.

However, the intrusion detection device described in Patent Document 1 requires the installation of a 3D camera inside the restricted area surrounded by traffic cones, making it impossible to monitor multiple restricted areas with a single 3D camera, and there is no flexibility in the installation location of the 3D camera.

In railway construction, for example in the case of quadruple or triple track construction, the tracks to be constructed shift depending on the time of day according to the construction plan, and the no-entry areas change accordingly. However, with the intrusion detection device described in Patent Document 1, tuning work such as setting no-entry areas while finely adjusting the space is required each time, which is time-consuming and makes it difficult to immediately start intrusion detection for safety management. The same can be said about the projection image designation device described in Patent Document 2, which sets the area to be monitored by installing markers.

In addition, safety management for workers at construction sites is not limited to work on the ground; workers also work at heights such as on utility poles and steel towers, making it necessary to set up no-entry areas in the air. However, the intrusion detection device described in Patent Document 1 cannot set up no-entry areas in the air by placing triangular cones to mark the no-entry areas.

In addition, the intrusion detection device described in Patent Document 1 sets up restricted areas using triangular cones that serve as markers to indicate no-entry areas, so there is a possibility that the triangular cones may get in the way of work.

Furthermore, the construction machine described in Patent Document 3 and the robot systems described in Patent Documents 4 and 5 only monitor a preset range centered on the construction machine or robot, and are not configured to set up new no-entry spaces.

The main objective of the present invention is to provide an object detection device, system, method, and program that can contribute to the efficient setting of new no-entry spaces.

The object detection device according to the first viewpoint includes a no-entry space designation unit configured to detect a landmark corresponding to a form related to a preset landmark in the photographed data, create a plurality of no-entry spaces according to a preset form based on the detected landmark , and select one of the plurality of created no-entry spaces ; and a detection processing unit configured to detect a detection target object that matches predetermined or newly set detection condition information based on a point cloud within the selected no-entry space in the photographed data.

The object detection system relating to the second viewpoint includes a three-dimensional sensor configured to output image data obtained by photographing a specified space, and an object detection device relating to the first viewpoint.

An object detection method relating to a third perspective is an object detection method that detects objects using hardware resources, and includes the steps of detecting landmarks in photographed data that correspond to a form related to a predetermined landmark, creating a plurality of no-entry spaces according to a predetermined form based on the detected landmarks , and selecting one of the plurality of created no-entry spaces, and detecting a detection target object that matches previously or newly set detection condition information based on a point cloud within the selected no-entry space in the photographed data.

A program relating to the fourth perspective is a program that causes a hardware resource to execute a process of detecting an object, and causes the hardware resource to execute the following processes : detect landmarks in the photographed data that correspond to a form related to a preset landmark, create a plurality of no-entry spaces based on the detected landmarks in accordance with a preset form, and select one of the plurality of created no-entry spaces; and detect a detection target object that matches preset or newly set detection condition information based on a point cloud within the selected no-entry space in the photographed data.

The program can be recorded on a computer-readable storage medium. The storage medium can be a non-transient medium such as a semiconductor memory, a hard disk, a magnetic recording medium, or an optical recording medium. In the present disclosure, the program can also be embodied as a computer program product. The program is input to the computer device from an input device or an external device via a communication interface, stored in a storage device, and drives the processor according to a predetermined step or process, and can display the processing results, including intermediate states as necessary, at each stage via a display device, or can communicate with the outside via the communication interface. As an example, a computer device for this purpose typically includes a processor, a storage device, an input device, a communication interface, and a display device as necessary, all of which can be connected to each other via a bus.

The first to fourth aspects above can contribute to efficient work of setting up new no-entry spaces.

1 is a block diagram illustrating a configuration and one mode of use of an object detection system according to a first embodiment. 1 is a block diagram illustrating a schematic configuration of an object detection device in an object detection system according to a first embodiment. FIG. 2 is an image diagram illustrating an example of a display screen in the object detection system according to the first embodiment. 3 is an image diagram showing an example of captured data after preprocessing in a preprocessing unit of the object detection device in the object detection system according to the first embodiment; FIG. 1 is an image showing a schematic example of photographed data after a detection line relating to a railroad track is created by a landmark recognition section of a no-entry space designation section of an object detection device in an object detection system according to embodiment 1. 1 is an image that illustrates an example of photographed data after a no-entry space candidate is automatically created by a landmark recognition section of a no-entry space designation section of an object detection device in an object detection system according to embodiment 1. 13 is an image that illustrates an example of photographed data after a no-entry space has been created by a space designation section of the no-entry space designation section of the object detection device in the object detection system according to the first embodiment. 1 is a schematic image showing an example of photographed data after a no-entry space for the working hours of 23:00 to 1:29 is selected in the time management section of the no-entry space designation section of an object detection device in an object detection system according to embodiment 1. 1 is a schematic image showing an example of photographed data after a no-entry space for operation hours 1:30 to 3:29 is selected in the time management section of the no-entry space designation section of an object detection device in an object detection system according to embodiment 1. This is an image that shows a schematic example of photographed data after a detection line related to a triangular cone is created by the landmark recognition section of the no-entry space designation section of the object detection device in the object detection system of embodiment 1, and a candidate no-entry space is automatically created. This is an image that shows a schematic example of photographed data after a detection line related to a person is created by the landmark recognition section of the no-entry space designation section of the object detection device in the object detection system of embodiment 1, and a candidate no-entry space is automatically created. This is an image that shows a schematic example of photographed data after a marking recognition section of a no-entry space designation section of an object detection device in an object detection system of embodiment 1 creates a detection line related to an electric wire and automatically creates a no-entry space candidate. 13 is an image showing a schematic representation of another example of photographed data after a detection line relating to a triangular cone is created by the landmark recognition section of the no-entry space designation section of the object detection device in the object detection system of embodiment 1 to automatically create a no-entry space candidate. 4 is a flow chart diagram illustrating an example of an operation of the object detection device in the object detection system according to the first embodiment. FIG. FIG. 11 is a block diagram illustrating a schematic configuration of an object detection device in an object detection system according to a second embodiment. FIG. 11 is a block diagram illustrating a configuration of an object detection system according to a third embodiment. FIG. 11 is a block diagram illustrating a schematic configuration of an object detection device in an object detection system according to a third embodiment. FIG. 11 is a block diagram illustrating a configuration of an object detection system according to a fourth embodiment. FIG. 13 is a block diagram illustrating a schematic configuration of an object detection device in an object detection system according to a fourth embodiment. FIG. 13 is a block diagram illustrating a configuration of an object detection device according to a fifth embodiment. FIG. 2 is a block diagram illustrating a schematic configuration of hardware resources.

The following describes the embodiments with reference to the drawings. Note that when reference symbols are used in this application, they are intended to aid understanding and are not intended to limit the invention to the illustrated form. The following embodiments are merely illustrative and do not limit the invention. The connection lines between blocks in the drawings and the like referred to in the following description include both bidirectional and unidirectional lines. One-way arrows are used to diagrammatically indicate the flow of the main signal (data) and do not exclude bidirectionality. Furthermore, although not explicitly shown in the circuit diagrams, block diagrams, internal configuration diagrams, connection diagrams, and the like shown in this disclosure, input ports and output ports exist at the input and output ends of each connection line. The same is true for input/output interfaces. The program is executed via a computer device, which includes, for example, a processor, a storage device, an input device, a communication interface, and a display device as necessary, and the computer device is configured to be able to communicate with devices inside or outside the device (including a computer) via the communication interface, whether wired or wireless.

[Embodiment 1]
An object detection system according to embodiment 1 will be described with reference to the drawings. Fig. 1 is a block diagram showing a schematic configuration and one mode of use of the object detection system according to embodiment 1. Fig. 2 is a block diagram showing a schematic configuration of an object detection device in the object detection system according to embodiment 1.

The object detection system 1 is a system that detects a detection target object 11 that exists in a no-entry space 31 using a three-dimensional sensor 300 (see FIG. 1). In the object detection system 1, the three-dimensional sensor 300, the object detection device 200, and the mobile communication terminal 500 are connected to each other so as to be able to communicate with each other via a network 400. In the object detection system 1, the object detection device 200 selectively detects the detection target object 11 that exists in the no-entry space 31 based on the photographed data (100 in FIG. 2) photographed by the three-dimensional sensor 300. Here, the photographed data 100 is three-dimensional point cloud data created by photographing the three-dimensional sensor 300. The object detection system 1 has a function of recognizing a landmark (railroad tracks 14 in FIG. 1) and creating a no-entry space 31. For example, as illustrated in FIG. 3, the object detection system 1 has a function of displaying the detected detection target object 11 together with a segment 40a.

The three-dimensional sensor 300 is a sensor that senses the surface of an object (including the object to be detected 11, the object to be non-detected 12, the railroad tracks 14, the ground, etc. in FIG. 1) in the shooting space (20 in FIG. 4) and takes a three-dimensional image (see FIG. 1). The three-dimensional sensor 300 is communicably connected to the object detection device 200 and the mobile communication terminal 500 via the network 400. The three-dimensional sensor 300 takes an image of the shooting space 20 to create shooting data (100 in FIG. 2), and outputs the created shooting data 100 to the object detection device 200 and the mobile communication terminal 500. The shooting data 100 is point cloud data (three-dimensional point cloud data) drawn with a point cloud (three-dimensional coordinates of a large number of points). The three-dimensional sensor 300 can be selected according to the shooting distance, angle of view, indoors and outdoors, environmental conditions such as the presence or absence of sunlight, etc., required for detecting the object to be detected 11, and customer requests. In addition, the three-dimensional sensor 300 can be a product sold by various manufacturers. As the three-dimensional sensor 300, for example, a stereo camera, a ToF (Time of Flight) camera, a 3D-LIDAR (Three Dimensions - Laser Imaging Detection and Ranging) or other three-dimensional sensor can be used. The three-dimensional sensor 300 is installed outside the no-entry space 31 created by the object detection device 200 or outside the area of the construction site. This makes it possible to set multiple no-entry spaces, and it is also possible to change the installation location and utilize it. The three-dimensional sensor 300 is attached or installed at a position where it can photograph the shooting space 20, and if there is no suitable support (such as a wall or a pillar) for mounting or installing, it may be attached or installed using a tripod 301 as shown in FIG. 1. There may be at least one three-dimensional sensor 300 in the shooting space 20, but there may be multiple three-dimensional sensors in the shooting space 20. In addition, multiple three-dimensional sensors 300 may be used when the shooting space 20 is large and it is difficult to shoot with one sensor. In addition, when there are multiple three-dimensional sensors 300 in the shooting space 20, the types and manufacturers may be different. Furthermore, if there are multiple 3D sensors 300 in the shooting space 20, the shooting data captured by each 3D sensor 300 can be synthesized by the object detection device 200.

The photography space 20 is a space in which a no-entry space (31 in FIG. 1) can be set, and in which an object that has entered the no-entry space can be detected. The no-entry space (31 in FIG. 1) is a space in which the entry of objects is prohibited.

The network 400 is a wired or wireless communication network that connects the three-dimensional sensor 300, the object detection device 200, and the mobile communication terminal 500 so that they can communicate with each other. The network 400 may be, for example, a personal area network (PAN), a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), or a global area network (GAN).

The portable communication terminal 500 is a terminal capable of displaying information from the three-dimensional sensor 300 and the object detection device 200 (see FIG. 1). The portable communication terminal 500 is used by users such as site supervisors and workers. A portable communication terminal is used for the portable communication terminal 500, and may be, for example, a smartphone, a mobile phone, or a tablet terminal. The portable communication terminal 500 is communicatively connected to the object detection device 200 and the three-dimensional sensor 300 via the network 400. The portable communication terminal 500 can receive and display the photographing data (100 in FIG. 2) from the three-dimensional sensor 300. The portable communication terminal 500 can receive and display the results (110 in FIG. 2) and warning information (if received) from the object detection device 200.

The object detection device 200 is a device that creates a no-entry space 31 based on a predetermined mark in the shooting data 100 and selectively detects a detection target object 11 that has entered the created no-entry space 31 (see Figures 1 and 2). The object detection device 200 is used by a user such as a supervisor. The object detection device 200 can be a device (computer device) having functional units that constitute a computer (e.g., a processor, a storage device, an input device, a communication interface, and a display device). For the object detection device 200, for example, hardware resources such as a calculator, a personal computer, a notebook personal computer, a tablet terminal, etc. can be used. The object detection device 200 has a time function. The object detection device 200 realizes a pre-processing unit 210, a no-entry space designation unit 220, a detection processing unit 230, a result generation unit 240, and an interface unit 250 by executing a predetermined program (software).

The pre-processing unit 210 is a functional unit that performs predetermined pre-processing (here, format conversion and noise removal) on the shooting data 100 (see FIG. 2). The pre-processing unit 210 acquires the shooting data 100 from a sensor (300 in FIG. 1). The pre-processing unit 210 outputs the pre-processed shooting data 100 to the no-entry space designation unit 220, the detection processing unit 230, and the display unit 252 of the interface unit 250. The pre-processing unit 210 includes a format conversion unit 211 and a noise removal unit 212.

The format conversion unit 211 is a functional unit that converts the format of the shooting data 100, which differs depending on the type of sensor (300 in FIG. 1), into a common format that can be commonly used in the object detection device 200 (see FIG. 2). The format conversion unit 211 acquires the shooting data 100 from the sensor (300 in FIG. 1). The format conversion unit 211 outputs the shooting data 100 converted into the common format to the noise removal unit 212.

The noise removal unit 212 is a functional unit that removes noise (points unnecessary for detection) from the point cloud in the shooting data 100 (see FIG. 2). The noise removal unit 212 acquires the shooting data 100 from the format conversion unit 211. The noise removal unit 212 outputs the shooting data 100 from which noise has been removed to the no-entry space designation unit 220, the detection processing unit 230, and the display unit 252 of the interface unit 250. Examples of noise removal methods include smoothing processing, filtering (e.g., moving average filter processing, median filter processing, etc.), and outlier removal processing (e.g., outlier removal processing by chi-squared test).

The intrusion-prohibited space designation unit 220 is a functional unit that designates an intrusion-prohibited space (31a in FIG. 8, 31b in FIG. 9) in the shooting space (20 in FIG. 4) of the shooting data 100 (see FIG. 2). The intrusion-prohibited space designation unit 220 acquires the preprocessed shooting data 100 from the preprocessing unit 210. The intrusion-prohibited space designation unit 220 detects a mark (railroad tracks 14a, 14b in FIG. 5) corresponding to a shape related to a preset mark (for example, a pair of parallel lines) in the shooting space (20 in FIG. 4) of the shooting data 100, and creates a detection line (22a, 22b in FIG. 5) related to the mark. The intrusion-prohibited space designation unit 220 automatically creates one or more intrusion-prohibited space candidates (30a, 30b in FIG. 6) according to a preset form based on the created detection line (22a, 22b in FIG. 5). The intrusion-prohibited space designation unit 220 saves information related to the created intrusion-prohibited space candidate (30a, 30b in FIG. 6). The intrusion-prohibited space designation unit 220 acquires setting condition information (width, depth, height, angle, setting time) from the operation unit 251 of the interface unit 250. The intrusion-prohibited space designation unit 220 creates intrusion-prohibited spaces (31a, 31b in FIG. 7) by designating or changing the setting conditions of the created intrusion-prohibited space candidate (30a, 30b in FIG. 6) based on the setting condition information (here, width, depth, height, angle). The intrusion-prohibited space designation unit 220 saves information related to the created intrusion-prohibited space (31a, 31b in FIG. 7). In addition, when there is no setting condition information, the intrusion-prohibited space designation unit 220 can directly set the created intrusion-prohibited space candidate (30a, 30b in FIG. 6) as an intrusion-prohibited space. The no-entry space designation unit 220 outputs information related to the created no-entry space (31a in FIG. 8, 31b in FIG. 9) during the set time (work time) based on the set condition information (here, the set time) to the detection processing unit 230 and the result generation unit 240. The no-entry space designation unit 220 includes a mark recognition unit 221, a space designation unit 222, an angle designation unit 223, and a time management unit 224.

The mark recognition unit 221 is a functional unit that recognizes marks (railroad tracks 14a, 14b in FIG. 5) corresponding to a shape related to a preset mark (e.g., a pair of parallel lines in FIG. 5) from the shooting space (20 in FIG. 4) based on the preprocessed shooting data 100 from the preprocessing unit 210, and automatically creates a no-entry space candidate (30a, 30b in FIG. 6) (see FIG. 2). The mark recognition unit 221 detects marks (railroad tracks 14a, 14b in FIG. 5) corresponding to a shape related to a preset mark (e.g., a pair of parallel lines in FIG. 4) in the shooting space (20 in FIG. 4) based on the shooting data 100, and creates a detection line (22a, 22b in FIG. 5) related to the mark. The mark recognition unit 221 automatically creates one or more no-entry space candidates (30a, 30b in FIG. 6) according to a preset form based on the created detection line (22a, 22b in FIG. 5). This eliminates the complexity of setting the no-entry spaces, and makes it easier to customize the no-entry spaces. The mark recognition unit 221 saves information related to the created no-entry space candidates (30a, 30b in FIG. 6). The mark recognition unit 221 outputs information related to the created no-entry space candidates (30a, 30b in FIG. 6) to the space designation unit 222.

Here, as the markers, for example, continuous or intermittent structures and displays that exist as infrastructure, such as railroad tracks (14a, 14b in FIG. 4), electric wires (17a, 17b in FIG. 12), communication lines, lanes, pipelines, guardrails, utility poles, guideposts, snow poles, etc., can be used. In addition, as the markers, for example, triangular cones (see 15 in FIG. 10), people (see 16 in FIG. 11), poles, etc., that are installed later by humans, can be used. In addition, as the shape of the preset markers, for example, continuous or intermittent shapes (lines, shapes, etc.) that exist as infrastructure, such as a pair of parallel lines like railroad tracks, a catenary line like an electric wire, a broken or continuous line like a road lane, etc. can be used. In addition, as the shape of the preset markers, for example, characteristic shapes of markers, such as a triangular pyramid surface, a human-shaped surface, a cylindrical surface, etc., can be used. The preset form may be, for example, a box-shaped (rectangular) space of a given or arbitrary width, depth, and height that includes the created detection line (22a, 22b in FIG. 5, 23 in FIG. 10, 24 in FIG. 11, 25a, 25b in FIG. 12), or a box-shaped (rectangular) space of a given or arbitrary width, depth, and height that is a given distance away from the created detection line. Note that the preset form is not limited to a box-shaped (rectangular) space, and may be cylindrical, polygonal prism, spherical, etc.

The space designation unit 222 is a functional unit that designates a no-entry space (31a, 31b in FIG. 7) in the shooting data 100 (see FIG. 2). The space designation unit 222 acquires the shooting data 100 from the pre-processing unit 210, acquires information related to the no-entry space candidate (30a, 30b in FIG. 6) from the landmark recognition unit 221, and acquires setting condition information (width, depth, height, angle, setting time) from the operation unit 251 of the interface unit 250. The space designation unit 222 saves the width, depth, and height of the acquired setting condition information. The space designation unit 222 creates a no-entry space (31a, 31b in FIG. 7) by designating the setting conditions of the acquired no-entry space candidate (30a, 30b in FIG. 6) based on the width, depth, and height of the acquired setting condition information. If there is no setting condition information, the space designation unit 222 can directly designate the acquired candidate for intrusion-prohibited space (30a, 30b in FIG. 6) as the intrusion-prohibited space without creating a new intrusion-prohibited space. The space designation unit 222 saves information related to the created intrusion-prohibited space (31a, 31b in FIG. 7). The space designation unit 222 outputs information related to the created intrusion-prohibited space (31a, 31b in FIG. 7) to the angle designation unit 223.

The angle designation unit 223 is a functional unit that designates the angle of the no-entry space (31a, 31b in FIG. 7) for the shooting data 100 (see FIG. 2). The angle designation unit 223 acquires the shooting data 100 from the pre-processing unit 210, acquires information related to the no-entry space (31a, 31b in FIG. 7) from the space designation unit 222, and acquires setting condition information (width, depth, height, angle, setting time) from the operation unit 251 of the interface unit 250. The angle designation unit 223 saves the angle of the acquired setting condition information. The angle designation unit 223 changes the angle of the no-entry space 31 by rotating the point cloud of the shooting data 100 (or the no-entry spaces 31a, 31b from the space designation unit 222) based on the angle of the acquired setting condition information. If there is no setting condition information, the angle designation unit 223 does not change the angle of the no-entry space 31, and can keep the angle of the no-entry space (31a, 31b in FIG. 7) obtained from the space designation unit 222. The angle designation unit 223 saves information related to the no-entry space (31a, 31b in FIG. 7) whose angle has been changed. The angle designation unit 223 outputs information related to the no-entry space (31a, 31b in FIG. 7) whose angle has been changed to the time management unit 224.

The time management unit 224 is a functional unit that sets the time (e.g., work time: start time and end time) when each of the no-entry spaces (31a and 31b in FIG. 7) is effective (see FIG. 2). The time management unit 224 acquires setting condition information (width, depth, height, angle, set time) from the operation unit 251 of the interface unit 250. The time management unit 224 saves the set time from the acquired setting condition information. The time management unit 224 selects the no-entry space corresponding to the effective time based on the set time from the acquired setting condition information, and outputs information related to the selected no-entry space to the detection processing unit 230. For example, the time management unit 224 can select the no-entry space 31a from 23:00 to 1:29 in the work time as shown in FIG. 8 based on the set time and output information related to the no-entry space 31a to the detection processing unit 230, and select the no-entry space 31b from 1:30 to 3:29 in the work time as shown in FIG. 9 and output information related to the no-entry space 31b to the detection processing unit 230. This allows the no-entry spaces to be changed automatically in sequence. When the set time in the setting condition information is changed, the time management unit 224 changes the time period during which the no-entry spaces 31a and 31b are selected accordingly. This makes it possible to advance or delay the time during which the no-entry spaces are selected, or to change it, depending on the progress of the construction work.

The detection processing unit 230 is a functional unit that performs detection processing of a detection target object (11 in FIG. 1) that matches detection condition information based on a point cloud in the intrusion-prohibited space (31 in FIG. 3) in the shooting data 100 (see FIG. 2). The detection processing unit 230 acquires the shooting data 100 from the pre-processing unit 210, acquires information related to the intrusion-prohibited space 31 from the intrusion-prohibited space designation unit 220, and acquires detection condition information (e.g., information related to the height range, size range, volume range, and excluded area that is checked in chronological order to discover a moving object) from the operation unit 251 of the interface unit 250. The detection processing unit 230 may not only automatically execute the detection processing periodically, but may also manually execute it at any time by operating the operation unit 251 of the interface unit 250. The detection processing unit 230 outputs detection information (including detection conditions and information related to the detection processing) to the result generation unit 240. The detection processing unit 230 includes a condition designation unit 231 and a detection unit 232.

The condition specification unit 231 is a function for specifying detection conditions related to the detection target object 11 (see FIG. 2). The condition specification unit 231 acquires and stores detection condition information (e.g., height range, size range, volume range, confirmation in time series to discover moving objects, exclusion area, etc.) from the operation unit 251 of the interface unit 250. The condition specification unit 231 outputs the acquired detection conditions to the detection unit 232.

Here, examples of detection condition information include a condition to detect an object that is 90 cm or more in height and is not in the exclusion area, or a condition to detect an object that is moving and is not in the exclusion area.

The detection unit 232 is a functional unit that selectively detects a detection target object (11 in FIG. 1) that matches detection condition information (e.g., height range, size range, volume range, time series confirmation to find a moving object, exclusion area, etc.) based on a point cloud in a no-entry space (e.g., 31 in FIG. 3) in the shooting data 100. The detection unit 232 acquires shooting data 100 from the pre-processing unit 210, acquires information related to the no-entry space 31 from the no-entry space designation unit 220, and acquires detection condition information from the condition designation unit 231. In detecting the detection target object 11, for example, as shown in FIG. 3, first, the point cloud in the no-entry space 31 in the shooting data (100 in FIG. 2) is deleted. Next, one or more segments 40a are created that are separated according to the distance for each point cloud of objects (detection target object 11, non-detection target object 12) in the no-entry space 31. Next, based on the point cloud in the created one or more segments 40a, the detection target object 11 that matches the specified detection condition is detected. The detection unit 232 may label and store information related to the segment 40a of the detection target object 11 so that the detection target object 11 can be tracked. The detection unit 232 outputs the detection information (including the detection conditions and the segment 40a) to the result generation unit 240.

The result generation unit 240 is a functional unit that generates (or calculates) a result to be notified to the administrator based on information related to the no-entry space (31 in FIG. 3) and detection information (including detection conditions and segment (40a in FIG. 3)) (see FIG. 2). The result generation unit 240 acquires information related to the no-entry space 31 from the no-entry space designation unit 220, and acquires detection information (including detection conditions and segment 40a) from the detection processing unit 230. The generated result can be a figure of the no-entry space (31 in FIG. 3) and a figure or specifications of the detection information (for example, the height in the detection conditions, the excluded area, the figure or height of segment 40a, etc.). The result generation unit 240 outputs the generated result to the interface unit 250. The result generation unit 240 may determine whether or not a segment 40a (e.g., a labeled segment) of the detection target object 11 is present in the detection information, and if a segment 40a of the detection target object 11 is present, may output warning information together with the generated result to the interface unit 250. The result generation unit 240 includes a figure generation unit 241 and a specification calculation unit 242.

The figure generating unit 241 is a functional unit that generates figures based on information related to the no-entry space (31 in FIG. 3) and detection information (including detection conditions and segments (40a in FIG. 3)) (see FIG. 2). Examples of figures to be generated include figures of the no-entry space 31 and figures of the detection information (for example, figures of the excluded areas of the detection conditions, figures of segments 40a of the detection target object 11, etc.). The figures are generated taking into consideration the position. In addition, in generating figures, it is possible to omit the generation of some elements of the no-entry space 31 and the detection information. Note that the figure generating unit 241 can avoid generating figures of segments 40a of the non-detection target object (12 in FIG. 7) in order to clarify the difference between the detection target object (11 in FIG. 1) and the non-detection target object (12 in FIG. 1). The figure generating unit 241 outputs the generated figures to the display unit 252 and the communication unit 253 of the interface unit 250 as results.

The specification calculation unit 242 is a functional unit that calculates specifications (dimensions, volume, movement speed, etc.) based on the detection information (including the detection conditions and the segment (40a in FIG. 3)) (see FIG. 2). Examples of the calculated specifications include the specifications of the detection information (for example, the height of the height range of the detection conditions, the height of the segment 40a, etc.). The specification calculation unit 242 outputs the calculated specifications as results to the display unit 252 and communication unit 253 of the interface unit 250.

The interface unit 250 is a functional unit that exchanges information between the user and the object detection device 200 (see FIG. 2). The interface unit 250 has an operation unit 251, a display unit 252, and a communication unit 253.

The operation unit 251 is a functional unit that receives user operations (see FIG. 2). The user operates the operation unit 251 (for example, keyboard operation, mouse click operation, touch panel tap operation, etc.) to input data (for example, letters, numbers, positions, areas, etc.) and outputs the data to the intrusion-prohibited space designation unit 220 or the detection processing unit 230. The operation unit 251 outputs information related to the conditions (width, depth, height, angle, set time) input by the user's operation to the intrusion-prohibited space designation unit 220. The operation unit 251 outputs the detection conditions input by the user's operation (for example, height range, size range, volume range, checking in chronological order to find a moving object, excluded area, etc.) to the detection processing unit 230. The operation unit 251 instructs the start and end of the entire processing in the function of the object detection device 200 by the user's operation.

The display unit 252 is a functional unit that displays various information (see FIG. 2). The display unit 252 displays the preprocessed shooting data 100 from the noise removal unit 212 of the preprocessing unit 210. The display unit 252 displays the results from the result generation unit 240 and warning information (if acquired). The warning information can be displayed, for example, by changing the background color or by blinking. The displayed shooting data 100 and results may be saved as screenshots, numerical data, or the like. The information displayed on the display unit 252 may be output as audio using a speaker, printed on a printer, or transmitted and output to another terminal.

The communication unit 253 is a functional unit that transmits and receives various information (see FIG. 2). The communication unit 253 transmits the result 110 from the result generation unit 240 and warning information (if acquired) to the mobile communication terminal (500 in FIG. 1) via the network (400 in FIG. 1).

Next, an example of designating a no-entry space using an object detection device in the object detection system according to embodiment 1 will be described with reference to the drawings.

Here, Figs. 4 to 9 show the procedure for specifying a no-entry space when railroad tracks are used as a landmark. Fig. 10 shows an example of automatic creation of no-entry space candidates when a triangular cone is used as a landmark. Fig. 11 shows an example of automatic creation of no-entry space candidates when a person is used as a landmark. Fig. 12 shows an example of automatic creation of no-entry space candidates when an electric wire is used as a landmark. Fig. 13 shows an example of automatic creation of no-entry space candidates at a position a specified distance away from the detection line. Please refer to Figs. 1 and 2 for the components of the object detection device. Also, the object detection device 200 performs processing by the monitor operating the operation unit 251 of the interface unit 250.

Figure 4 shows an image of the photographed data 100 after preprocessing by the preprocessing unit 210 of the object detection device 200. An undetected object 12 is present in the photographed space 20 of the photographed data 100.

Figure 5 shows an image of the shooting data 100 after the mark recognition unit 221 of the no-entry space designation unit 220 of the object detection device 200 detects marks (railroad tracks 14a, 14b) that correspond to the shape of a pre-set mark (here, a pair of parallel lines) and creates detection lines 22a, 22b related to the marks.

Figure 6 shows an image of the photographed data 100 after the landmark recognition unit 221 of the no-entry space designation unit 220 of the object detection device 200 automatically creates corresponding no-entry space candidates 30a, 30b so as to encompass the detection lines 22a, 22b according to a preset form based on the created detection lines 22a, 22b.

Figure 7 shows an image of the shooting data 100 after the space designation section 222 of the no-entry space designation unit 220 of the object detection device 200 has specified conditions for the no-entry space candidates (30a, 30b in Figure 6) based on the width, depth and height of the set condition information (coordinates ( _x10 , _y10 , _z10 ) and its opposite coordinates ( _x11 , _y11 , _z11 ) for the no-entry space candidate 30a, and coordinates ( _x20 , _y20 , _z20 ) and its opposite coordinates ( _x21 , _y21 , _z21 ) for the no-entry space candidate 30b) to create the no-entry spaces 31a, 31b.

Figure 8 shows an image of the photographed data 100 after the time management unit 224 of the no-entry space designation unit 220 of the object detection device 200 has selected a no-entry space 31a for the working hours of 23:00 to 1:29 based on the set time in the set condition information.

Figure 9 shows an image of the photographed data 100 after the time management unit 224 of the no-entry space designation unit 220 of the object detection device 200 has selected the no-entry space 31b for the working hours of 1:30 to 3:29 based on the set time in the set condition information.

For example, in railway construction, such as quadruple or triple track construction, the tracks being constructed shift depending on the time of day according to the construction plan, and the no-entry spaces change accordingly. However, manually performing the setting work associated with switching no-entry spaces each time is extremely time-consuming. Therefore, by automatically switching no-entry spaces that have been automatically created as shown in Figures 8 and 9, the setting work associated with switching no-entry spaces is unnecessary, and it is possible to automatically shift the created no-entry spaces according to the work hours.

Figure 10 shows an image of the photographed data 100 after the mark recognition unit 221 of the no-entry space designation unit 220 of the object detection device 200 detects a mark (a triangular cone 15) corresponding to a shape (here, a triangular vertical surface) related to a preset mark, creates a detection line 23 related to the mark, and automatically creates one or more no-entry space candidates 32 so as to include the detection line 23 according to a preset form based on the created detection line 23. The subsequent creation and selection of no-entry spaces are similar to those in Figures 7 to 9. The mark, the triangular cone 15, has the characteristics of being red/plastic, and when the triangular cone 15 is photographed by the three-dimensional sensor 300, reflection intensity information can be obtained along with distance information. Therefore, since the triangular cone 15 can obtain reflection intensity information with a reflection intensity different from the surrounding point cloud information (size/shape), it is possible to automatically extract the triangular cone 15 as a mark from the photographed photographed data 100.

Figure 11 shows an image of the captured image data 100 after the mark recognition unit 221 of the no-entry space designation unit 220 of the object detection device 200 detects a mark (person 16) that corresponds to the shape of a preset mark (here, a human-shaped face), creates a detection line 24 related to the mark, and automatically creates a no-entry space candidate 33 based on the created detection line 24 in a preset form so as to encompass the detection line 24. The subsequent creation and selection of no-entry spaces are similar to those in Figures 7 to 9.

For example, the 3D sensor 300 can capture landmarks such as railroad tracks and power lines in close-up photography, but as the photography distance increases, the captured point cloud data becomes coarse and it may not be possible to clearly capture the landmark information. In such cases, landmarks such as cones and people can be used as shown in Figures 10 and 11, and objects reflected in the point cloud data can be added as landmarks and used as a supplementary method for setting no-entry spaces. Note that after creating a no-entry space candidate, even if the landmarks (cones and people) disappear, there is no problem because the no-entry space candidate is stored in the software. Also, although there are four cones 15 and people 16 that serve as landmarks in Figures 10 and 11, three or more landmarks are sufficient.

Figure 12 shows an image of the captured image data 100 after the mark recognition unit 221 of the no-entry space designation unit 220 of the object detection device 200 detects marks (electric wires 17a, 17b) that correspond to a shape related to a preset mark (here, a catenary line), creates detection lines 25a, 25b related to the marks, and automatically creates corresponding no-entry space candidates 34a, 34b based on the created detection lines 25a, 25b in a preset form so as to include the detection lines 25a, 25b. The subsequent creation and selection of no-entry spaces is the same as in Figures 7 to 9.

Figure 13 shows an image of the captured image data 100 after the mark recognition unit 221 of the no-entry space designation unit 220 of the object detection device 200 detects a mark (a triangular cone 18; other marks are also acceptable) corresponding to a shape related to a preset mark (here, a triangular pyramid surface), creates a detection line 26 related to the mark, and automatically creates no-entry space candidates 35a, 35b based on the created detection line 26 so as to be a predetermined distance (height) away from the detection line 26 according to a preset form. The creation and selection of no-entry spaces thereafter are similar to those in Figures 7 to 9. For example, the height from the ground of electric wires is often determined depending on the type of wire, such as low-voltage wire or high-voltage wire, so in Figure 13, no-entry space candidates 35a, 35b are automatically created at a height according to the type of wire based on the mark on the ground.

Next, the operation of the object detection device in the object detection system according to the first embodiment will be described with reference to the drawings. FIG. 14 is a flow chart diagram that shows a schematic of the operation of the object detection device in the object detection system according to the first embodiment. Please refer to FIG. 1 and FIG. 2 for the configurations of the object detection system and the object detection device.

First, the pre-processing unit 210 of the object detection device 200 acquires the image data 100 captured by the 3D sensor 300 (step A1).

Next, the format conversion unit 211 of the pre-processing unit 210 converts the format of the acquired shooting data 100 into a common format (step A2).

Next, the noise removal unit 212 of the pre-processing unit 210 removes noise from the shooting data 100 that has been converted to the common format (step A3).

Next, the landmark recognition unit 221 of the no-entry space designation unit 220 detects a landmark (railroad tracks 14a, 14b in FIG. 5) that corresponds to a shape related to a pre-set landmark (e.g., a pair of parallel lines in FIG. 4) in the shooting space (20 in FIG. 4) of the noise-removed shooting data 100 (step A4).

Next, the landmark recognition unit 221 of the no-entry space designation unit 220 creates detection lines (22a, 22b in FIG. 5) related to the detected landmarks (railroad tracks 14a, 14b in FIG. 5) (step A5).

Next, the landmark recognition unit 221 of the no-entry space designation unit 220 automatically creates one or more no-entry space candidates (30a, 30b in FIG. 6) according to a preset form based on the created detection lines (22a, 22b in FIG. 5) (step A6).

Next, the intrusion-prohibited space designation unit 220 acquires setting condition information (width, depth, height, angle, and set time) from the operation unit 251 of the interface unit 250 (step A7). If there is no setting condition information, step A7 is skipped.

Then, the space designation unit 222 of the intrusion-prohibited space designation unit 220 creates intrusion-prohibited spaces (31a, 31b in FIG. 7) by designating the setting conditions of the created intrusion-prohibited space candidate (30a, 30b in FIG. 6) based on the width, depth, and height of the acquired setting condition information (step A8). Note that if the setting condition information does not include the width, depth, and height, step A8 is skipped.

Next, the angle designation unit 223 of the no-entry space designation unit 220 changes the angle of the no-entry spaces (31a, 31b in FIG. 7) by rotating the point cloud of the shooting data 100 (or the no-entry spaces 31a, 31b from the space designation unit 222) based on the angle in the acquired setting condition information (step A9). Note that if there is no angle in the setting condition information, step A9 is skipped.

Next, the time management unit 224 of the no-entry space designation unit 220 acquires the setting condition information (width, depth, height, angle, set time) from the operation unit 251 of the interface unit 250, and selects the no-entry space (31a in FIG. 8) that corresponds to the effective time (23:00 to 1:29 in FIG. 8) based on the set time in the acquired setting condition information (step A10). If there is no set time in the setting condition information, it is possible to manually select either the no-entry space candidate (30a, 30b in FIG. 6) or the no-entry space (31a, 31b in FIG. 7), select all of the no-entry space candidates (30a, 30b in FIG. 6) created in step A6 as no-entry spaces, or select all of the no-entry spaces (31a, 31b in FIG. 7) whose angles have been changed in step A9.

Next, the detection processing unit 230 of the object detection device 200 acquires detection condition information (e.g., height range, size range, volume range, time series confirmation to discover moving objects, exclusion area, etc.) from the operation unit 251 of the interface unit 250 (step A11). If there is no detection condition information, step A11 is skipped.

Next, the detection processing unit 230 of the object detection device 200 detects the detection target object 11 that matches the acquired detection condition information (e.g., height range, size range, volume range, time series confirmation to find a moving object, exclusion area, etc.) based on the point cloud in the intrusion prohibition space (31a in FIG. 8) in the shooting data 100 using the noise-removed shooting data 100 and the selected intrusion prohibition space (31a in FIG. 8) (step A12). In addition, if there is no detection condition information, it is possible to use the initial detection condition information that has been set in advance or the detection condition information acquired last time. In addition, in the detection of the detection target object 11, the point cloud of the intrusion prohibition space 31 is deleted from the point cloud in the intrusion prohibition space 31 in the shooting data 100, and a segment 40a is created that is separated by the distance for each point cloud of the object (detection target object 11, non-detection target object 12) on the intrusion prohibition space 31, and the detection target object 11 that matches the specified detection condition is detected based on the point cloud in each of the created segments 40a. If a detection target object 11 is detected, the segment 40a of the detection target object 11 is labeled. If a detection target object 11 is not detected in step A12, the process proceeds to step A13.

Next, the graphic generator 241 of the result generator 240 of the object detection device 200 uses the selected no-entry space (31a in FIG. 8) and the acquired detection information (particularly, the segment (40a in FIG. 3)) to generate the no-entry space (31a in FIG. 8) and a graphic of the detection information (step A13). Note that in generating the graphic, the generation of the no-entry space (31a in FIG. 8) and some elements of the detection information can be omitted. In addition, when generating the graphic, it is determined whether or not there is a segment 40a (with a label) of the object to be detected 11 in the detection information, and if there is a segment 40a of the object to be detected 11, warning information is generated. In addition, in generating the graphic, if there is no change in the no-entry space (31a in FIG. 8) and the detection information when generating a new graphic after generating the graphic, the previously generated graphic can be maintained and step A13 can be skipped.

Next, the specification calculation unit 242 of the result generation unit 240 calculates the specifications (dimensions, volume, movement speed, etc.) based on the detection information (detection conditions, segment 40a) (step A14). Note that, in the calculation of the specifications, if there is no change in the detection information when the specifications are newly calculated after the specifications are calculated, step A14 can be skipped while maintaining the previously calculated specifications.

Finally, the display unit 252 of the interface unit 250 of the object detection device 200 displays the noise-removed shooting data 100, the generated graphic, and the calculated specifications (step A15). Furthermore, if warning information has been generated, the display unit 252 displays the warning information. When displaying, the communication unit 253 of the interface unit 250 transmits the displayed content (result) to the mobile communication terminal 500. This allows the mobile communication terminal 500 to display the same content as that displayed on the display unit 252. Thereafter, the flow ends, and the next flow is executed.

According to the first embodiment, new no-entry space candidates are automatically created by recognizing specific landmarks in the image data 100, which contributes to efficient work of setting new no-entry spaces.

[Embodiment 2]
The object detection system according to the second embodiment will be described with reference to the drawings. Fig. 15 is a block diagram showing a schematic configuration of an object detection device in the object detection system according to the second embodiment.

Embodiment 2 is a modified example of embodiment 1, in which the selection of the entry-prohibited area can be switched by gesture instead of time. In the object detection device 200 in the object detection system according to embodiment 2, a gesture recognition unit 260 is newly added, and in the entry-prohibited space designation unit 220, a gesture instruction management unit 225 is provided instead of the time management unit (224 in FIG. 2) of embodiment 1. Note that in the entry-prohibited space designation unit 220, the time management unit (224 in FIG. 2) and the gesture instruction management unit 225 may be configured in parallel.

The gesture recognition unit 260 is a functional unit that recognizes a gesture (e.g., a predetermined gesture or hand movement) of a person (supervisor) in the shooting data 100 and outputs selection instruction information (an instruction to select a no-entry space or an instruction to change the selection) corresponding to the recognized gesture (see FIG. 15). The gesture recognition unit 260 acquires the shooting data 100 preprocessed by the preprocessing unit 210. The gesture recognition unit 260 outputs the selection instruction information corresponding to the recognized gesture to the no-entry space designation unit 220. The gesture recognition unit 260 includes a gesture detection unit 261 and an instruction unit 262.

The gesture detection unit 261 is a functional unit that detects the gestures of the supervisor in the shooting data 100 (see FIG. 15). The gesture detection unit 261 recognizes the gestures of the supervisor in the shooting data 100 as a three-dimensional shape, and outputs information related to the recognized three-dimensional shape to the instruction unit 262.

The instruction unit 262 is a functional unit that outputs selection instruction information related to a selection instruction or selection change instruction for a no-entry space that corresponds to the information related to the three-dimensional shape from the gesture detection unit 261 (see FIG. 15). The instruction unit 262 outputs the selection instruction information related to a selection instruction or selection change instruction for a no-entry space to the gesture instruction management unit 225 of the no-entry space designation unit 220.

The gesture instruction management unit 225 is a functional unit that performs an initial selection or a selection change of a no-entry space (31a, 31b in FIG. 7) based on selection instruction information from the instruction unit 262 of the gesture recognition unit 260 (see FIG. 15). The gesture instruction management unit 225 acquires selection instruction information from the instruction unit 262 of the gesture recognition unit 260. The gesture instruction management unit 225 stores the acquired selection instruction information. The gesture instruction management unit 225 performs an initial selection or a selection change of a no-entry space (31a, 31b in FIG. 7) whose angle has been changed by the angle designation unit 223 based on the acquired selection instruction information. When there is no selection instruction information, the gesture instruction management unit 225 can manually select one of the no-entry spaces (31a, 31b in FIG. 7) or can select all of the no-entry spaces whose angles have been changed (31a, 31b in FIG. 7), or can select one of the no-entry spaces (31a, 31b in FIG. 7) according to time using the time management unit (224 in FIG. 2) of embodiment 1. The gesture instruction management unit 225 outputs information related to the selected no-entry space to the detection processing unit 230 and the result generation unit 240.

Other configurations and operations in embodiment 2 are similar to those of the object detection device in the object detection system of embodiment 1.

According to the second embodiment, as in the first embodiment, new no-entry space candidates are automatically created by recognizing specific landmarks in the shooting data 100, which contributes to efficient work of setting new no-entry spaces and allows no-entry spaces to be selected according to the gestures of the site supervisor who is familiar with the situation at the site.

[Embodiment 3]
An object detection system according to embodiment 3 will be described with reference to the drawings. Fig. 16 is a block diagram showing a schematic configuration of the object detection system according to embodiment 3. Fig. 17 is a block diagram showing a schematic configuration of an object detection device in the object detection system according to embodiment 3.

Embodiment 3 is a modification of embodiment 1, and adds a function of adding position identification information to each automatically created no-entry space candidate. Embodiment 3 can be applied to embodiment 2.

The object detection system 1 according to the third embodiment further includes a map information providing server 600 that provides map information to the object detection device 200 in response to a request from the object detection device 200.

In addition, in the object detection system 1 according to the third embodiment, the three-dimensional sensor 300 includes a three-dimensional sensor unit 310 that senses and captures the surfaces of objects (including the detection target object 11, non-detection target object 12, railroad tracks 14, ground, etc. in FIG. 1) within the capture space (20 in FIG. 4), as well as a position detection unit 320 that detects the position of the three-dimensional sensor 300 itself, and an orientation detection unit 330 that detects the line of sight orientation of the three-dimensional sensor unit 310 (see FIG. 16).

The sensor unit 310 photographs the photography space 20 to create photography data (100 in FIG. 2), and outputs the created photography data 100 to the object detection device 200 and the mobile communication terminal 500 (see FIGS. 16 and 17). A GPS (Global Positioning System) receiver can be used for the position detection unit 320. The position detection unit 320 outputs the detected position data 120 of the three-dimensional sensor 300 itself to the object detection device 200. The orientation detection unit 330 outputs the detected line-of-sight orientation data 130 of the three-dimensional sensor unit 310 to the object detection device 200.

In addition, in the object detection system 1 according to the third embodiment, a position identification unit 226 is added to the no-entry space designation unit 220 of the object detection device 200, which adds position identification information to each automatically created no-entry space candidate. The position identification unit 226 acquires the shooting data 100 from the pre-processing unit 210, acquires information related to the no-entry space candidates (30a, 30b in FIG. 6) from the landmark recognition unit 221, and acquires the position data 120 and the orientation data 130 from the three-dimensional sensor 300. Based on the acquired position data 120, the position identification unit 226 acquires map information 140 of a predetermined range of the position related to the position data 120 from the map information providing server 600. The position identification unit 226 identifies the position of each entry-prohibited space candidate (30a, 30b in FIG. 6) based on the acquired shooting data 100, position data 120, orientation data 130, and map information 140, extracts position identification information related to the identified positions from the map information 140, and adds (associates, links) the extracted position identification information to the entry-prohibited space candidate. The position identification unit 226 outputs information related to the entry-prohibited space candidate with the position identification information added to it to the space designation unit 222. The space designation unit 222 creates an entry-prohibited space using the information related to the entry-prohibited space candidate (with the position identification information added) from the position identification unit 226. This makes it easier to identify the position of the entry-prohibited space.

When the position of the candidate for intrusion-prohibited space is specified by the position specifying unit 226, the position of the candidate for intrusion-prohibited space on the map information 140 can be specified based on the position data 120 and the orientation data 130 of the 3D sensor 300 and the point cloud within the candidate for intrusion-prohibited space in the photographic data 100. When the position specifying unit 226 acquires the position specifying information, the position specifying information relating to the candidate for intrusion-prohibited space (for example, for railways near Shinjuku, line information such as the Yamanote Line, Chuo Line, Sobu Line, and Saikyo Line) can be acquired from the map information 140 based on the position of the identified candidate for intrusion-prohibited space.

Other configurations and operations in embodiment 3 are similar to those of the object detection device in the object detection system of embodiment 1.

According to the third embodiment, as in the first embodiment, new no-entry space candidates are automatically created by recognizing specific landmarks in the shooting data 100, which contributes to efficient work of setting new no-entry spaces and makes it easier to identify the location of no-entry spaces.

[Embodiment 4]
An object detection system according to embodiment 4 will be described with reference to the drawings. Fig. 18 is a block diagram showing a schematic configuration of the object detection system according to embodiment 4. Fig. 19 is a block diagram showing a schematic configuration of an object detection device in the object detection system according to embodiment 4.

The fourth embodiment is a modification of the first embodiment, in which an RGB (Red Green Blue) camera 700 is provided together with the three-dimensional sensor 300 in the object detection system 1 in order to make it easier to check the three-dimensional point cloud data obtained by photographing the landmark (see FIG. 18). The fourth embodiment can also be applied to the second and third embodiments.

The RGB camera 700 is a camera that senses the colors of objects in the shooting space (20 in FIG. 4) and captures them in two dimensions (see FIG. 18). The RGB camera 700 captures the shooting space (20 in FIG. 4) captured by the 3D sensor 300 so as to include landmarks (14a, 14b in FIG. 4). The RGB camera 700 captures the shooting space (20 in FIG. 4) and creates RGB data (150 in FIG. 19), and outputs the created RGB data 150 to the landmark recognition unit 221 of the no-entry space designation unit 220 of the object detection device 200.

The landmark recognition unit 221 detects a landmark (railroad tracks 14a, 14b in FIG. 5) that corresponds to a shape related to a preset landmark (e.g., a pair of parallel lines and the color of tracks 14a, 14b in FIG. 4) in the shooting space (20 in FIG. 4) based on the shooting data 100 and the RGB data 150.

Other configurations and operations in embodiment 4 are similar to those of the object detection device in the object detection system of embodiment 1.

According to the fourth embodiment, as in the first embodiment, predetermined landmarks in the photographed data 100 are recognized and new candidates for no-entry spaces are automatically created, which contributes to efficient work of setting new no-entry spaces, and by adding RGB data 150 of colors that are actually visible to the naked eye to the photographed data 100 (three-dimensional point cloud data) and performing sensor fusion (combining multiple sensors with different detection principles to improve recognition), visibility is improved and recognition of landmarks for setting no-entry spaces can be improved.

[Embodiment 5]
An object detection device according to the fifth embodiment will be described with reference to the drawings. Fig. 20 is a block diagram showing a schematic configuration of the object detection device according to the fifth embodiment.

The object detection device 200 is a device that detects a specific object in the shooting data 100. The object detection device 200 includes a no-entry space designation unit 220 and a detection processing unit 230.

The no-entry space designation unit 220 detects a mark in the image capture data 100 that corresponds to a form related to a preset mark. The no-entry space designation unit 220 creates one or more no-entry spaces according to a preset form based on the detected mark. The no-entry space designation unit 220 selects one of the created no-entry spaces.

The detection processing unit 230 detects a detection target object that matches the previously or newly set detection condition information based on the point cloud within the selected no-entry space in the shooting data 100.

According to the fifth embodiment, a new no-entry space is created by recognizing a specific landmark in the image data 100, which contributes to efficient work of setting up a new no-entry space.

The object detection device according to the first to fifth embodiments can be configured with so-called hardware resources (information processing device, computer), and a device having the configuration shown in FIG. 21 can be used. For example, the hardware resource 1000 includes a processor 1001, a memory 1002, a network interface 1003, and the like, which are connected to each other by an internal bus 1004.

Note that the configuration shown in FIG. 21 is not intended to limit the hardware configuration of the hardware resource 1000. The hardware resource 1000 may include hardware (e.g., an input/output interface) that is not shown. Furthermore, the number of units such as the processor 1001 included in the device is not intended to be limited to the example shown in FIG. 21, and for example, multiple processors 1001 may be included in the hardware resource 1000. For example, a CPU (Central Processing Unit), an MPU (Micro Processor Unit), a GPU (Graphics Processing Unit), etc. may be used as the processor 1001.

The memory 1002 may be, for example, a random access memory (RAM), a read only memory (ROM), a hard disk drive (HDD), or a solid state drive (SSD).

The network interface 1003 may be, for example, a LAN (Local Area Network) card, a network adapter, a network interface card, etc.

The functions of the hardware resource 1000 are realized by the above-mentioned processing module. The processing module is realized, for example, by the processor 1001 executing a program stored in the memory 1002. The program can be downloaded via a network or updated using a storage medium on which the program is stored. Furthermore, the processing module may be realized by a semiconductor chip. In other words, it is sufficient that the functions performed by the processing module can be realized by the execution of software on some kind of hardware.

A part or all of the above embodiments may be described as follows, but is not limited to the following:

[Appendix 1]
an entry-prohibited area designation unit configured to detect a mark corresponding to a form related to a preset mark in the photographed data, create one or more entry-prohibited spaces in accordance with a preset form based on the detected mark, and select one of the created entry-prohibited spaces;
A detection processing unit configured to detect a detection target object that matches previously or newly set detection condition information based on a point cloud in the selected no-entry space in the photographed data;
An object detection device comprising:
[Appendix 2]
Further comprising an operation unit for receiving an operation by a user,
The intrusion-prohibited area designation unit is
a mark recognition unit configured to detect a mark corresponding to a form related to the preset mark in the photographed data, and to automatically create at least one entry-prohibited space candidate in accordance with the preset form based on the detected mark;
a space designation unit configured to designate a setting condition for the at least one entry-prohibited space candidate based on a width, a depth, and a height included in setting condition information input from the operation unit, and to create at least one entry-prohibited space;
A management unit configured to select a no-entry space from the at least one no-entry space based on a predetermined criterion; and
2. The object detection device of claim 1, comprising:
[Appendix 3]
The object detection device described in Appendix 2, wherein the landmark recognition unit is configured to detect a landmark in the photographic data that corresponds to a form related to the predetermined landmark, create a detection line related to the detected landmark, and automatically create the at least one no-entry space candidate based on the created detection line in accordance with the predetermined form.
[Appendix 4]
An object detection device as described in Appendix 2 or 3, wherein the management unit is configured to select one of the at least one no-entry spaces based on selection information input from the operation unit as the specified criterion.
[Appendix 5]
An object detection device described in any one of Appendix 2 to 4, wherein the management unit is a time management unit configured to select, when the setting condition information includes a set time as the specified criterion, from the at least one no-entry space, one of the no-entry spaces that corresponds to the set time based on the set time.
[Appendix 6]
A gesture recognition unit configured to recognize a gesture of a person in the photographed data and output selection instruction information corresponding to the recognized gesture,
An object detection device described in any one of Appendix 2 to 5, wherein the management unit is a gesture instruction management unit configured to select, from the at least one no-entry space, one of the no-entry spaces that corresponds to the selection instruction information based on the selection instruction information as the specified criterion.
[Appendix 7]
The object detection device of any one of Appendix 2 to 6, wherein the no-entry area designation unit further includes a position identification unit configured to acquire position data and orientation data of a sensor that captures the shooting data, acquire map information of a predetermined range of a position related to the position data from a map information providing server based on the acquired position data, identify a position for each of the no-entry space candidates based on the shooting data, the position data, the orientation data, and the map information, extract position identification information related to the identified positions from the map information, add the extracted position identification information to the corresponding no-entry space candidate, and output information related to the no-entry space candidate with the position identification information added to the space designation unit.
[Appendix 8]
The object detection device described in any one of Appendix 2 to 7, wherein the landmark recognition unit is configured to acquire RGB data from an RGB camera that captures images so as to include the landmark in the shooting data, and to detect a landmark in the shooting data that corresponds to a shape related to the predetermined landmark using the RGB data.
[Appendix 9]
An object detection device according to any one of appendices 1 to 8, wherein the predetermined landmark has a continuous or intermittent shape existing as infrastructure.
[Appendix 10]
A sensor configured to output imaging data obtained by imaging a predetermined space;
An object detection device according to any one of claims 1 to 9;
An object detection system comprising:
[Appendix 11]
An object detection method for detecting an object using hardware resources, comprising:
detecting a mark corresponding to a form related to a preset mark in the photographed data, creating one or more no-entry spaces according to a preset form based on the detected mark, and selecting one of the created no-entry spaces;
detecting a detection target object that matches previously or newly set detection condition information based on a point cloud within the selected no-entry space in the photographed data;
The object detection method includes:
[Appendix 12]
A program for causing a hardware resource to execute a process for detecting an object,
A process of detecting a mark corresponding to a form related to a preset mark in the photographed data, creating one or more entry-prohibited spaces according to a preset form based on the detected mark, and selecting one of the created entry-prohibited spaces;
A process of detecting a detection target object that matches previously or newly set detection condition information based on a point cloud within the selected no-entry space in the photographed data;
A program for causing the hardware resource to execute the above.

The disclosures of the above patent documents are incorporated herein by reference and may be used as the basis or part of the present invention as necessary. Within the framework of the entire disclosure of the present invention (including the claims and drawings), and further based on the basic technical ideas, modifications and adjustments of the embodiments and examples are possible. Furthermore, within the framework of the entire disclosure of the present invention, various combinations or selections (or non-selection as necessary) of various disclosed elements (including each element of each claim, each element of each embodiment or example, each element of each drawing, etc.) are possible. In other words, the present invention naturally includes various modifications and corrections that a person skilled in the art would be able to make in accordance with the entire disclosure, including the claims and drawings, and the technical ideas. Furthermore, with regard to the numerical values and numerical ranges described in this application, any intermediate value, lower numerical value, and small range are considered to be described even if not specified. Furthermore, the disclosures of the above cited documents may be used in part or in whole in combination with the descriptions in this document as part of the disclosure of the present invention as necessary, in accordance with the spirit of the present invention, and are considered to be included (belong) to the disclosures of this application.

REFERENCE SIGNS LIST 1 object detection system 11 detection target object 12 non-detection target object 13 train 14, 14a, 14b railroad tracks (markers)
15, 18 Triangular cones (landmarks)
16 people (mark)
17a, 17b Electric wire (marker)
20 Shooting space 22a, 22b, 23, 24, 25a, 25b, 26 Detection line 30, 30a, 30b, 32, 33 Intrusion-prohibited space candidate 31, 31a, 31b Intrusion-prohibited space 34a, 34b, 35a, 35b Intrusion-prohibited space candidate 40a Segment 100 Shooting data 110 Result 120 Position data 130 Orientation data 140 Map information 150 RGB data 200 Object detection device 210 Preprocessing unit 211 Format conversion unit 212 Noise removal unit 220 Intrusion-prohibited space designation unit 221 Landmark recognition unit 222 Space designation unit 223 Angle designation unit 224 Time management unit (management unit)
225 Gesture instruction management unit (management unit)
226 Position identification unit 230 Detection processing unit 231 Condition specification unit 232 Detection unit 240 Result generation unit 241 Figure generation unit 242 Specifications calculation unit 250 Interface unit 251 Operation unit 252 Display unit 253 Communication unit 260 Gesture recognition unit 261 Gesture detection unit 262 Instruction unit 300 Three-dimensional sensor 301 Tripod 310 Three-dimensional sensor unit 320 Position detection unit 330 Orientation detection unit 400 Network 500 Portable communication terminal 600 Map information providing server 700 RGB camera 1000 Hardware resources 1001 Processor 1002 Memory 1003 Network interface 1004 Internal bus

Claims (12)

an entry-prohibited space designation unit configured to detect a mark corresponding to a form related to a preset mark in the photographed data, create a plurality of entry-prohibited spaces according to a preset form based on the detected mark, and select one of the plurality of created entry-prohibited spaces ;
A detection processing unit configured to detect a detection target object that matches previously or newly set detection condition information based on a point cloud in the selected no-entry space in the photographed data;
An object detection device comprising: Further comprising an operation unit for receiving an operation by a user,
The no-entry space designation unit is
a mark recognition unit configured to detect a mark corresponding to a form related to the preset mark in the photographed data, and to automatically create a plurality of entry-prohibited space candidates according to the preset form based on the detected mark;
a space designation unit configured to designate setting conditions for the plurality of entry-prohibited space candidates based on width, depth, and height included in setting condition information input from the operation unit, and to create a plurality of entry-prohibited spaces;
A management unit configured to select any one of the plurality of no-entry spaces based on a predetermined criterion; and
The object detection device according to claim 1 . The object detection device according to claim 2, wherein the landmark recognition unit is configured to detect a landmark corresponding to a form related to the preset landmark in the photographic data, create a detection line related to the detected landmark, and automatically create the multiple no-entry space candidates according to the preset form based on the created detection line. The object detection device according to claim 2 or 3, wherein the management unit is configured to select one of the multiple no-entry spaces based on selection information input from the operation unit as the predetermined criterion. The object detection device according to any one of claims 2 to 4, wherein the management unit is a time management unit configured to select, when the setting condition information includes a set time as the predetermined criterion, from among the multiple no-entry spaces, one of the no-entry spaces that corresponds to the set time based on the set time. A gesture recognition unit configured to recognize a gesture of a person in the photographed data and output selection instruction information corresponding to the recognized gesture,
The object detection device according to any one of claims 2 to 5, wherein the management unit is a gesture instruction management unit configured to select, from among the plurality of no-entry spaces, one of the no-entry spaces that corresponds to the selection instruction information based on the selection instruction information as the predetermined criterion. 7. The object detection device according to claim 2, further comprising a position identification unit configured to acquire position data and orientation data of a three-dimensional sensor that captures the shooting data, acquire map information of a predetermined range of a position related to the position data from a map information providing server based on the acquired position data, identify a position for each of the candidate no-entry spaces based on the shooting data, the position data, the orientation data, and the map information, extract position identification information related to the identified positions from the map information, add the extracted position identification information to the corresponding candidate no-entry space, and output information related to the candidate no-entry space with the position identification information added to it to the space designation unit. The object detection device according to any one of claims 2 to 7, wherein the landmark recognition unit is configured to acquire RGB data from an RGB camera that captures images so as to include the landmarks in the captured data, and to detect, in the captured data, a landmark that corresponds to a shape related to the preset landmark, also using the RGB data. The object detection device according to any one of claims 1 to 8, wherein the shape of the pre-set landmark is a continuous or intermittent shape that exists as infrastructure. A sensor configured to output imaging data obtained by imaging a predetermined space;
An object detection device according to any one of claims 1 to 9;
An object detection system comprising: An object detection method for detecting an object using hardware resources, comprising:
detecting a mark corresponding to a form related to a preset mark in the photographed data, creating a plurality of entry-prohibited spaces according to a preset form based on the detected mark, and selecting one of the plurality of created entry-prohibited spaces;
detecting a detection target object that matches previously or newly set detection condition information based on a point cloud within the selected no-entry space in the photographed data;
The object detection method includes: A program for causing a hardware resource to execute a process for detecting an object,
A process of detecting a mark corresponding to a form related to a preset mark in the photographed data, creating a plurality of entry-prohibited spaces according to a preset form based on the detected mark, and selecting one of the plurality of created entry-prohibited spaces;
A process of detecting a detection target object that matches previously or newly set detection condition information based on a point cloud within the selected no-entry space in the photographed data;
A program for causing the hardware resource to execute the above.

Images