JP7149990B2 - HUMAN DETECTION DEVICE, INDUSTRIAL VEHICLE, AND HUMAN DETECTION METHOD - Google Patents

Description

The present disclosure relates to a human detection device, an industrial vehicle, and a human detection method.

Techniques have been proposed for detecting a person based on an image of the exterior of a vehicle and notifying an operator. For example, Patent Literature 1 discloses an image processing device configured to detect pedestrians by analyzing image data obtained from an in-vehicle camera that captures an image of the exterior of a vehicle.

JP 2009-237773 A

If an in-vehicle camera is used as in Patent Literature 1, it may be possible to detect a person even in an area that is not directly visible to the operator, as long as it is within the imaging area of the in-vehicle camera. However, it is difficult to detect people located in blind spots that are not visible to the vehicle (ie, areas that are not directly visible to the on-board camera and the operator). In particular, industrial vehicles such as forklifts sometimes travel in environments where blind spots are likely to occur due to the presence of surrounding structures (eg, containers and walls of buildings) and loads. In this case, safety cannot be ensured unless a person (for example, a worker) positioned in the blind spot can be detected. Patent Document 1 does not disclose a configuration for ensuring such safety.

In view of the circumstances described above, an object of the present disclosure is to provide a human detection device or the like capable of improving safety.

The human detection device according to the present disclosure is
a receiving device configured to receive image data obtained by imaging the exterior of an industrial vehicle from the industrial vehicle;
analyzing the image data to generate at least one index indicating the possibility of the presence of a person in an area corresponding to the image data, and determining that a person exists in the area when the index exceeds a threshold value; a determination device configured to
with
When a first index obtained by analyzing first image data obtained by imaging an area including a mirror installed outside the industrial vehicle exceeds a first threshold value, the determination device determines that the area reflected in the mirror configured to determine that a person exists in
The first threshold is set lower than the second threshold used for comparison with the second index obtained by analyzing the second image data obtained by imaging the area not including the mirror.

The industrial vehicle according to the present disclosure is
An industrial vehicle comprising the above human detection device,
a first imaging device for imaging the mirror to obtain the first image data;
a second imaging device for imaging the vicinity of the industrial vehicle and acquiring the second image data for surrounding monitoring;
Prepare.

The industrial vehicle according to the present disclosure is
An industrial vehicle comprising the above human detection device,
A first imaging device configured to be switchable between a first mode for acquiring the first image data and a second mode for acquiring the second image data is provided.

A human detection method according to the present disclosure includes:
a step of acquiring image data of the exterior of an industrial vehicle captured from the industrial vehicle;
analyzing the image data to generate at least one indicator indicating the possibility of the presence of a person in an area corresponding to the image data, and determining that a person is present in the area when the indicator exceeds a threshold; When,
including
In the determining step, when a first index obtained by analyzing first image data obtained by imaging an area including a mirror installed outside the industrial vehicle exceeds a first threshold, the area reflected in the mirror determine that there is a person within
The first threshold is set lower than the second threshold used for comparison with the second index obtained by analyzing the second image data obtained by imaging the area not including the mirror.

According to the present disclosure, it is possible to provide a human detection device and the like capable of improving safety.

It is a block diagram showing roughly functional composition of a human detection device concerning one embodiment. It is a block diagram showing roughly the hardware constitutions of the human detection device concerning one embodiment. 1 is a perspective view schematically showing an example of usage environment of a human detection device according to an embodiment; FIG. It is a top view which shows typically an example of the usage environment of the human detection apparatus which concerns on one Embodiment. It is a figure which shows an example of the mirror which the human detection apparatus which concerns on one Embodiment utilizes. It is a figure which shows an example of 1st image data. 6B is a diagram for explaining processing for extracting a mirror region from the first image data shown in FIG. 6A; FIG. FIG. 6C is a diagram showing image data from which the area of the mirror in FIG. 6B is extracted; FIG. 8 is a diagram showing image data after the distortion of the image data shown in FIG. 7 has been corrected; It is a figure which shows roughly an example of the human detection result by the human detection apparatus which concerns on one Embodiment. 6 is a flowchart for explaining human detection processing executed by the human detection device according to the embodiment;

Several embodiments will now be described with reference to the accompanying drawings. However, the dimensions, materials, shapes, relative arrangements, etc. of the components described as embodiments or shown in the drawings are not intended to limit the scope of the invention, but are merely illustrative examples. .
For example, expressions denoting relative or absolute arrangements such as "in a direction", "along a direction", "parallel", "perpendicular", "center", "concentric" or "coaxial" are strictly not only represents such an arrangement, but also represents a state of relative displacement with a tolerance or an angle or distance to the extent that the same function can be obtained.
For example, expressions such as "identical", "equal", and "homogeneous", which express that things are in the same state, not only express the state of being strictly equal, but also have tolerances or differences to the extent that the same function can be obtained. It shall also represent the existing state.
For example, expressions that express shapes such as squares and cylinders do not only represent shapes such as squares and cylinders in a geometrically strict sense. The shape including the part etc. shall also be represented.
On the other hand, the expressions "comprising", "comprising", "having", "including", or "having" one component are not exclusive expressions excluding the presence of other components.

(Configuration of human detection device)
The configuration of the human detection device 100 according to one embodiment will be described below. FIG. 1 is a block diagram schematically showing the configuration of a human detection device 100 according to one embodiment. FIG. 2 is a block diagram schematically showing the hardware configuration of the human detection device 100 according to one embodiment. The human detection device 100 is applied to industrial vehicles 600 such as excavators, bulldozers, and forklifts, for example. In one embodiment, the human detection device 100 is mounted on a forklift.

As shown in FIG. 1 , the human detection device 100 includes a receiver 10 , a determination device 20 and a transmitter 30 . As shown in FIG. 2, the human detection device 100 includes a processor 72 such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), a RAM (Random Access Memory) 74, a ROM (Read Only Memory) 76, and an HDD ( hard disk drive) 78, an input/output I/F 80, and an output I/F 82, which are connected to each other via a bus 84. The processor 72 of the human detection device 100 executes programs stored in memories such as the ROM 76 and the RAM 74 to realize various functions described later.

Receiving device 10 and transmitting device 30 are configured to communicate with other devices. In some embodiments, the human detection device 100 is communicatively connected to a first imaging device 40, a second imaging device 50, and an output device 60, for example, as shown in FIG. The first imaging device 40 , the second imaging device 50 , and the output device 60 may be configured to be mounted on the industrial vehicle 600 together with the human detection device 100 . Note that when the first imaging device 40 also functions as the second imaging device 50 by mode switching or the like, which will be described later, the human detection device 100 may be configured to include only the first imaging device 40, or may be configured to include other devices. may be configured to receive the image data from.

The receiving device 10 is configured to receive image data obtained by imaging the outside of the industrial vehicle 600 from the industrial vehicle 600 from another device. The receiving device 10 may be configured to acquire image data through short-range wireless communication or wired communication with the first imaging device 40 and/or the second imaging device 50 .

The determining device 20 is configured to analyze the image data received by the receiving device 10 to generate at least one indication of the likely presence of a person in the area corresponding to the image data. The determination device 20 may be configured to perform human detection using an object detection algorithm using machine learning, for example, an SSD (Single Shot Multibox Detector) of an object detection neural network. In such an object detection technique, the position of an object in an image is specified and surrounded by a rectangle (bounding box), and the object class (person, vehicle, etc.) is predicted for each rectangle. As at least one index indicating the possibility of the presence of a person, the certainty (reliability) of this class prediction, that is, the possibility that the object predicted as a person in the area enclosed by the rectangle is actually a person. A judgment score can be used that indicates the height of the In addition, if the size of the person to be detected is defined by the size on the image, a score indicating the effectiveness of human detection calculated based on information such as the height and width of the rectangle is used as an index. can be The determination device 20 is configured to determine that a person exists in the area when the generated index (eg, determination score) exceeds a threshold. "A region corresponding to image data" is a region appearing in the image data. That is, the area corresponding to the image data may be an area that is directly imaged, or an area that is indirectly imaged via a mirror 200, which will be described later.

The transmission device 30 is configured to transmit a signal according to the determination result of the determination device 20 to the output device 60 . Note that the receiving device 10 and the transmitting device 30 may be further configured to wirelessly communicate with a server device (not shown) external to the industrial vehicle 600 .

For example, a server device external to the industrial vehicle 600 collects information such as image data, determination results, position information, etc. from the transmission devices 30 of the plurality of human detection devices 100 or imaging devices provided on the industrial vehicle 600 and structures. You may The receiving device 10 may be configured to receive useful information from the server device. That is, the human detection device 100 may further have a human detection function based on information obtained from a server device outside the industrial vehicle 600 in addition to the human detection function based on the determination result of the determination device 20 .

The output device 60 is configured to output the detection result of the human detection device 100 or information based on the detection result (information indicating the presence or absence of a person, an alarm, or the like). The output device 60 is provided inside an industrial vehicle 600 (for example, a forklift). For example, the output device 60 includes a speaker and/or display and is configured to communicate information to the operator of the industrial vehicle 600 through audio output and/or display images. The output device 60 may notify an alarm by emitting light.

Here, the first imaging device 40 and the second imaging device 50 will be described. The first imaging device 40 is an imaging device for imaging a mirror 200, which will be described later. It is an imaging device. The first imaging device 40 is provided at a position capable of imaging the mirror 200 positioned in the traveling direction of the industrial vehicle 600 , for example, on the ceiling of the industrial vehicle 600 . The second imaging device 50 is provided at a position capable of imaging the surroundings of the industrial vehicle 600 , for example, at an upper position outside the industrial vehicle 600 .

The angle of view of the first imaging device 40 is preferably narrower than the angle of view of the second imaging device 50 . The first imaging device 40 preferably has a narrow-angle telephoto lens with an angle of view of 29° or less. The telephoto lens preferably has a focal length of, for example, 85 mm or more and 600 mm or less. With such a configuration, detection accuracy can be ensured even when the mirror 200 is small.

The angle of view of the second imaging device 50 is preferably wider than the angle of view of the first imaging device 40 . For example, the second imaging device 50 preferably has a wide-angle lens with an angle of view of 63° or more. Also, the wide-angle lens preferably has a focal length of 35 mm or less. According to such a configuration, a person positioned around the industrial vehicle is detected by the second imaging device 50 that takes an image at a wider angle than the first imaging device 40, and a person positioned in a blind spot is detected by the first imaging device 40 as a mirror. 200 can be detected. Therefore, the target range of human detection can be expanded.

Here, types of image data captured by the first imaging device 40 or the second imaging device 50 will be described. The first image data is image data obtained by capturing an area including a mirror 200 (described later) installed outside the industrial vehicle 600 by the first imaging device 40 . The second image data is image data obtained by capturing an area not including the mirror 200, such as a near (surrounding) area of the industrial vehicle, by the second imaging device 50. FIG. As described above, when the first imaging device 40 also functions as the second imaging device 50 by mode switching or the like, the first imaging device captures both the first image data and the second image data. can be

The first imaging parameter when the first imaging device 40 acquires the first image data is set in the second imaging device 50, or is set when the first imaging device 40 acquires the second image data. The second imaging parameter is set to an imaging parameter suitable for an environment with low illuminance compared to the second imaging parameter. For example, in setting the imaging parameters of the first imaging device 40, the shutter speed is slowed down (exposure time is lengthened) or the brightness of the image is adjusted according to the average brightness of the image data.

The determination device 20 is configured to determine that a person exists within the area reflected on the mirror 200 when the first index obtained by analyzing the first image data exceeds the first threshold. The first threshold is set lower than the second threshold used for comparison with the second index obtained by analyzing the second image data. The determination device 20 analyzes the first image data and the second image data by the same method (human detection algorithm), and at least one index indicating the possibility of the presence of a person in the area corresponding to each image data, here A first index and a second index are generated. The first index and the second index are indexes of the same kind and of the same dimension.

The determination device 20 may include a time-series filter for suppression of false detection. For the first image data, the time-series filter of the determination device 20 is configured to determine that human detection has been established when the frequency or duration at which the first index exceeds the first threshold exceeds the third threshold. may The determination device 20 may be configured to output an alarm according to the human detection result. For example, the determination device 20 is configured to output an alert by sending a signal to the output device 60 to output the alert. The third threshold is set lower than the fourth threshold that is set in the time-series filter when determining whether human detection is established based on the second image data.

The determination device 20 may include a time-series filter corresponding to each of the first image data and the second image data, and may be configured to be able to analyze any image data. Further, the determination device 20 may be a device for the first imaging device 40 and independent of the determination device for the second imaging device 50 .

When the first imaging device 40 for imaging the mirror 200 also has the function of the second imaging device 50 for monitoring the surroundings for imaging a monitoring area other than the mirror 200, for example, the vicinity of the industrial vehicle 600, the first imaging device 40 A first mode for acquiring the first image data and a second mode for acquiring the second image data may be switchable. That is, the first imaging device 40 may be configured to acquire not only the first image data but also the second image data. In this case, the determination device 20 is configured to detect a person using the first threshold when the image data acquired from the first imaging device 40 is the first image data. Further, the determination device 20 is configured to detect a person using the second threshold when the image data acquired from the first imaging device 40 is the second image data.

The determination device 20 may be configured to correct distortion specific to the mirror 200 included in the first image data based on a calibration pattern of a sheet-like object 300 (described later) reflected on the mirror 200 . The calibration pattern may be the shape of the sheet-like object 300 or the pattern of the sheet-like object 300 . The pattern may be a grid pattern (for example, a checkerboard pattern). The sheet-shaped object 300 may be a mat placed on the ground or floor surface by the operator, or a poster to be pasted on the wall.

When the imaging range includes a marker 210 (described later), the first imaging device 40 identifies the position of the mirror 200 based on the marker 210, extracts the area of the mirror 200, and acquires image data. It may be configured as Note that the first imaging device 40 may be configured to only perform imaging without specifying the marker 210 or extracting the region, and the determination device 20 may perform the above processing.

(Concrete example of usage environment of human detection device)
A specific example of the usage environment of the human detection device 100 will be described. First, as described above, human detection device 100 is configured to detect a person positioned in a blind spot using mirror 200 . Blind spots are hidden behind structures such as ships and buildings (warehouses and factories) when viewed from an industrial vehicle, and are directly visible from an imaging device (vehicle camera) mounted on the industrial vehicle 600 and an operator of the industrial vehicle 600. It is a region that cannot be observed or is difficult to observe directly.

FIG. 3 is a perspective view schematically showing an example of usage environment of the human detection device 100 according to one embodiment. In this example, when the industrial vehicle 600 is located in the port 500 , the blind spot is inside near the doorway of the ship 400 anchored in the port 500 . In this example, industrial vehicle 600 is a forklift. As seen from the industrial vehicle 600, inside the doorway of the ship 400 there is an area that is darker than outdoors and hidden behind the structure of the ship 400. Although the human detection device 100 is omitted from the drawing, it is actually provided in the industrial vehicle 600 .

FIG. 4 is a plan view schematically showing an example of the usage environment of the human detection device 100 according to one embodiment. This figure corresponds to the example shown in FIG. As shown in FIGS. 3 and 4 , a mirror 200 is provided on the ceiling of the doorway of the ship 400 so as to reflect blind spots in the ship 400 . Also, on the floor of the doorway of the ship 400, a checkered mat is arranged as the sheet-like object 300. FIG. A mirror 200 and a sheet-like object 300 are arranged so that the sheet-like object 300 is reflected on the mirror 200 .

The mirror 200 preferably has a convex reflective surface. The curvature of the convex surface is preferably larger than the curvature of curved mirrors (road reflectors) generally installed on public roads and the like. This makes it possible to project a wide range. However, since the distortion increases, it is necessary to correct the distortion of the image data in order to improve the human detection accuracy.

FIG. 5 is a diagram showing an example of the mirror 200 used by the human detection device 100 according to one embodiment. As shown in FIG. 5, mirror 200 may be a dome mirror. For example, the mirror 200 may have a mounting portion 200b for mounting on the ceiling of a structure, and a hemispherical reflecting surface 200a capable of reflecting the entire downward direction when mounted on the ceiling. The mirror 200 has a shape obtained by cutting out a part of a spherical surface (for example, a shape obtained by dividing a hemispherical surface into four parts), and may be provided at a corner (corner) of a structure. The attachment portion 200b may include a magnet base and may be configured to be easily attachable to a metal member.

In such an environment, as shown in FIGS. 3 and 4, the human detection device 100 acquires the first image data obtained by capturing the area including the mirror 200 by the first imaging device 40 via the receiving device 10. . The determination device 20 of the human detection device 100 corrects the distortion of the acquired first image data using the pattern of the sheet-like object 300 reflected on the mirror 200 as a calibration pattern. Further, the determination device 20 analyzes the first image data to obtain the first index, and compares it with the first threshold to detect the person H within the area reflected on the mirror 200 . In this manner, the human detection device 100 can detect the human H positioned in the blind spot via the mirror 200 .

As in the example shown in FIGS. 3 and 4, while the industrial vehicle 600 is moving to the entrance of the ship 400, the time during which a person present in the blind spot of the entrance of the ship 400 can be detected via the mirror 200 is short, and the number of data points is may be less. Even in such a case, the time-series filter of the determination device 20 is set to the third threshold lower than the fourth threshold when performing human detection based on the first image data. The person H can be detected from the first image data even if the imaged time is extremely short. Therefore, safety can be improved.

When the industrial vehicle 600 enters the entrance/exit of the ship 400 for loading and unloading of cargo, wireless communication with a server device outside the industrial vehicle may be difficult. As a result, the self-position cannot be acquired, and the ship 400 is not always anchored at the same position, so it may be difficult to estimate the self-position. Even in such a case, the human detection device 100 can easily acquire image data from the first imaging device 40 and/or the second imaging device 50 by short-range wireless communication or wired communication. Therefore, it is possible to detect a person without performing self-position estimation or information acquisition through communication with a server device.

FIG. 6A is a diagram showing an example of first image data. FIG. 6B is a diagram for explaining processing for extracting the area (mirror image area) of the mirror 200 from the first image data shown in FIG. 6A. FIG. 7 is a diagram showing image data extracted from the area of the mirror 200 in FIG. 6B. FIG. 8 is a diagram showing image data after the distortion of the image data shown in FIG. 7 has been corrected. FIG. 9 is a diagram schematically showing an example of human detection results by the human detection device 100 according to one embodiment.

As shown in FIG. 6A, a marker 210 representing a two-dimensional code may be provided around the mirror 200 . For example, four markers 210 may be provided surrounding the mirror 200 . Note that the number and arrangement of the markers 210 are not limited to the illustrated example. The first imaging device 40 or the determination device 20 extracts the area of the mirror 200 within the first image data based on the marker 210 . Note that the first imaging device 40 or the determination device 20 may be configured to recognize the mirror 200 by matching the shape pattern of the mirror 200 instead of the marker 210 .

As a result, for example, a rectangular area surrounded by markers 210 is extracted as shown in FIG. 6B. As a result, image data including the mirror image of the mirror 200 is acquired as shown in FIG. By correcting the distortion of this image data, the image data shown in FIG. 8 is obtained. By analyzing the image data after the distortion correction, as shown in FIG. 9, the person H is detected, and the possibility that the detected person H is actually a person is obtained as at least one index indicating the possibility of the existence of the person. A judgment score (e.g., 80%) indicating the high degree of sexuality is obtained.

(Flow of human detection processing)
The flow of human detection processing by the human detection device 100 according to one embodiment will be described below. FIG. 10 is a flowchart for explaining human detection processing executed by the human detection device 100 according to one embodiment. In this embodiment, an example of human detection processing after the mirror 200 and the sheet-like object 300 are arranged will be described.

In this example, (1) the first imaging device 40 may be configured to acquire only the first image data, and the second imaging device 50 may be configured to acquire only the second image data, or ( 2) When the first imaging device 40 also has the function of the second imaging device 50 by mode switching or the like, the second imaging device 50 is omitted, and the human detection process is performed only by the first imaging device 40, and the first imaging device 40 may be configured to acquire both the first image data and the second image data. In other words, the imaging device may be used for both purposes for acquiring various types of image data, or may be used for exclusive use. In this way, the configuration of the human detection device 100 and the content of the human detection process are not limited to the illustrated content, and can be modified as appropriate.

First, the human detection device 100 acquires image data (step S1). Specifically, the human detection device 100 acquires image data from the first imaging device 40 or the second imaging device 50 via the receiving device 10 .

Here, the human detection device 100 determines whether or not the acquired image data is the first image data (step S2). For example, whether or not the transmission source of the image data is the first imaging device 40, whether or not the image data is captured in the first mode, whether or not the image data includes the mirror 200 or the marker 210, etc. Processing is performed.

If it is determined that the image data is not the first image data (step S2; No), the determination device 20 sets the second threshold as the threshold used in the human detection process in step S8 (step S3), and performs the human detection process ( step S8).

When it is determined that the image data is the first image data (step S2; Yes), the human detection device 100 extracts the area of the mirror 200 from the image data (step S4). Note that the marker 210 or shape pattern matching may be used in the processes of steps S4 and S2.

The human detection device 100 determines whether or not the sheet-shaped object 300 is included in the extracted image data (step S5). If it is determined that the sheet-shaped object 300 is included (step S5; Yes), the determination device 20 performs distortion correction using the calibration pattern (step S6). If it is determined that the sheet-shaped object 300 is not included (step S5; No), step S6 is skipped. Further, the determination device 20 sets a first threshold as a threshold to be used for human detection processing in step S7 (step S7), and performs human detection processing (step S8).

In the human detection process, the determination device 20 analyzes either the first image data after the distortion correction or the second image data. The determination device 20 uses the first threshold value for the first image data and the second threshold value for the second image data to determine the presence or absence of a person. Further, the determination device 20 determines whether or not it is determined that human detection has been established by comparing with the third threshold value and the fourth threshold value, and transmits a signal to the output device 60 according to the determination result.

For example, as at least one index indicating the possibility of the presence of a person in the area corresponding to the image data, a judgment score indicating the human-likeness of the object in the target rectangular area output by an arbitrary object detection (human detection) algorithm is used. , the first threshold is set to 60% and the second threshold is set to 90%. In this case, the determination score obtained as a result of analyzing the object in the second image data in which the second imaging device 50 captures an area not including the mirror 200, such as a near (surrounding) area of the industrial vehicle 600, is determined by the second It is not determined that a person exists unless the person is recognized with a high degree of reliability that exceeds the threshold value of 90%. On the other hand, the determination score obtained as a result of analyzing the object in the first image data obtained by imaging the area including the mirror 200 installed outside the industrial vehicle 600 by the first imaging device 40 is the first threshold. If a person is recognized with a reliability of 60% or more, it is determined that a person exists. Note that the values of the first threshold and the second threshold are merely examples, and the present invention is not limited to these.

Further, for example, as a time-series filter, for the first image data, a third threshold of 0.5 is used as a third threshold for comparison with the time when the first index (determination score) exceeds the first threshold (for example, 60% as described above). For the second image data, when 2 seconds is set as the fourth threshold for comparison with the time when the second index (determination score) exceeds the second threshold (e.g., 90% as described above) think of. In this case, the determination score obtained as a result of analyzing the object in the second image data in which the second imaging device 50 captures an area not including the mirror 200, such as a near (surrounding) area of the industrial vehicle 600, is determined by the second It is determined that human detection has been established when the time for which the threshold value of 90% is exceeded continues for two seconds or longer. On the other hand, the determination score obtained as a result of analyzing the object in the first image data obtained by imaging the area including the mirror 200 installed outside the industrial vehicle 600 by the first imaging device 40 is the first threshold. It is determined that human detection has been established when the time exceeding 60% continues for 0.5 seconds or longer. Note that the values of the third threshold value and the fourth threshold value are merely examples, and the present invention is not limited to these values.

The flow of human detection processing has been described above. The above series of steps may be repeatedly performed while industrial vehicle 600 is being driven. The order of each step can be changed as appropriate. For example, various threshold settings may be performed at earlier timing than the illustrated example. Alternatively, any threshold may be set in advance, and the threshold may be appropriately changed according to the type of image data.

The present disclosure is not limited to the above-described embodiments, and includes modifications of the above-described embodiments and modes in which a plurality of embodiments are appropriately combined.

(summary)
The contents described in each of the above embodiments are understood as follows, for example.

(1) The industrial vehicle human detection device (100) according to the present disclosure is
a receiving device (10) configured to receive image data obtained by imaging the outside of an industrial vehicle (600) from the industrial vehicle (600);
analyzing the image data to generate at least one index indicating the possibility of the presence of a person in an area corresponding to the image data, and determining that a person exists in the area when the index exceeds a threshold value; a determination device (20) configured in
with
The determination device (20) determines that a first index obtained by analyzing first image data of an area including a mirror (200) installed outside the industrial vehicle (600) exceeds a first threshold. is configured to determine that a person exists within the area reflected in the mirror (200) when
The first threshold is set lower than the second threshold used for comparison with the second index obtained by analyzing the second image data obtained by imaging the area not including the mirror (200).

According to the above configuration, even a person in a blind spot can be detected through the mirror (200). High-precision detection is difficult with the human detection technology via the mirror (200). However, in order to ensure safety, it is necessary to suppress detection omissions. In this respect, in the above configuration, human detection is performed using the first threshold lower than the second threshold for the first image data obtained by imaging the area including the mirror (200), so detection omissions are suppressed and safety is improved. can be improved.

(2) In some embodiments, in the configuration described in (1) above,
The determination device (20) includes a time series filter for suppression of overdetection,
For the first image data, the time-series filter of the determination device (20) establishes human detection when the frequency or duration of the first index exceeding the first threshold exceeds a third threshold. is configured to determine
The third threshold is set lower than the fourth threshold set in the time-series filter when it is determined that the human detection has been established based on the second image data.

With respect to the first image data, it is determined that human detection has been established using the third threshold lower than the fourth threshold, so that it is easier to determine that human detection has been established than with the second image data. Therefore, detection omission can be suppressed and safety can be improved.

(3) The industrial vehicle (600) according to the present disclosure is
An industrial vehicle (600) comprising the human detection device (100) according to (1) or (2) above,
a first imaging device (40) for imaging the mirror (200) to obtain the first image data;
a second imaging device (50) for imaging the vicinity of the industrial vehicle (600) and acquiring the second image data for surrounding monitoring;
Prepare.

According to the above configuration, by including the human detection device (100) in the industrial vehicle (600), it is possible to suppress detection omissions and improve safety. In addition, since an imaging device is provided for each purpose of imaging the mirror (200) and monitoring the surroundings, it is possible to perform imaging under imaging conditions suitable for each purpose.

(4) The industrial vehicle (600) according to the present disclosure is
An industrial vehicle (600) comprising the human detection device (100) according to (1) or (2) above,
A first imaging device (40) configured to be switchable between a first mode for acquiring the first image data and a second mode for acquiring the second image data is provided.

According to the above configuration, the first imaging device (40) can acquire the first image data and the second image data, so the overall configuration can be simplified.

(5) In some embodiments, in the configuration described in (3) or (4) above,
A first imaging parameter for acquiring the first image data is set to an imaging parameter suitable for an environment with low illuminance compared to a second imaging parameter for acquiring the second image data.

A blind spot reflected in the mirror (200) is often a dark place. In this respect, according to the above configuration, the first imaging parameter when acquiring the first image data is set to an imaging parameter suitable for an environment with low illuminance compared to the second imaging parameter. Therefore, the first image data suitable for image analysis can be obtained.

(6) In some embodiments, in the configuration described in (3) above,
The angle of view of the first imaging device (40) is narrower than the angle of view of the second imaging device.

According to the above configuration, detection accuracy can be ensured even when the mirror (200) is small.

(7) In some embodiments, in the configuration described in any one of (3) to (6) above,
The determination device (20) corrects the distortion peculiar to the mirror (200) included in the first image data based on the calibration pattern of the sheet-like object (300) reflected on the mirror (200). configured to

According to the above configuration, since the distortion peculiar to the mirror (200) (distortion due to the shape and curvature of the mirror (200)) is corrected, the human detection accuracy is improved. Also, if the structure is owned by a customer, it is difficult to install an object for human detection. Even in such a case, a sheet-like object (300) (for example, a mat or a poster) is advantageous in that the worker can easily set it up temporarily. The sheet-like object (300) is less likely to interfere with the traffic of industrial vehicles (600) and people, so it is also advantageous in terms of workability.

(8) In some embodiments, in the configuration described in any one of (3) to (7) above,
The mirror (200) has a convex reflecting surface (200b).

According to the above configuration, since a wide area is reflected on the mirror (200), it becomes easier to detect a person in a blind spot through the mirror (200).

(9) In some embodiments, in the configuration described in any one of (3) to (8) above,
The first imaging device (40) identifies the position of the mirror (200) based on the marker (210) when the marker (210) is included in the imaging range, and area is extracted to obtain the image data.

According to the above configuration, since the marker (210) is used, the presence or absence of the mirror (200) can be detected, and the area of the mirror (200) included in the imaging range can be easily extracted.

(10) In some embodiments, in the configuration described in any one of (3) to (9) above,
It further comprises an output device (60) for outputting a detection result of the human detection device (100) or an alarm based on the detection result.

According to the above configuration, it is possible to notify the operator of the industrial vehicle (600) of the human detection result, thereby improving safety.

(11) The human detection method according to the present disclosure includes:
a step of acquiring image data of the exterior of the industrial vehicle (600) captured from the industrial vehicle (600);
analyzing the image data to generate at least one indicator indicating the possibility of the presence of a person in an area corresponding to the image data, and determining that a person is present in the area when the indicator exceeds a threshold; When,
including
In the determining step, when a first index obtained by analyzing first image data obtained by imaging an area including a mirror (200) installed outside the industrial vehicle (600) exceeds a first threshold value , determining that a person exists within the area reflected in the mirror (200),
The first threshold is set lower than the second threshold used for comparison with the second index obtained by analyzing the second image data obtained by imaging the area not including the mirror (200).

According to the above method, even a person in a blind spot can be detected through the mirror (200). In addition, since human detection is performed using the first threshold lower than the second threshold for the first image data obtained by imaging the area including the mirror (200), it is possible to suppress detection omissions and improve safety. becomes.

10 Receiving device 20 Determining device 30 Transmitting device 40 First imaging device 50 Second imaging device 60 Output device 72 Processor 74 RAM
76 ROMs
78 HDDs
80 Input I/F
82 Output I/F
84 Bus 100 Human detection device 200 Mirror 200a Reflective surface 200b Mounting portion 210 Marker 300 Object 400 Ship 500 Port 600 Industrial vehicle

Claims (8)

a receiving device configured to receive image data obtained by imaging the exterior of an industrial vehicle from the industrial vehicle;
At least one indicator that analyzes the image data and indicates the possibility of the presence of a person in an area corresponding to the image data, wherein the image data is corresponded to by detecting a person using an object detection algorithm using machine learning. generating at least one indicator for determining the presence of a person in an area, and using the object detection algorithm to determine that a person is present in the area when the indicator exceeds a threshold a device;
An industrial vehicle comprising a human detection device comprising
When a first index obtained by analyzing first image data obtained by imaging an area including a mirror installed outside the industrial vehicle and separated from the industrial vehicle exceeds a first threshold, the determination device , configured to determine that a person exists within the area reflected in the mirror,
The first threshold is set lower than a second threshold used for comparison with a second index obtained by analyzing second image data obtained by imaging an area not including the mirror ,
The industrial vehicle is
a first imaging device for imaging the mirror to obtain the first image data;
The industrial vehicle further comprising a second imaging device for imaging the vicinity of the industrial vehicle to acquire the second image data for surrounding monitoring . a receiving device configured to receive image data obtained by imaging the exterior of an industrial vehicle from the industrial vehicle;
At least one indicator that analyzes the image data and indicates the possibility of the presence of a person in an area corresponding to the image data, wherein the image data is corresponded to by detecting a person using an object detection algorithm using machine learning. generating at least one indicator for determining the presence of a person in an area, and using the object detection algorithm to determine that a person is present in the area when the indicator exceeds a threshold a device;
An industrial vehicle comprising a human detection device comprising
When a first index obtained by analyzing first image data obtained by imaging an area including a mirror installed outside the industrial vehicle and separated from the industrial vehicle exceeds a first threshold, the determination device , configured to determine that a person exists within the area reflected in the mirror,
The first threshold is set lower than a second threshold used for comparison with a second index obtained by analyzing second image data obtained by imaging an area not including the mirror ,
The industrial vehicle is
An industrial vehicle further comprising a first imaging device configured to be switchable between a first mode for acquiring the first image data and a second mode for acquiring the second image data . The determination device includes a time series filter for suppressing overdetection,
For the first image data, the time-series filter of the determination device determines that human detection has been established when the frequency or time at which the first index exceeds the first threshold exceeds a third threshold. configured as
3. The third threshold according to claim 1 or 2 , wherein the third threshold is set lower than the fourth threshold set in the time-series filter when it is determined that the human detection has been established based on the second image data. industrial vehicle . 2. A first imaging parameter for acquiring the first image data is set to an imaging parameter suitable for an environment with low illuminance compared to a second imaging parameter for acquiring the second image data. 4. The industrial vehicle according to any one of items 1 to 3 . The industrial vehicle according to claim 1 , wherein the angle of view of the first imaging device is narrower than the angle of view of the second imaging device. The mirror has a convex reflecting surface,
The first image data is image data obtained by imaging an area including the mirror and a calibration pattern of a sheet-like object reflected on the mirror,
6. The determination device according to any one of claims 1 to 5 , wherein the determination device is configured to correct distortion specific to the mirror included in the first image data based on the calibration pattern in the first image data. Industrial vehicles as described in paragraph. 2. When the imaging range includes a marker, the first imaging device identifies the position of the mirror based on the marker, extracts the area of the mirror, and acquires the image data. 7. The industrial vehicle according to any one of items 1 to 6 . The industrial vehicle according to any one of claims 1 to 7 , further comprising an output device for outputting a detection result of said human detection device or an alarm based on said detection result.

Images