JP7831019B2 - Driving support device and load collapse detection method - Google Patents

Description

This disclosure relates to a driver assistance system and a method for detecting cargo collapse.

In the cargo compartment monitoring device disclosed in Patent Document 1 below, the cargo compartment image acquisition unit acquires images of the vehicle's cargo compartment. The edge image extraction unit extracts edge images of the cargo from the cargo compartment images captured by the cargo compartment image acquisition unit. The closed region extraction unit extracts closed regions from the edge images extracted by the edge image extraction unit. The closed region monitoring unit monitors the position of the closed regions extracted by the closed region extraction unit. The cargo collapse determination unit determines that cargo collapse has occurred in the cargo compartment if the amount of movement of the closed regions exceeds a predetermined threshold, or if the closed regions disappear or appear.

WO2020-245905 publication

For example, when a camera is installed on the towing vehicle of a towing vehicle, and cargo shifting is detected based on the image of the towed vehicle captured by the camera, there is a risk of being affected by changes in the coupling angle between the towing vehicle and the towed vehicle, or by objects present around the towed vehicle. In other words, there was a risk of incorrect detection of cargo shifting due to changes in the coupling angle or the inclusion of objects surrounding the towed vehicle in the image.

The driving assistance device according to this disclosure is a driving assistance device for detecting cargo collapse of a cargo bed towed by a towing vehicle, and comprises: an edge image generation unit that acquires virtual images of the rear region of the towing vehicle viewed from above over time and generates edge images including at least the edges of the cargo bed and the cargo from each of the virtual images; a determination region detection unit that detects a determination region corresponding to the cargo bed from the edge image; and a cargo collapse determination unit that determines whether or not cargo collapse has occurred. The cargo collapse determination unit determines whether or not cargo collapse has occurred by comparing a first edge of the cargo extracted from a first virtual image of the virtual images with a second edge of the cargo extracted from a second virtual image of the virtual images that was generated before the first virtual image.

The cargo collapse detection method according to this disclosure is a method for detecting cargo collapse of a cargo loaded on a cargo bed towed by a towing vehicle. The method involves acquiring virtual images of the rear region of the towing vehicle viewed from above over time, generating edge images from each of the virtual images that include at least the edges of the cargo bed and the cargo, detecting a determination region corresponding to the cargo bed from the edge images, and determining whether cargo collapse has occurred by comparing a first edge of the cargo extracted from a first virtual image of the virtual images with a second edge of the cargo extracted from a second virtual image of the virtual images that was generated before the first virtual image.

According to this disclosure, it is possible to more reliably detect cargo shifting that occurs on the loading platform.

This is a block diagram showing an example of the overall configuration of the driver assistance device according to the first embodiment. This is a plan view showing a vehicle to which the driving support device and the cargo collapse detection method according to the embodiment are applied. This is a side view showing a vehicle to which the driver assistance device and the cargo collapse detection method according to the embodiment are applied. This is a schematic diagram showing the virtual image at time T2, generated by the ambient image generation device. This is a schematic diagram showing the virtual image at time T1, generated by the ambient image generation device. This is a schematic diagram showing the edge image at time T2 generated by the edge image generation unit according to the first embodiment. This is a schematic diagram showing the edge image at time T1 generated by the edge image generation unit according to the first embodiment. This diagram illustrates a load collapse detection unit according to the first embodiment, and is a schematic diagram showing the positional relationship between the first edge and the second edge. This diagram illustrates a load collapse detection unit according to the first embodiment, and is a schematic diagram showing the positional relationship between the first edge and the second edge. This figure illustrates an example of load collapse determination according to the first embodiment, and is a schematic diagram showing an edge image at time T1. This figure illustrates an example of load collapse determination according to the first embodiment, and is a schematic diagram showing an edge image at time T1. This is a diagram illustrating a notification unit according to the first embodiment, and is a partially schematic diagram showing an example of a highlighting displayed on a virtual image at time T1. This is a diagram illustrating a notification unit according to the first embodiment, and is a partially schematic diagram showing an example of a highlighting displayed on a virtual image at time T1. This flowchart shows an example of the operation of the driver assistance device according to the first embodiment. This flowchart shows an example of the operation of the load collapse detection unit according to the first embodiment. This is a block diagram showing an example of the overall configuration of the driver assistance system according to the second embodiment. This flowchart shows an example of the operation of the driver assistance device according to the second embodiment.

The following describes several exemplary embodiments with reference to the drawings. Elements having the same function are denoted by the same reference numerals, and redundant descriptions are omitted.

In each diagram, FR and RR indicate the front and rear of the towing vehicle V1, i.e., the front and rear in the vehicle's longitudinal direction. LH and RH indicate the left and right sides of the towing vehicle V1, i.e., the vehicle's width direction. UP and DN indicate the up and down of the towing vehicle V1, i.e., the up and down in the vehicle's vertical direction. Note that the front of the towing vehicle V1 refers to the direction of travel when the towing vehicle V1 is moving forward. The rear of the towing vehicle V1 refers to the direction opposite to the direction of travel when the towing vehicle V1 is moving forward.

According to the driving support device and cargo collapse detection method of this embodiment, it is possible to detect cargo collapse of the cargo 61 loaded on the cargo bed 60 towed by the towing vehicle V1. The driving support device and cargo collapse detection method may also be applied to a vehicle V, which is a towing vehicle configured by connecting a towing vehicle V1 and a towed vehicle V2, as shown in the example in Figures 2 and 3. The towed vehicle V2 is the vehicle towed by the towing vehicle V1 and is connected to the rear part of the towing vehicle V1.

The towed vehicle V2 is equipped with a cargo bed 60 capable of loading cargo 61. The cargo bed 60 is located in the area behind the towing vehicle V1. The cargo 61 may be slabs, billets, coils, etc. For example, the cargo 61 may include multiple slabs stacked vertically. In the illustrated example, the cargo bed 60 has a roughly rectangular shape in plan view. In this case, the lengthwise dimension L1 of the cargo bed 60 may be longer than the widthwise dimension W1. That is, the cargo bed 60 may be a portion that extends in its lengthwise direction and has a roughly rectangular shape in plan view. In addition, multiple metal columns (not shown) may be erected on the cargo bed 60 to restrict the displacement of the cargo 61. That is, the towed vehicle V2 may be a stanchion-type trailer. In the illustrated example, the cargo 61 has a roughly rectangular shape in plan view. Also, in the state illustrated in the figure, the longitudinal direction of the towing vehicle V1 and the longitudinal direction of the towed vehicle V2 are roughly parallel. Therefore, the longitudinal direction of the towing vehicle V1 and the longitudinal direction of the cargo bed 60 coincide. Also, the width direction of the towed vehicle V2 and the width direction of the cargo bed 60 coincide.

The surrounding image generation device 3 is a device that generates virtual images 5 over time. The virtual image 5 is an overhead view of at least the rear area of the towing vehicle V1. The rear area is the area opposite to the direction of travel when the towing vehicle V1 is moving forward. The surrounding image generation device 3 is equipped with a camera 4 capable of capturing images of the area around the towing vehicle V1 over time. Camera 4 includes cameras 4a and 4b, which are capable of capturing images of the rear area of the towing vehicle V1 over time. The surrounding image generation device 3 can synthesize the images captured by cameras 4a and 4b and generate and output virtual images 5 at each time point T over time. The surrounding image generation device 3 is mounted on the towing vehicle V1.

As shown in Figures 2 and 3, cameras 4a and 4b may be mounted facing backward, for example, on the rear side of the cabin 62 of the towing vehicle V1. This allows cameras 4a and 4b to continuously photograph the towed vehicle V2 with the cargo 61 loaded on the cargo bed 60. In manufacturing plants that produce slabs, or cargo terminals that transship truck cargo, the number of towed vehicles V2 may exceed the number of towing vehicles V1. Even in such cases, since camera 4 is mounted on the towing vehicle V1, the rear area of the towing vehicle V1 can be photographed regardless of whether camera 4 is installed on the towed vehicle V2. In other words, multiple towed vehicles V2 can share a towing vehicle V1 equipped with camera 4, thereby improving cost-effectiveness.

Cameras 4a and 4b are positioned above the cargo 61 and at a height that allows simultaneous imaging of the rear end 60b of the cargo bed 60 and the rear end 61b of the cargo 61. Cameras 4a and 4b may also be mounted at a height higher than the height of the towing vehicle V1 via fixing devices such as brackets (not shown). In the illustrated example, two cameras 4a and 4b are used to image the rear area of the towing vehicle V1, but this is not the only option. One, three, or more cameras may be used to image the rear area of the towing vehicle V1.

The towing vehicle V1 may also be equipped with cameras 4c, 4d, and 4e. Camera 4c is mounted on the front of the towing vehicle V1 and is capable of capturing images of the area in front of the towing vehicle V1 over time. Camera 4d is mounted on the left side of the towing vehicle V1 and is capable of capturing images of the area to the left of the towing vehicle V1 over time. Camera 4e is mounted on the right side of the towing vehicle V1 and is capable of capturing images of the area to the right of the towing vehicle V1 over time. In the examples shown in Figures 2 and 3, cameras 4c, 4d, and 4e are mounted in approximately the same position as cameras 4a and 4b in the vertical direction, but this is not limited to this. For example, cameras 4c, 4d, and 4e may be installed at a higher or lower position than cameras 4a and 4b. The surrounding image generation device 3 may generate a virtual image 5 by combining the images captured by cameras 4a to 4e.

As illustrated in Figures 4 and 5, the virtual image 5 includes at least an image of the rear area of the towing vehicle V1 viewed from above. More specifically, the virtual image 5 includes an image of the rear area of the towing vehicle V1, with the cargo bed 60 and cargo 61 arranged thereon, viewed from above. The virtual image 5 may also include images of buildings surrounding the towed vehicle V2, or structures located on the road.

The first virtual image 5a illustrated in Figure 5 is the image at the current time T1 among the virtual images 5 generated over time by the ambient image generation device 3. That is, the first virtual image 5a is the most recently generated image among the virtual images 5 generated by the ambient image generation device 3, in terms of the chronological order of the captured images.

The second virtual image 5b illustrated in Figure 4 is an image at a certain time T2 among the virtual images 5 generated over time by the ambient image generation device 3. Time T2 is a predetermined time prior to time T1. That is, the second virtual image 5b is an image generated before the first virtual image 5a. This predetermined time may be, for example, the same as the shooting interval of the camera 4, or it may be an integer multiple of the shooting interval. For example, the second virtual image 5b may be a virtual image 5 generated based on an image captured one frame before the first virtual image 5a, or a virtual image 5 generated based on images captured multiple frames before the first virtual image 5a.

In the examples shown in Figures 4 and 5, the towing vehicle V1 performs a left turn as indicated by arrow A by changing the steering angle. The second virtual image 5b shows the state where the longitudinal direction of the towed vehicle V2 intersects the longitudinal direction of the towing vehicle V1 at a coupling angle θ2. On the other hand, the first virtual image 5a is a virtual image 5 after a predetermined time has elapsed from the state shown in the second virtual image 5b, i.e., one or more frames later. The first virtual image 5a shows the state where the longitudinal direction of the towed vehicle V2 intersects the longitudinal direction of the towing vehicle V1 at a coupling angle θ1. This coupling angle θ1 is larger than the coupling angle θ2, for example, as shown in Figure 5.

(First Embodiment)
Referring to Figure 1, the configuration of the driver assistance device 1a according to the first embodiment will be described. The driver assistance device 1a according to this embodiment includes a controller 2a that performs processing necessary for assisting the driving of the vehicle V. The driver assistance device 1a is connected to an ambient image generation device 3 and an HMI 7 (Human Machine Interface).

First, let's describe the HMI 7. The HMI 7 is a device for inputting and outputting data to the controller 2a, and includes a display unit 8, for example, a touch panel display (see Figure 1). The display unit 8 may display the virtual image 5 described above, or highlighting by the notification unit 40 described later. An input unit 9 is also provided on the display surface of the display unit 8. The HMI 7 may, for example, be installed inside the towing vehicle V1, but is not limited to this. For example, the HMI 7 may be installed in a control facility that monitors the status of the towing vehicle V1. In that case, the HMI 7 and the controller 2a may be wirelessly connected. Furthermore, the HMI 7 may be installed both inside the towing vehicle V1 and in the control facility.

The input unit 9 is configured to allow input of predetermined codes, numerical values, etc., to the controller 2a. For example, the input unit 9 can accept an identification code assigned to the towed vehicle V2. The identification code is a code set for each towed vehicle V2 and is associated with the lengthwise dimension L1 and widthwise dimension W1 of the cargo bed 60. That is, by inputting the identification code into the input unit 9, the controller 2a can identify the dimensions L1 and W1 of the cargo bed 60. The identification code may include letters, numbers, symbols, etc. The input unit 9 may also be configured to accept input of the values of the dimensions L1 and W1 of the cargo bed 60.

The input unit 9 may be configured with a start switch (not shown) for inputting a first trigger signal to the controller 2a indicating that preparations for starting the driving assistance system are complete. The HMI 7 may also output a first trigger signal in response to the input of an identification code or the values of dimensions L1 and W1.

Next, the controller 2a will be described. The controller 2a is a general-purpose microcomputer equipped with a CPU (Central Processing Unit), memory, input/output unit, etc. The microcomputer's memory contains a computer program that includes predefined rules and instructions for assisting the driving of vehicle V. By executing this computer program, the microcomputer can provide driving assistance for the towing vehicle V1.

In the example shown in Figure 1, the controller 2a comprises an edge image generation unit 10, a determination area detection unit 20, and a load collapse determination unit 30.

The edge image generation unit 10 acquires virtual images 5 from the surrounding image generation device 3 over time. Then, by extracting edges from each of the virtual images 5, it generates an edge image 11. The edge image 11 is an image that includes at least the edges of the loading platform 60 and the cargo 61.

As illustrated in Figure 6, the edge image generation unit 10 generates a second edge image 11b from the second virtual image 5b (see Figure 4). In the illustrated example, the second edge image 11b includes a second cargo bed edge 14, which is a contour line constituting the outer edge of the cargo bed 60 extracted from the second virtual image 5b, and a second edge 15, which is a contour line constituting at least a portion of the outer edge of the cargo 61.

Furthermore, as illustrated in Figure 7, the edge image generation unit 10 generates a first edge image 11a from the first virtual image 5a (see Figure 5). In the illustrated example, the first edge image 11a includes a first cargo bed edge 12, which is a contour line constituting the outer edge of the cargo bed 60 extracted from the first virtual image 5a, and a first edge 13, which is a contour line constituting at least a part of the outer edge of the cargo 61.

Note that the outline of the towing vehicle V1 is omitted in Figures 6 and 7. Furthermore, the edges included in the edge image 11 are not limited to the illustrated examples. For example, the edge image 11 may include outlines indicating the boundary lines of each surface constituting the outer shape of the cargo 61, outlines constituting the outer shape of buildings surrounding the towed vehicle V2, and outlines constituting the outer shape of structures placed on the road. In the figures showing the first edge image 11a, the edges of the cargo 61 related to the second edge image 11b are shown with dashed lines for explanatory purposes.

When the edge image generation unit 10 generates an edge image 11 from the virtual image 5, known edge detection processes can be used. For example, the edge image 11 may be generated by applying edge detection processes such as the gradient method or the Laplacian method to the virtual image 5. The gradient method is a process that extracts edges by applying a differential filter to an image to determine the magnitude of the gradient of each pixel in the image, and then binarizing it using a predetermined threshold. The Laplacian method is a process that extracts edges by applying a Laplacian filter to an image, finding zero crossings, and then binarizing it using a predetermined threshold.

Next, the determination region detection unit 20 will be described. The determination region detection unit 20 detects the determination region 21 corresponding to the cargo bed 60 from the edge image 11. In the example shown in Figure 6, the determination region detection unit 20 detects the second determination region 23 as the determination region 21 within the second edge image 11b. The second determination region 23 is a closed region enclosed by the cargo bed edges 14a, 14b, 14c, and 14d, which are roughly rectangular closed regions formed by the contour lines included in the second edge image 11b.

Furthermore, in the example shown in Figure 7, the determination region detection unit 20 detects the first determination region 22 as the determination region 21 within the first edge image 11a. The first determination region 22 is a closed region enclosed by the cargo bed edges 12a, 12b, 12c, and 12d, within the substantially rectangular closed region formed by the contour lines included in the first edge image 11a.

The cargo bed edges 12a and 14a are contour lines corresponding to the front end 60a of the cargo bed 60, and the cargo bed edges 12b and 14b are contour lines corresponding to the rear end 60b of the cargo bed 60 (see Figure 2). Furthermore, the cargo bed edges 12c and 14c are contour lines corresponding to one side end 60c in the width direction of the cargo bed 60, and the cargo bed edges 12d and 14d are contour lines corresponding to the other side end 60d in the width direction of the cargo bed 60.

The determination area 21 is detected based on the predetermined lengthwise dimension L1 and widthwise dimension W1 of the loading platform 60. For example, the dimensions of the detection determination area 21 may be set based on an identification code pre-assigned to the loading platform 60. In the example shown in Figure 6 or 7, loading platform edges 12a, 12b, 14a, and 14b are edges consisting of a number of pixels corresponding to the predetermined dimension W1 of the loading platform 60. Furthermore, loading platform edges 12c, 12d, 14c, and 14d are edges consisting of a number of pixels corresponding to the predetermined dimension L1 of the loading platform 60.

When the determination area detection unit 20 detects the determination area 21, known pattern detection processes can be used. For example, the determination area 21 may be detected by applying a process such as template matching to the edge image 11. In template matching, patterns identical or similar to a template having a predetermined shape, dimensions, etc., are detected from the image. The dimensions of the template are set based on the dimensions W1 and L1 of the cargo bed 60, which are specified in advance.

The determination area detection unit 20 may track the position or orientation of the determination area 21 over time and associate the determination areas 21 detected from each of the virtual images 5 taken at different times. In the virtual images 5 shown in Figures 4 and 5, due to the left-turning motion of the towing vehicle V1, the connecting angle θ1 at time T1 is larger than the connecting angle θ2 at time T2. Therefore, the position or orientation of the first determination area 22 based on the first virtual image 5a differs from the position or orientation of the second determination area 23 based on the second virtual image 5b. Even in such cases, the determination area detection unit 20 can more reliably compare the first determination area 22 and the second determination area 23 by associating them as corresponding determination areas 21.

In the edge images 11 shown in Figures 6 and 7, the first determination region 22 and the second determination region 23 are associated such that their orientations coincide. For example, the first determination region 22 and the second determination region 23 may be associated by appropriately rotating the first edge image 11a or the second edge image 11b to align their coordinate axes X and Y. Note that the illustrated coordinate axis X corresponds to the length direction of the cargo bed 60, and the coordinate axis Y corresponds to the width direction of the cargo bed 60.

Next, the load collapse detection unit 30 will be described. The load collapse detection unit 30 determines whether or not load collapse has occurred on the loading platform 60 by comparing the edge of the load 61 within the first determination area 22 with the edge of the load 61 within the second determination area 23. More specifically, the load collapse detection unit 30 determines whether or not load collapse has occurred by comparing the first edge 13 in the first determination area 22 with the second edge 15 in the second determination area 23. In the examples shown in Figures 6 and 7, the first edge 13 is the edge 13a corresponding to the front end 61a of the load 61 at time T1. The second edge 15 is the edge 15a corresponding to the front end 61a of the load 61 at time T2. If the load collapse detection unit 30 determines that load collapse has occurred, it may output a second trigger signal indicating the occurrence of load collapse.

The load collapse detection unit 30 may set an edge search range 31 based on the first edge 13. In the example shown in Figure 7, the edge search range 31 is set to an area within a predetermined distance from the center O1 of the first edge 13. In the illustrated example, the center O1 is the midpoint of the line segment constituting the first edge 13. Furthermore, this predetermined distance may be appropriately set according to the resolution of cameras 4a, 4b, etc., or the magnitude of vibration of the load 61 expected in the moving vehicle V. In one embodiment, this predetermined distance may be 5 to 15 cm.

Next, referring to Figures 8 and 9, we will explain the comparison between the first edge 13 and the second edge 15 by the load collapse detection unit 30, and the determination of whether or not a correspondence between the first edge 13 and the second edge 15 is possible. "Correspondence" refers to linking the edges of the load 61 at time T1 with the edges of the load 61 at time T2. For example, a correspondence between the first edge 13 and the second edge 15 indicates that the second edge 15 at time T2 corresponds to the first edge 13 at time T1.

In the example shown in Figure 8, at least a portion of the second edge 15 is included within the edge search range 31. In this case, the load collapse determination unit 30 may determine whether the second edge 15 can be associated with the first edge 13 based on the difference in length between the first edge 13 and the second edge 15. For example, if the absolute value of the difference between the length of the first edge 13 and the length of the second edge 15 is 0 cm or more and less than or equal to a predetermined value, the load collapse determination unit 30 determines that the second edge 15 can be associated with the first edge 13. That is, the load collapse determination unit 30 determines that at least a portion of the second edge 15 that can be associated with the first edge 13 is included within the edge search range 31. Then, the load collapse determination unit 30 associates the first edge 13 with the second edge 15.

On the other hand, the load collapse detection unit 30 determines that load collapse has occurred if the absolute value of the difference in length between the first edge 13 and the second edge 15 exceeds the predetermined value, because the second edge 15 cannot be associated with the first edge 13. In other words, the load collapse detection unit 30 determines that load collapse has occurred if the second edge 15, which can be associated with the first edge 13, is not included within the edge search range 31. The predetermined value can be appropriately set according to the resolution of the cameras 4a and 4b, or the magnitude of vibration of the load 61 expected in the moving vehicle V. In one embodiment, the predetermined value may be between 1 cm and 10 cm.

Furthermore, the load collapse determination unit 30 may determine whether the second edge 15 can be associated with the first edge 13 based on the difference between the orientation of the first edge 13 and the orientation of the second edge 15. For example, if the angle θ3 between the first edge 13 and the second edge 15 is 0 degrees or greater and less than or equal to a predetermined angle, the load collapse determination unit 30 determines that the second edge 15 can be associated with the first edge 13. That is, the load collapse determination unit 30 determines that at least a portion of the second edge 15 that can be associated with the first edge 13 is included within the edge search range 31. Then, the load collapse determination unit 30 associates the first edge 13 and the second edge 15.

On the other hand, the load collapse detection unit 30 determines that load collapse has occurred if the angle θ3 exceeds the predetermined angle, because the second edge 15 cannot be associated with the first edge 13. That is, the load collapse detection unit 30 determines that load collapse has occurred because the second edge 15, which can be associated with the first edge 13, is not included within the edge search range 31. The predetermined angle can be appropriately set according to the resolution of the cameras 4a and 4b, or the magnitude of vibration of the load 61 expected in the moving vehicle V. In one embodiment, the predetermined angle may be between 1 and 45 degrees. The angle θ3 is the smaller of the angles made between the first edge 13 and the second edge 15.

In the example shown in Figure 8, the center O2 of the second edge 15 is located within the edge search range 31 based on the first edge 13. On the other hand, in the example shown in Figure 9, the center O2 of the second edge 15 is located outside the edge search range 31. In the case of Figure 9, the load collapse determination unit 30 may determine that the second edge 15 is not included within the edge search range 31 and therefore cannot be associated with the first edge 13. That is, the load collapse determination unit 30 may determine whether the first edge 13 and the second edge 15 can be associated based on whether the center O2 of the second edge 15 is located within the edge search range 31.

In the example shown in Figure 7, in addition to edge 13a, which is the first edge 13, edges 13b, 13c, and 13d are extracted as contour lines of the cargo 61. In such cases, the edge search range 31 is set for each of edges 13b to 13d in the same way as edge 13a, and it is determined whether an edge of the cargo 61 that can be associated with it is included in the second determination area 23. If any of edges 13b to 13d cannot be associated with an edge of the cargo 61 in the second determination area 23, it is determined that cargo collapse has occurred.

On the other hand, if each of edges 13b to 13d can be associated with any of the edges of the load 61 in the second determination area 23, the load collapse determination unit 30 may associate each of edges 13b to 13d with each of the edges that can be associated with them. For example, in the example shown in Figure 7, if each combination of edge 13b and edge 15b, edge 13c and edge 15c, and edge 13d and edge 15d can be associated, the edges may be associated with each other.

Edges 13b, 13c, and 13d correspond to the rear end 61b, one end 61c, and the other end 61d of the load 61 at time T1, respectively. Similarly, edges 15b, 15c, and 15d correspond to the rear end 61b, one end 61c, and the other end 61d of the load 61 at time T2, respectively.

The cargo collapse detection unit 30 may compare the edges of the cargo 61 in the first detection area 22 with the edges of the cargo 61 in the second detection area 23. If an edge is detected that exists in the second detection area 23 but not in the first detection area 22, it may determine that cargo collapse has occurred.

In the example shown in Figure 10, for example, the cargo 61 has fallen from the loading platform 60, so the first determination area 22 does not include the edge of the cargo 61. On the other hand, the second determination area 23 includes the second edge 15. That is, the second edge 15 of the cargo 61 in the second determination area 23 cannot be associated with the edge of the cargo 61 in the first determination area 22. In such a case, the cargo collapse determination unit 30 determines that cargo collapse has occurred.

Furthermore, in the example shown in Figure 11, for example, due to the horizontal displacement of the load 61 on the loading platform 60, one end 61c of the load 61 is located further outward in the width direction than one end 60c of the loading platform 60. Therefore, edge 13c is not included within the first determination area 22. Thus, even if, for example, each combination of edge 13a and edge 15a, edge 13b and edge 15b, and edge 13d and edge 15d are associated, edge 15c cannot be associated with any edge within the first determination area 22. In such cases, the load collapse determination unit 30 determines that load collapse has occurred. That is, the load collapse determination unit 30 may also determine that load collapse has occurred when at least one edge of the load 61 in the second determination area 23 is not associated with an edge of the load 61 in the first determination area 22.

Furthermore, the controller 2a may also include a notification unit 40. The notification unit 40 has a function to notify the occurrence of a cargo collapse when the cargo collapse detection unit 30 determines that a cargo collapse has occurred. The notification unit 40 may also notify the occurrence of a cargo collapse in response to receiving a second trigger signal from the cargo collapse detection unit. The notification unit 40 may also notify the occurrence of a cargo collapse by a warning sound or a warning display on the display unit 8. For example, the notification unit 40 may emit a warning sound indicating a cargo collapse from the speaker (not shown) of the HMI 7, or it may display characters, figures, symbols, etc., indicating a cargo collapse on the display unit 8. This allows the notification unit 40 to notify the occupants of the towing vehicle V1 or the monitors monitoring the vehicle V at the control facility of the occurrence of a cargo collapse.

Furthermore, the notification unit 40 may perform highlighting to emphasize the occurrence of cargo collapse. The highlighting displays the edge of the cargo 61 that formed the basis for the determination that cargo collapse had occurred, shown on the display unit 8. This highlighting allows the driver assistance device 1a to more clearly notify the driver of the location where cargo collapse occurred within the cargo 61. As shown in Figures 12 and 13, the highlighting may be superimposed on the virtual image 5. This allows the driver assistance device 1a to more clearly indicate the location where cargo collapse occurred within the cargo bed 60. The mode of highlighting is not limited to the illustrated examples; the color tone, thickness, etc., of the highlighted edge may be appropriately set to more clearly notify the driver of cargo collapse.

In the example shown in Figure 12, the notification unit 40 highlights edges of the cargo 61 in the first determination area 22 that are not associated with edges of the cargo 61 in the second determination area 23 by superimposing them onto the virtual image 5. In the illustrated example, edges 13a and 13b are not associated with edges of the cargo 61 in the second determination area 23. Therefore, the notification unit 40 displays edges 13a and 13b on the display unit 8.

In the example shown in Figure 13, the notification unit 40 highlights edges of the cargo 61 in the second determination area 23 that are not associated with edges of the cargo 61 in the first determination area 22. In the illustrated example, edge 15c is not associated with an edge of the cargo 61 in the first determination area 22. Therefore, the notification unit 40 highlights edge 15c by displaying it on the display unit 8.

Next, an example of the operation of the driving support device 1a will be explained with reference to the flowchart in Figure 14. By repeating the processes of steps S101 to S104 illustrated in the figure, it is possible to more reliably detect cargo shifting that occurs while the vehicle V is in motion.

To initiate driver assistance using the driver assistance device 1a, the first step is to connect the towing vehicle V1 and the towed vehicle V2. This connection may be performed within the premises of a manufacturing plant or cargo terminal. Next, the occupant operates the input unit 9 of the HMI 7 to input the identification code assigned to the cargo bed 60. Subsequently, the cargo 61 is placed on the cargo bed 60 of the towed vehicle V2. In response to the operation of the start switch, a first trigger signal is input to the controller 2a, and driver assistance begins. Note that the cargo 61 may already be placed on the cargo bed 60 when connecting the towing vehicle V1 and the towed vehicle V2. Alternatively, the identification code may be input before connecting the towing vehicle V1 and the towed vehicle V2. Furthermore, if the first trigger signal is output in response to the input of the identification code to the input unit 9, the operation of the start switch can be omitted. At any time before or after these operations, a virtual image 5 may be displayed on the display unit 8 of the HMI 7. For example, when an identification code is input to the input unit 9, the first virtual image 5a at time T1 may be displayed on the display unit 8.

In step S101, the edge image generation unit 10 extracts edges from the virtual image 5 acquired from the surrounding image generation device 3 and generates an edge image 11. For example, at time T2, the edge image generation unit 10 generates a second edge image 11b by extracting the second cargo bed edge 14 of the cargo bed 60 and the second edge 15 of the cargo 61 from the second virtual image 5b. At time T1, the edge image generation unit 10 generates a first edge image 11a by extracting the first cargo bed edge 12 of the cargo bed 60 and the first edge 13 of the cargo 61 from the first virtual image 5a. The process then proceeds to step S102.

In step S101, in addition to edge 15a, which is the second edge 15 at time T2, one or more other edges may be extracted. In the example shown in Figure 6, in addition to the second edge 15, edges 15b to 15d are extracted. Also, at time T1, in addition to edge 13a, which is the first edge 13, one or more other edges may be extracted. In the example shown in Figure 7, in addition to the first edge 13, edges 13b to 13d are extracted.

In step S102, the determination region detection unit 20 detects a determination region 21 corresponding to the cargo bed 60 from the edge image 11. For example, at time T2, the determination region detection unit 20 detects a second determination region 23 within the second edge image 11b. Also, at time T1, the determination region detection unit 20 detects a first determination region 22 within the first edge image 11a. The process then proceeds to step S103.

In step S103, the load collapse detection unit 30 determines whether or not load collapse has occurred on the loading platform 60. For example, the load collapse detection unit 30 determines whether or not load collapse has occurred by comparing the first edge 13 and the second edge 15. If it is determined that load collapse has occurred (Yes in step S103), the process proceeds to step S104. The load collapse detection unit 30 may also output a second trigger signal at this time. If it is determined that load collapse has not occurred (No in step S103), the series of processes ends.

In step S104, the notification unit 40 notifies the system of the occurrence of a load collapse. The notification unit 40 may also notify the system of the occurrence of a load collapse in response to receiving a second trigger signal from the load collapse determination unit. For example, the notification unit 40 may notify the system of the occurrence of a load collapse by emitting a warning sound from the speaker of the HMI 7. Furthermore, the notification unit 40 may perform a highlighting display to emphasize the occurrence of the load collapse. For example, the notification unit 40 may superimpose the edges of the cargo 61 that formed the basis for the determination that a load collapse had occurred onto the first virtual image 5a displayed on the display unit 8 of the HMI 7. After step S104, the series of processes ends.

Next, referring to the flowchart in Figure 15, we will explain in more detail the process of determining whether or not a load collapse occurred in step S103.

In step S300, the load collapse detection unit 30 sets an edge search range 31 based on the first edge 13. For example, the edge search range 31 is set to an area within a predetermined distance from the center O1 of the first edge 13. The process then proceeds to step S301.

In step S301, the load collapse determination unit 30 determines whether the second edge 15, which can be associated with the first edge 13, is included within the edge search range 31. The determination in step S301 may be based on the difference in length or orientation between the first edge 13 and the second edge 15. For example, if the difference in length and orientation between the first edge 13 and the second edge 15 is within a predetermined value, the load collapse determination unit 30 determines that the second edge 15, which can be associated with the first edge 13, is included within the edge search range 31 (Yes in step S301). The process then proceeds to step S302. On the other hand, if the difference in length or orientation between the first edge 13 and the second edge 15 exceeds a predetermined value, the load collapse determination unit 30 determines that the second edge 15, which can be associated with the first edge 13, is not included within the edge search range 31 (No in step S301). The process then proceeds to step S305.

In step S302, the load collapse determination unit 30 associates the edges of the load 61 in the first determination area 22, which were determined to be matchable in step S301, with the edges of the load 61 in the second determination area 23. For example, it associates the first edge 13 with the second edge 15. The process then proceeds to step S303.

In step S303, the load collapse determination unit 30 determines whether the determination in step S301 has been performed for all edges of the load 61 within the first determination area 22. For example, if, in step S101 at time T1, in addition to the first edge 13 (edge 13a), edges 13b to 13d were extracted, the unit determines whether the determination in step S301 has been performed for each of edges 13a to 13d. If the determination in step S301 has been performed for all edges (Yes in step S303), the process proceeds to step S304. If there are edges within the first determination area 22 that have not been determined in step S301 (No in step S303), the process returns to step S301, and the determination in step S301 is performed for the undetermined edges.

Furthermore, as shown in the example in Figure 10, if there are no edges of the cargo 61 in the first determination region 22 at time T1, steps S300 to S303 may be omitted.

In step S304, the load collapse determination unit 30 determines whether the edges of the load 61 in the second determination area 23 include any edges that have not been associated in step S302. If all edges of the load 61 in the second determination area 23 have been associated in step S302 (No in step S304), the determination of whether load collapse has occurred is completed. On the other hand, if at least one edge of the load 61 in the second determination area 23 is not associated with an edge of the load 61 in the first determination area 22 (Yes in step S304), the process proceeds to step S305.

In the example shown in Figure 10, the second determination area 23 includes the second edge 15, but the first determination area 22 does not include the edges of the cargo 61. Therefore, the second edge 15 is not associated with the edges of the cargo 61 within the first determination area 22. Thus, the cargo collapse determination unit 30 determines "Yes" in step S304, and the process proceeds to step S305.

Furthermore, if multiple edges of the cargo 61 are extracted in step S101 at time T2, the cargo collapse determination unit 30 determines whether the correspondence in step S302 has been performed for each of those multiple edges. In the example shown in Figure 11, edge 13c is not included in the first determination area 22. Therefore, even if, for example, the combinations of edge 13a and edge 15a, edge 13b and edge 15b, and edge 13d and edge 15d are each associated, edge 15c cannot be associated with any edge within the first determination area 22. Thus, the cargo collapse determination unit 30 determines Yes in step S304, and the process proceeds to step S305.

In step S305, the load collapse determination unit 30 determines that load collapse has occurred on the loading platform 60. It then saves information such as the position, length, and orientation of the edges of the load 61 in the determination area 21, which formed the basis for the determination that load collapse had occurred. For example, as shown in Figure 12, if edges 13a and 13b are not associated with the edges of the load 61 in the second determination area 23, the load collapse determination unit 30 saves information such as the position, length, and orientation of edges 13a and 13b in the first determination area 22. In the example shown in Figure 13, edge 15c in the second determination area 23 is not associated with the edges of the load 61 in the first determination area 22. Therefore, the load collapse determination unit 30 saves information such as the position, length, and orientation of edge 15c in the second determination area 23. After step S305, the determination of whether or not load collapse has occurred is completed.

(Second Embodiment)
Next, with reference to Figure 16, the configuration of the driver assistance device 1b according to the second embodiment will be described. The driver assistance device 1b includes a controller 2b that performs processing necessary for assisting the driving of the vehicle V. Also, similar to the driver assistance device 1a, the driver assistance device 1b is connected to an ambient image generation device 3 and an HMI 7. The controller 2b is a microcomputer equivalent to the controller 2a and has the same functions as the controller 2a, but also includes a vehicle control unit 50. Furthermore, the driver assistance device 1b is connected to a brake drive unit 51.

The driver assistance device 1b can be applied to a vehicle V equipped with an automatic driving control function, for example. The automatic driving control function automatically controls the vehicle's movement based on the surrounding conditions or the vehicle's state, without relying on driver operations such as steering or pedal operation. When vehicle V is equipped with an automatic driving control function, automatic driving is possible, where acceleration, deceleration, and steering are performed automatically without driver intervention. Furthermore, this automatic driving may include cases where vehicle V automatically accelerates, decelerates, and steers without driver intervention in a predetermined driving environment. This predetermined driving environment may be, for example, the premises of a manufacturing plant or a freight terminal.

The vehicle control unit 50 has the function of controlling the brake drive unit 51. This control may be performed in response to a second trigger signal being input from the load collapse detection unit 30 to the vehicle control unit 50. If the load collapse detection unit 30 detects the occurrence of a load collapse, the vehicle control unit 50 may output a third trigger signal to decelerate or stop the towing vehicle V1.

The brake drive unit 51 controls the drive of a brake actuator (not shown) or the like in response to the input of the third trigger signal, thereby decelerating or stopping the towing vehicle V1. Note that the controller 2b may also be connected to a steering drive unit, an accelerator drive unit, etc. (not shown). In that case, the vehicle control unit 50 may have the function of controlling the steering drive unit and the accelerator drive unit. This allows, for example, when the vehicle control unit 50 decelerates or stops the vehicle V, to control the steering or accelerator of the towing vehicle V1, thereby more appropriately controlling the stopping position or timing.

Next, an example of the operation of the driving support device 1b will be described with reference to the flowchart in Figure 17. By repeating the processes of steps S101 to S105 illustrated in the figure, it is possible to more reliably detect cargo collapse that occurs while the vehicle V is in motion. Furthermore, if cargo collapse occurs, the towing vehicle V1 can be slowed down or stopped. Note that steps S101 to S104 are the same as in the first embodiment, so their explanation will be omitted.

In step S105, the vehicle control unit 50 outputs a third trigger signal to decelerate or stop the towing vehicle V1. This controls the brake drive unit 51, causing the towing vehicle V1 to decelerate or stop. The series of processes then ends.

In the illustrated example, step S105 is performed after step S104, but this is not limiting. Steps S104 and S105 may be performed in reverse order or simultaneously. That is, the vehicle control unit 50 may output the third trigger signal before the notification unit 40 notifies of the occurrence of cargo collapse, or the vehicle control unit 50 may output the third trigger signal and the notification unit 40 may notify of the occurrence of cargo collapse simultaneously.

In the above explanation, the determination area 21 is described as being detected based on a pre-entered identification code or the dimensions L1 and W1 of the cargo bed 60, but it is not limited to this. For example, the determination area detection unit 20 may use AI (Artificial Intelligence) that has learned the shape characteristics of the cargo beds of various towed vehicles to identify the area corresponding to the cargo bed from the edge image 11 and detect the determination area 21. In this case, input of the identification code and the values of the dimensions L1 and W1 of the cargo bed 60 via HMI 7 can be omitted. In other words, by detecting the determination area 21 with the determination area detection unit 20 equipped with the AI, it becomes unnecessary to specify the dimensions L1 and W1 based on the input identification code or numerical values. Therefore, the driving support devices 1a and 1b can provide driving support for the vehicle V more easily. In such cases, the HMI 7 does not need to be connected to the controllers 2a and 2b.

Next, the operation and effects of the driving support device and the load collapse detection method according to the embodiment will be described.

(1) The driving support devices 1a and 1b according to the embodiment are driving support devices that detect cargo collapse of cargo 61 loaded on a cargo bed 60 towed by a towing vehicle V1. The devices include: an edge image generation unit 10 that acquires virtual images 5 of the rear of the towing vehicle V1 viewed from above over time and generates edge images 11 from each of the virtual images 5 that include at least the edges of the cargo bed 60 and the cargo 61; a determination region detection unit 20 that detects a determination region 21 corresponding to the cargo bed 60 from the edge images 11; and a cargo collapse determination unit 30 that determines whether or not cargo collapse has occurred. The cargo collapse determination unit 30 determines whether or not cargo collapse has occurred by comparing a first edge 13 of the cargo 61 extracted from a first virtual image 5a of the virtual image 5 with a second edge 15 of the cargo 61 extracted from a second virtual image 5b of the virtual image 5 that was generated before the first virtual image 5a.

According to the driving support devices 1a and 1b of this embodiment, an edge image 11 is generated based on a virtual image 5 of the rear region of the towing vehicle V1 viewed from above. Then, based on the edges of the cargo 61 in the determination region 21 detected from the edge image 11, cargo collapse can be detected. Therefore, the influence of changes in the coupling angle between the towing vehicle V1 and the towed vehicle V2, or objects present around the towed vehicle, can be suppressed when determining cargo collapse. Consequently, the driving support devices 1a and 1b can more reliably detect cargo collapse occurring on the cargo bed 60.

(2) The determination region detection unit 20 tracks the determination region 21 over time and associates the first determination region 22 detected in the first edge image 11a generated from the first virtual image 5a with the second determination region 23 detected in the second edge image 11b generated from the second virtual image 5b. The load collapse determination unit 30 may then determine whether or not a load collapse has occurred by comparing the first edge 13 in the first determination region 22 with the second edge 15 in the second determination region 23.

This allows for a more reliable comparison of the first determination region 22 in the first edge image 11a and the second determination region 23 in the second edge image 11b, even if differences in position or orientation occur due to the passage of time. Therefore, the edges of the cargo 61 in the first determination region 22 and the edges of the cargo 61 in the second determination region 23 can be compared more reliably. Consequently, the driver assistance devices 1a and 1b can more reliably detect cargo shifting that occurs on the cargo bed.

(3) The load collapse determination unit 30 may set an edge search range 31 based on the first edge 13, and determine that a load collapse has occurred if the second edge 15, which can be associated with the first edge 13, is not included in the edge search range 31.

This allows the load collapse detection unit 30 to more reliably compare the first edge 13 and the second edge 15 and determine whether or not load collapse has occurred. Therefore, the driving support devices 1a and 1b can more reliably detect load collapse occurring on the loading platform.

(4) The edge search range 31 may be set to an area within a predetermined distance from the center O1 of the first edge 13.

This makes it possible to suppress errors in the generation of the virtual image 5 when comparing the first edge 13 and the second edge 15, or the effects of vibrations of the cargo 61 caused by the movement of the vehicle V. Therefore, the driver assistance devices 1a and 1b can more reliably detect cargo shifting that occurs on the cargo bed.

(5) The load collapse determination unit 30 may determine that the first edge 13 and the second edge 15 can be matched together when the difference in length and orientation between the first edge 13 and the second edge 15 is within a predetermined value.

This allows the load collapse detection unit 30 to more reliably determine whether the first edge 13 and the second edge 15 can be associated with each other. Therefore, the driving support devices 1a and 1b can more reliably detect load collapse occurring on the loading platform.

(6) The load collapse determination unit 30 may determine that load collapse has occurred if the edge search range 31 includes a second edge 15 that can be associated with the first edge 13, associate the first edge 13 with the second edge 15, and at least one edge of the load 61 in the second determination area 23 is not associated with an edge of the load 61 in the first determination area 22.

For example, the number of edges of the cargo 61 in the first determination area 22 may be less than the number of edges of the cargo 61 in the second determination area 23 due to the cargo 61 falling from the loading platform 60 or horizontal displacement. Even in such cases, the cargo collapse determination unit 30 can still detect cargo collapse. Therefore, the driving support devices 1a and 1b can more reliably detect cargo collapse occurring on the loading platform.

(7) The dimensions of the determination area 21 may be set based on an identification code pre-assigned to the cargo bed 60.

The identification code is assigned to each towed vehicle V2 and is associated with the lengthwise dimension L1 and widthwise dimension W1 of the cargo bed 60. Therefore, the driving support devices 1a and 1b can identify the dimensions L1 and W1 of the cargo bed 60 according to the identification code. This allows the determination area detection unit 20 to detect the determination area 21 more reliably.

(8) The driving support devices 1a and 1b according to the embodiment may include a notification unit 40 that notifies the driver of the occurrence of a load collapse when the load collapse determination unit 30 determines that a load collapse has occurred.

This allows the driver assistance devices 1a and 1b to notify the occupants of the towing vehicle V1 or a supervisor monitoring vehicle V at the control facility of the occurrence of cargo shifting. Therefore, the driver assistance devices 1a and 1b can more reliably support the driving of vehicle V.

(9) The driving support device 1b according to this embodiment includes a vehicle control unit 50 that controls the movement of the towing vehicle V1. The vehicle control unit 50 may output a signal to decelerate or stop the towing vehicle V1 when the cargo collapse detection unit 30 determines that cargo collapse has occurred.

This allows the driver assistance device 1b to decelerate or stop the towing vehicle V1 when it detects a load shift. Therefore, driver assistance can be provided more reliably when a load shift is detected.

(10) The cargo collapse detection method according to the embodiment is a method for detecting cargo collapse of a load 61 loaded on a cargo bed 60 towed by a towing vehicle V1. It involves acquiring virtual images 5 of the rear region of the towing vehicle V1 viewed from above over time, generating edge images 11 from each of the virtual images 5 that include at least the edges of the cargo bed 60 and the load 61, detecting a determination region 21 corresponding to the cargo bed 60 from the edge images 11, and determining whether cargo collapse has occurred by comparing a first edge 13 of the load 61 extracted from a first virtual image 5a of the virtual image 5 with a second edge 15 of the load 61 extracted from a second virtual image 5b of the virtual image 5 that was generated before the first virtual image 5a.

According to the load collapse detection method of this embodiment, an edge image 11 is generated based on a virtual image 5 of the rear region of the towing vehicle V1 viewed from above. Then, load collapse can be detected based on the edges of the cargo 61 in the determination region 21 detected from the edge image 11. Therefore, the influence of changes in the coupling angle between the towing vehicle V1 and the towed vehicle V2, or objects present around the towed vehicle, can be suppressed when performing load collapse detection. Consequently, load collapse occurring on the cargo bed 60 can be detected more reliably.

This disclosure can contribute, for example, to Sustainable Development Goal (SDG) 9, "Build resilient infrastructure, promote inclusive and sustainable industrialization and foster innovation."

Although several embodiments have been described above, it is possible to modify or transform the embodiments based on the above disclosure. All components of the above embodiments, and all features described in the claims, may be individually selected and combined, provided they do not contradict each other.

1a Driving support device 1b Driving support device 5 Virtual image 5a First virtual image 5b Second virtual image 10 Edge image generation unit 11 Edge image 11a First edge image 11b Second edge image 13 First edge 15 Second edge 20 Judgment area detection unit 21 Judgment area 22 First judgment area 23 Second judgment area 30 Cargo collapse judgment unit 31 Edge search range 40 Notification unit 50 Vehicle control unit 60 Cargo bed 61 Cargo V1 Towing vehicle O1 Center

Claims (9)

A driving assistance device that detects cargo shifting on a cargo bed towed by a towing vehicle,
An edge image generation unit acquires virtual images of the rear region of the towing vehicle viewed from above over time, and generates edge images from each of the virtual images that include at least the edges of the cargo bed and the cargo.
A determination region detection unit detects a determination region corresponding to the cargo bed from the edge image,
A load collapse determination unit that determines whether or not the aforementioned load collapse has occurred,
Equipped with,
The determination region detection unit tracks the determination region over time and associates a first determination region detected within a first edge image generated from a first virtual image, which is a virtual image, with a second determination region detected within a second edge image generated from a second virtual image, which is a virtual image generated before the first virtual image.
The cargo collapse determination unit is a driving support device that determines whether or not cargo collapse has occurred by comparing a first edge in the first determination region of the cargo , extracted from the first virtual image , with a second edge in the second determination region of the cargo, extracted from the second virtual image. The driving support device according to claim 1, wherein the load collapse determination unit sets an edge search range based on the first edge, and determines that load collapse has occurred when the second edge that can be associated with the first edge is not included in the edge search range. The driving support device according to claim 2 , wherein the edge search range is set to an area within a predetermined distance from the center of the first edge. The driving support device according to claim 2 or 3, wherein the load collapse determination unit determines that the first edge and the second edge can be associated with each other when the difference in length and orientation between the first edge and the second edge is within a predetermined value. The driving support device according to any one of claims 2 to 4, wherein the load collapse determination unit associates the first edge with the second edge when the edge search range includes the second edge which can be associated with the first edge, and determines that load collapse has occurred when at least one of the edges of the load in the second determination area is not associated with the edge of the load in the first determination area. The dimensions of the determination area are set based on an identification code previously assigned to the cargo bed, according to any one of claims 1 to 5 . The driving support device according to any one of claims 1 to 6 , further comprising a notification unit for notifying the occurrence of a load collapse when the load collapse determination unit determines that a load collapse has occurred. The vehicle control unit controls the movement of the towing vehicle,
The driving support device according to any one of claims 1 to 7, wherein the vehicle control unit outputs a signal to decelerate or stop the towing vehicle when the cargo collapse determination unit determines that the cargo collapse has occurred. A method for detecting cargo collapse in a cargo bed towed by a towing vehicle,
A virtual image of the rear region of the towing vehicle viewed from above is acquired over time, and an edge image including at least the edges of the cargo bed and the cargo is generated from each of the virtual images.
From the aforementioned edge image, a determination region corresponding to the cargo bed is detected,
The determination region is tracked over time, and a first determination region detected in a first edge image generated from a first virtual image (which is a virtual image) is associated with a second determination region detected in a second edge image generated from a second virtual image (which is a virtual image generated before the first virtual image).
A cargo collapse detection method that determines whether or not cargo collapse has occurred by comparing a first edge in the first determination region of the cargo extracted from the first virtual image with a second edge in the second determination region of the cargo extracted from the second virtual image.