JP7666366B2 - Cargo space detection system, car space detection program, car space detection method, and trained model for corner detection - Google Patents

Description

This disclosure relates to a luggage compartment detection system, a luggage compartment detection program, a luggage compartment detection method, and a trained model for corner detection.

For example, the loading and unloading system described in Patent Document 1 discloses a technology that uses a machine learning model to estimate the height of a truck's loading platform from the ground through image analysis. The machine learning model is generated by machine learning using image data of a large number of trucks and the loading platform height from the ground as training data.

JP 2021-116140 A

The inventors of the present application have discovered that when attempting to detect the entire range of the cargo space through image analysis using the above-mentioned machine learning model, the positions of the cargo space in the many images used as training data are misaligned from one another, and the sizes, shapes, and loading order of the luggage loaded in the cargo space vary widely, which causes disturbances that make it difficult to accurately detect the range of the cargo space when analyzing images using a machine learning model.

The present disclosure can be realized in the following forms.
[Form 1] A cargo space detection system for detecting the extent of a cargo space, comprising: an acquisition unit that acquires an image including the entire cargo space by imaging the entire cargo space in a predetermined imaging direction with loaded luggage visible; a corner detection unit that detects at least one of the multiple corners of the cargo space from the image acquired by the acquisition unit using a trained model for corner detection trained by machine learning using teacher image data in which the position of at least one of the multiple corners of the cargo space when viewed in the imaging direction is labeled for the image acquired for machine learning; and a cargo space detection unit that detects the extent of the cargo space using position information of the at least one corner detected by the corner detection unit, wherein the trained model for corner detection is a trained model trained using the teacher image data in which the position of the lower area below the cargo space is labeled in addition to the positions of the corners.

(1) According to one aspect of the present disclosure, there is provided a luggage compartment detection system for detecting a range of a luggage compartment, the luggage compartment detection system including: an acquisition unit that acquires an image including the entire luggage compartment by capturing an image of the entire luggage compartment in a predetermined imaging direction while loaded luggage is visible, a corner detection unit that detects at least one of a plurality of corners of the luggage compartment from the image acquired by the acquisition unit using a trained model for corner detection trained by machine learning using teacher image data in which a position of at least one of a plurality of corners of the luggage compartment as viewed in the imaging direction is labeled for the image acquired for machine learning; and a luggage compartment detection unit that detects the range of the luggage compartment using position information of the at least one corner detected by the corner detection unit.
According to this embodiment of the luggage compartment detection system, corners are detected using a trained model for corner detection that has previously learned the positions of the luggage compartment corners, and the luggage compartment area is detected from the position information of the detected corners. Therefore, compared to a case where a trained model that has learned the luggage compartment itself is used, the influence of disturbances in an image including the entire luggage compartment can be reduced, output errors in the trained model are less likely to occur, and the luggage compartment area can be detected more appropriately.
(2) In the above-described cargo space detection system, the outer shape of the cargo space is rectangular when viewed in the imaging direction, and the trained model for corner detection may be a trained model trained using the teacher image data in which the positions of two corners diagonally opposite each other among the multiple corners of the cargo space are labeled.
According to this form of cargo space detection system, the trained model for corner detection is trained using teacher image data in which the positions of two diagonally opposite corners are labeled, so that the width and height of the cargo space can be calculated accurately with a small amount of learning.
(3) In the cargo compartment detection system of the above aspect, the trained model for corner detection may be a trained model trained using the teacher image data in which the positions of the corners and the positions of a lower region below the cargo compartment are labeled as features in addition to the positions of the corners. According to this aspect of the cargo compartment detection system, the trained model for corner detection labels the positions of the lower region of the cargo compartment as a feature in addition to the positions of the corners, so that the range of the cargo compartment can be detected with higher accuracy.
(4) The luggage compartment detection system of the above aspect may further include a loading rate calculation unit that calculates a loading rate of the luggage in the luggage compartment, and the loading rate calculation unit may calculate the loading rate using the range of the luggage compartment detected by the luggage compartment detection unit and luggage information including information on dimensions of the luggage in the luggage compartment. According to this aspect of the luggage compartment detection system, the loading rate can be accurately calculated using the accurately calculated range of the luggage compartment and the luggage information.
(5) In the luggage space detection system of the above aspect, the luggage information may be detected from the image acquired by the acquisition unit using a trained model for luggage detection that is machine-learned to learn features of the luggage. According to the luggage space detection system of this aspect, luggage information for calculating a load ratio can be suitably detected by the trained model for luggage detection.
(6) In the luggage compartment detection system of the above aspect, when the corner detection unit is unable to correctly detect at least one of the plurality of corners, the acquisition unit may acquire the image again, or the trained model for corner detection may be re-trained. According to the luggage compartment detection system of this aspect, even when the corner is unable to be correctly detected, the acquisition unit acquires the image again, or the trained model for corner detection is re-trained, so that the area of the luggage compartment can be calculated after the corner is correctly detected again.
The present disclosure may be realized in various forms other than the luggage compartment detection system, such as a luggage compartment detection method, a luggage compartment detection program, a non-transitory recording medium storing such a program, a trained model for corner detection, etc.

1 is a block diagram showing a hardware configuration of a luggage compartment detection system according to a first embodiment of the present disclosure. 2 is a block diagram showing a functional configuration of the luggage compartment detection system; 1 is a side view showing a schematic diagram of a vehicle equipped with a luggage compartment; 5 is a flowchart showing a processing procedure of a luggage compartment detection method. 11A and 11B are diagrams for explaining image processing in corner detection; 11A and 11B are diagrams for explaining image processing in luggage compartment detection. 1A and 1B are diagrams for explaining image processing in baggage detection; 10 is a flowchart showing a processing procedure of a luggage compartment detection method in a second embodiment.

A. First embodiment:
A luggage compartment detection system 10 and a luggage compartment detection method according to a first embodiment of the present disclosure will be described with reference to FIGS. 1 to 7. FIG.
A1. Configuration of the luggage compartment detection system 10:
[Hardware configuration of luggage compartment detection system 10]
FIG. 1 is a block diagram showing a hardware configuration of a luggage compartment detection system 10 according to a first embodiment of the present disclosure. The luggage compartment detection system 10 according to the present embodiment is configured to include a mobile terminal 12 having a photographing function, such as a smartphone or a tablet. The luggage compartment detection system 10 according to the first embodiment detects a luggage compartment of a vehicle to be described later and luggage in the luggage compartment, and calculates a loading rate, which is the ratio of luggage to the luggage compartment. The configuration of the vehicle and the details of the loading rate calculation will be described later. In this embodiment, the luggage compartment detection system 10 can be used by starting the application in a state where a dedicated application is installed in advance on the mobile terminal 12.

As shown in FIG. 1, the mobile terminal 12 is provided with a control unit 14. The control unit 14 is configured to include a CPU (Central Processing Unit: processor) 16, a ROM (Read Only Memory) 18, a RAM (Random Access Memory) 20, a storage 22 as a memory unit, a communication interface 24, and an input/output interface 26. Each component is connected to each other so as to be able to communicate with each other via a bus 28.

The CPU 16 is a central processing unit that executes various programs and controls each part. That is, the CPU 16 reads the programs from the ROM 18 or the storage 22, and executes the programs using the RAM 20 as a working area. The CPU 16 controls each of the above components and performs various calculation processes according to the programs recorded in the ROM 18 or the storage 22. The storage 22 stores a luggage compartment detection program.

The storage 22 also stores a trained model for corner detection that detects corners of the luggage compartment from an image that includes the luggage compartment. This trained model for corner detection was generated by machine learning using teacher image data in which the position of at least one of the multiple corners of the luggage compartment when viewed in the imaging direction is labeled for an image acquired for machine learning that includes the entire luggage compartment. In the teacher image data of the first embodiment, the positions of two corners of the luggage compartment that are diagonally opposite each other are labeled. Labeling is a machine learning process that labels a large amount of raw data obtained from an image, and in this embodiment, it means associating a label indicating the position of the corner with the coordinates corresponding to the position of the corner.

Furthermore, the storage 22 stores a trained model for luggage detection that detects luggage from an image of the luggage compartment that contains the luggage. This trained model for luggage detection is a model that has been machine-learned to learn the features of luggage from an image that includes the entire luggage compartment. Examples of the features of luggage include brightness, which differs depending on the material of the luggage, such as iron, plastic, or paper such as cardboard. The trained model for luggage detection can detect the external shapes indicating the dimensions of multiple pieces of luggage based on these features of the luggage.

The ROM 18 stores various programs and various data. The RAM 20 temporarily stores programs or data as a working area. The storage 22 is configured with a HDD (Hard Disk Drive) or SSD (Solid State Drive) and stores various programs including an operating system and various data. In this embodiment, the ROM 18 and the storage 22 store a program for performing the loading rate calculation process and various data including data related to the vehicle.

The communication interface 24 is an interface that allows the luggage compartment detection system 10 to communicate with a server and other devices (not shown), and uses standards such as Ethernet (registered trademark), LTE, FDDI, and Wi-Fi (registered trademark).

The input/output interface 26 is connected to a display screen 30 as a display unit, a microphone 32, a speaker 34, and a camera 36. The display screen 30 is provided on the mobile terminal 12, and displays various information to the user. The display screen 30 also displays image data captured by a camera 36, which will be described later. Note that, as an example, in this embodiment, the display screen 30 is a touch panel type, and is configured so that input can be received by touching the content displayed on the display screen 30.

The microphone 32 and speaker 34 are provided on the mobile terminal 12 and are used when the user makes a call, etc. The microphone 32 can also be used when the user issues voice instructions, and the speaker 34 can also be used when notifying the user by voice. The camera 36 is provided on the mobile terminal 12 and is configured so that image data captured by the camera 36 is displayed on the display screen 30.

[Functional configuration of luggage compartment detection system 10]
The luggage compartment detection system 10 realizes various functions using the above hardware resources. The functional configuration realized by the luggage compartment detection system 10 will be described with reference to Fig. 2. Fig. 2 is a block diagram showing the functional configuration of the luggage compartment detection system 10. Each functional configuration is realized by the CPU 16 reading out a program stored in the storage 22 and executing the program.

2, the luggage compartment detection system 10 is functionally configured to include an acquisition unit 41, a corner detection unit 42, a luggage compartment detection unit 43, a luggage detection unit 44, a loading rate calculation unit 45, and a learning unit 46. The acquisition unit 41 acquires an image including the entire luggage compartment to be detected by the camera 36. This image is obtained by capturing an image of the entire luggage compartment in a predetermined imaging direction with the loaded luggage visible. The corner detection unit 42 detects the corners of the luggage compartment from the image of the luggage compartment acquired by the acquisition unit 41 using a trained model for corner detection stored in the storage 22. The corner detection unit 42 also outputs position information of the detected corners. Details regarding the output of the corner position information will be described later.

The luggage compartment detection unit 43 detects the range of the luggage compartment from the position information of the corners detected by the corner detection unit 42. Specifically, the luggage compartment detection unit 43 has a function of numerically outputting the range of the luggage compartment from the position information of the corners, and further cutting out the range of the luggage compartment from the image as a partial image.

The luggage detection unit 44 detects luggage information in the luggage compartment from the image of the luggage compartment acquired by the acquisition unit 41 using the trained model for luggage detection stored in the storage 22. The luggage information includes information on the dimensions of the luggage in the luggage compartment. The loading rate calculation unit 45 calculates the loading rate of luggage in the luggage compartment. Specifically, it calculates the loading rate using the range of the luggage compartment detected by the luggage compartment detection unit 43 and the luggage information in the luggage compartment. The learning unit 46 creates an AI model using a neural network based on the captured images of corners, luggage, etc. The specific processing of each functional unit will be explained in detail in the luggage compartment detection method described below.

[Configuration of vehicle 50]
3 is a side view showing a vehicle 50 having a luggage compartment 51. Next, a description will be given of the configuration of the vehicle 50 having the luggage compartment 51 that is the detection target of the luggage compartment detection system 10. The vehicle 50 is, for example, a truck that transports a large amount of luggage 52 loaded thereon.

As shown in FIG. 3, the luggage compartment 51 is located behind the driver's cab 58 of the vehicle 50 and is formed on the luggage platform 57. When luggage 52 is being transported, the luggage compartment 51 is covered by side walls 53 and 54 in the front-rear direction (left-right direction in FIG. 3) of the vehicle 50, a side wall 55 on the right side in the direction of travel, a lower door 56 located on the left side in the direction of travel, and an upper door (not shown).

The upper door, for example, is an integrated door with a generally L-shaped cross section formed by the left side wall and a portion of the ceiling, and rotates upward to open the luggage compartment 51. The lower door 56 rotates downward from the bottom position of the loading platform 57 to open the luggage compartment 51. When the lower door 56 and the upper door are open, it is possible to capture an image of the entire luggage compartment 51, including the positions of the four corners of the luggage compartment 51, from the left side of the vehicle 50. In other words, it is possible to obtain an image including the entire luggage compartment 51 while the loaded luggage 52 is visible. In this embodiment, the predetermined imaging direction coincides with the direction in which the luggage compartment 51 is viewed from the left side of the vehicle 50, as shown in FIG. 3. The outer shape of the luggage compartment 51 is rectangular when viewed in the imaging direction.

A2. Luggage compartment detection method by luggage compartment detection system 10:
Next, a luggage compartment detection method executed by the luggage compartment detection system 10 will be described with reference to Fig. 4 to Fig. 7. Fig. 4 is a flowchart showing the processing procedure of the luggage compartment detection method. As shown in Fig. 4, the luggage compartment detection method includes an acquisition step (step 100, hereinafter "step" will be abbreviated as "S"), a corner detection step (S200), a luggage compartment detection step (S300), a baggage detection step (S400), and a loading rate calculation step (S500), and each of these steps is executed in order.

In the acquisition process (S100), a user of the system photographs the entire luggage compartment 51 with the camera 36, and the acquisition unit 41 functions to acquire an image Ia (see FIG. 3; hereinafter, simply referred to as "full image Ia of the luggage compartment 51"). In the corner detection process (S200), the corner detection unit 42 detects corners of the luggage compartment 51 from the full image Ia of the luggage compartment 51 acquired in the acquisition process (S100) (see FIG. 3 and FIG. 5) using a trained model for corner detection.

Figure 5 is a diagram for explaining image processing in corner detection. As shown in Figure 5, in this embodiment, of the four corners of the luggage compartment 51, two corners, the upper left corner Ic1 and the lower right corner Ic2, which are diagonally opposite each other when viewed from the left side, are detected. In Figure 5, the upper left corner Ic1 and the lower right corner Ic2 are illustrated surrounded by a two-dot chain line frame. In this embodiment, "corner" refers not only to the vertices of the four corners of the rectangle representing the luggage compartment 51, but also to a rectangle that includes parts of two line segments that intersect at approximately right angles with these vertices as endpoints.

In the cargo compartment detection process (S300), the cargo compartment detection unit 43 detects the range of the cargo compartment 51 from the position information of the corners Ic1 and Ic2 detected in the corner detection process (S200). In FIG. 5, when the horizontal direction is the X-axis and the vertical direction perpendicular to the X-axis is the Y-axis, the "position information of the corners Ic1 and Ic2" is the "X, Y coordinates of the corner points C1 and C2" shown in FIG. 5. Here, the X, Y coordinates of the upper left corner point C1 are (X1, Y1), and the X, Y coordinates of the lower right corner point C2 are (X2, Y2). In this embodiment, the corner points C1 and C2 are calculated by defining the center coordinates of each corner Ic1 and Ic2 of the rectangle as the corner points C1 and C2. The coordinates of each corner point C1 and C2 are calculated as the "position information of the corners Ic1 and Ic2".

The luggage compartment detection unit 43 extracts the outline of the luggage compartment 51 from the X and Y coordinates of the two diagonal corner points C1 and C2. Specifically, a rectangular image with the upper left corner point C1 (X1, Y1) and the lower right corner point C2 (X2, Y2) as diagonal corners is cut out as the outline of the luggage compartment 51. FIG. 6 is a diagram for explaining image processing in luggage compartment detection, and shows the cut-out luggage compartment image Ir. The luggage compartment image Ir is represented by a rectangular area surrounded by four vertices: the upper left corner point C1 (X1, Y1), the lower right corner point C2 (X2, Y2), the lower left corner point C3 (X1, Y2), and the upper right corner point C4 (X2, Y1).

In the luggage detection process (S400), the luggage detection unit 44 detects luggage information in the luggage compartment 51 from the overall image Ia of the luggage compartment 51 acquired in the acquisition process (S100) using a trained model for luggage detection. FIG. 7 is a diagram for explaining image processing in luggage detection, and shows an image Ib of detected luggage 52. In FIG. 7, the image Ib of each luggage 52 is illustrated surrounded by a two-dot chain line frame. As shown in FIG. 7, in the luggage detection process (S400), the outer shape of each luggage 52 is detected as the image Ib of the multiple luggage 52.

In the loading rate calculation process (S500), the loading rate calculation unit 45 calculates the loading rate based on the luggage compartment image Ir detected in the luggage compartment detection process (S300) and the images Ib of the multiple luggage 52 detected in the luggage detection process (S400). Specifically, the loading rate can be calculated by filling the images Ib of all the multiple luggage 52 loaded in the vehicle with black and dividing the number of filled pixels by the number of pixels in the luggage compartment image Ir.

The trained model for corner detection used in the above-mentioned luggage compartment detection system 10 that detects the range of the luggage compartment 51 can be used for other purposes besides calculating the loading rate, such as for example, determining whether the vehicle 50 is parked in the correct position by comparing the external position of the detected luggage compartment 51 with the parking position of the vehicle 50.

A3. Effects:
(1) According to the luggage compartment detection system 10 and luggage compartment detection method of the first embodiment, the corners Ic1 and Ic2 of the luggage compartment 51 can be detected using a trained model for corner detection that has previously learned the characteristics of the corners Ic1 and Ic2, and the range of the luggage compartment 51 is detected from the position information of the detected corners Ic1 and Ic2 (X, Y coordinates of the corner points C1 and C2). In the case of a trained model that trains the luggage compartment 51 itself, various luggage 52 are stored in the luggage compartment 51 and the loading order is not determined, which causes a disturbance when learning the outer shape of the luggage compartment 51, and the accuracy of outputting the outer shape of the luggage compartment 51 is reduced. In this regard, the corners Ic1 and Ic2 of the luggage compartment 51 are smaller than the entire luggage compartment 51 and are less susceptible to disturbances, so that the range of the luggage compartment 51 can be detected more accurately by using a trained model for corner detection that has learned the corners Ic1 and Ic2 to detect the luggage compartment 51.

(2) In addition, since the loading rate of the luggage 52 is calculated based on the accurately detected range of the luggage compartment 51, the loading rate can be calculated more accurately.

(3) The trained model for corner detection used in the cargo compartment detection system 10 and cargo compartment detection method of the first embodiment described above is generated by machine learning using teacher image data in which the positions of two corners Ic1 and Ic2 of the cargo compartment 51 that are diagonally opposite each other are labeled for an image including the entire cargo compartment 51. By labeling the positions of the two corners that are diagonally opposite each other, the width and height of the cargo compartment 51 can be calculated with high accuracy with a small amount of learning.

B. Second embodiment:
Next, a luggage compartment detection system 10 and a luggage compartment detection method according to a second embodiment of the present disclosure will be described with reference to Fig. 8. Note that the same reference numerals are used for configurations that are substantially the same as those in the first embodiment, and descriptions thereof will be omitted. The system configuration and functional configuration of the luggage compartment detection system 10 of the second embodiment are substantially the same as those of the luggage compartment detection system 10 of the first embodiment.

The second embodiment differs from the first embodiment in that, in addition to the positions of two corners Ic1 and Ic2 of the luggage compartment 51 that are diagonally opposite each other, the entire luggage compartment 51 and the position of the lower door 56 of the luggage compartment 51 are labeled as teacher image data for the trained model for corner detection. The "lower door 56" corresponds to the "lower region" below the luggage compartment 51.

Figure 8 is a flowchart showing the processing procedure of the luggage compartment detection method in the second embodiment. In the second embodiment, instead of the processing in the corner detection step (S200) in the first embodiment, S201, S202, and S203 are executed, and the other processing (S100, S300, S400, and S500) is the same, so the description is omitted.

As shown in FIG. 8, in S201, the corner detection unit 42 detects corners Ic1 and Ic2 (see FIG. 5), the lower door 56, and the luggage compartment 51 from the overall image Ia of the luggage compartment 51 acquired in the acquisition step (S100) using a trained model for corner detection. Next, in S202, the control unit 14 determines whether the detection was correct.

Here, "when not detected correctly" refers to cases where, for example, detection of two corners Ic1 and Ic2 is requested but only one is detected, or there is a discrepancy larger than a preset threshold between the width (or length) of the luggage compartment 51 calculated from the detected corners Ic1 and Ic2 and the width (or length) of the luggage compartment 51 calculated from the detected luggage compartment 51.

If it is determined in S202 that the detection was correct (S202: YES), the process proceeds to S300. On the other hand, if it is determined in S202 that the detection was not correct (S202: NO), the process proceeds to S203, where re-photographing or re-learning is performed. Specifically, when re-photographing, the CPU 16 guides the user to photograph the luggage compartment 51 again with the camera 36. When re-photographing, it is preferable that the photographing position, photographing angle, focus, etc. are appropriately corrected.

When re-learning is performed, the CPU 16 causes the learning unit 46 to acquire various captured images again and re-learn, thereby updating the existing trained model. After the process of S203, the process returns to S100. Note that in S203, first, re-capture is performed, and if corners are still not correctly detected even after re-capturing a predetermined number of times, re-learning may be performed.

The cargo room detection system 10 and cargo room detection method of the second embodiment can achieve the same effects as those of the first embodiment. Furthermore, the cargo room 51 is detected using teacher image data that labels the positions of the two corners Ic1, Ic2 of the cargo room 51 that are diagonally opposite each other, as well as the entire cargo room 51 and the position of the lower door 56 of the cargo room 51. The lower door 56 is an element that is of the same color and is less susceptible to external disturbances. By adding and using such elements, the range and loading rate of the cargo room 51 can be calculated with greater accuracy.

C. Other embodiments:
(C1) In the cargo compartment detection system 10 and cargo compartment detection method of each of the above embodiments, the trained model for corner detection uses teacher image data in which the positions of two corners Ic1 and Ic2 of the cargo compartment 51 that are diagonally opposite each other are labeled for the entire image Ia of the cargo compartment 51, but the position of the corners to be labeled may be one or may be three or more. When the position of only one corner is labeled, the positions of the other corners may be detected by extracting from the corner features, for example, that at corners Ic1 and Ic2, the two sides that make up the cargo compartment 51 intersect at about 90°, and an area of about 270° exists as a portion that does not make up the cargo compartment 51.

(C2) In addition, to detect the positions of other corners, as described in the second embodiment, the width and height of the luggage compartment 51 may be detected by appropriately combining with learning of the outer shape of the luggage compartment 51 and the lower door 56.

(C3) In the above embodiments, the luggage compartment detection system 10 and luggage compartment detection method can be used by activating a dedicated application installed in advance on the mobile terminal 12. Alternatively, the user's smartphone or tablet may be configured to only take an image of the luggage compartment 51 and send the image to a general computer having a processor and memory as a system management terminal, and various programs may be executed in the destination computer to perform processes such as corner detection, luggage compartment detection, and loading rate calculation. In this case, the processing results may be sent back to the user's smartphone or tablet.

(C4) In the above embodiments, the luggage compartment detection system 10 and luggage compartment detection method are intended to detect the luggage compartment 51 of the vehicle 50, but the luggage compartment 51 is not limited to what the vehicle 50 has. The luggage compartment 51 may be, for example, a chute in which parts are appropriately stored and supplied at a production factory for the vehicle 50 parts. If the luggage compartment 51 is a chute, the portion corresponding to the lower door 56 in the above second embodiment can be implemented as the space between the bottom of the chute and the floor on which the chute is placed. This space can be distinguished from the luggage compartment 51 portion of the chute, and therefore can be learned as a feature separate from the luggage compartment 51 portion.

(C5) In the above embodiments, the luggage compartment detection system 10 and luggage compartment detection method have a loading rate calculation unit 45 and perform loading rate calculation, but the luggage compartment detection system 10 and luggage compartment detection method may not have a loading rate calculation unit 45 and may only detect the luggage compartment 51 without calculating the loading rate.

(C6) In the luggage compartment detection system 10 of each of the above embodiments, the acquisition unit 41 acquires an image including the entire luggage compartment to be detected using the camera 36 of the luggage compartment detection system 10, but an image captured by another camera may also be acquired via the communication interface 24.

The present disclosure is not limited to the above-described embodiments, and can be realized in various configurations without departing from the spirit of the disclosure. For example, the technical features in each embodiment corresponding to the technical features in each form described in the Summary of the Invention column can be replaced or combined as appropriate to solve some or all of the above-described problems or to achieve some or all of the above-described effects. Furthermore, if a technical feature is not described as essential in this specification, it can be deleted as appropriate.

10... luggage compartment detection system, 12... mobile terminal, 14... control unit, 16... CPU, 18... ROM, 20... RAM, 22... storage, 24... communication interface, 26... input/output interface, 28... bus, 30... display screen, 32... microphone, 34... speaker, 36... camera, 41... acquisition unit, 42... corner detection unit, 43... luggage compartment detection unit, 44... luggage detection unit, 45... loading rate calculation unit, 46... learning unit, 50... vehicle, 51... luggage compartment, 52... luggage, 53, 54, 55... side wall, 56... lower door (lower area), 57... luggage platform, C1... upper left corner point, C2... lower right corner point, C3... lower left corner point, C4... upper right corner point, Ia... image including entire luggage compartment, Ib... luggage image, Ic1... upper left corner, Ic2... lower right corner, Ir... luggage compartment image

Claims (7)

A luggage compartment detection system for detecting a luggage compartment range,
an acquisition unit that acquires an image including the entire luggage compartment by capturing an image of the entire luggage compartment in a predetermined imaging direction while the loaded luggage is visible;
a corner detection unit that detects at least one of the multiple corners of the luggage compartment from the image acquired by the acquisition unit using a trained model for corner detection that has been machine-learned using teacher image data in which the position of at least one of the multiple corners of the luggage compartment when viewed in the imaging direction is labeled for the image acquired for machine learning; and
a luggage compartment detection unit that detects the range of the luggage compartment by using position information of the at least one corner detected by the corner detection unit;
Equipped with
A cargo compartment detection system, wherein the trained model for corner detection is a trained model trained using the teacher image data in which the position of the lower area below the cargo compartment is labeled in addition to the position of the corners . The outer shape of the luggage compartment is rectangular when viewed in the imaging direction,
The cargo compartment detection system of claim 1, wherein the trained model for corner detection is a trained model trained using the teacher image data in which the positions of two corners diagonally opposite each other among the multiple corners of the cargo compartment are labeled. A loading rate calculation unit is further provided for calculating a loading rate of the luggage in the luggage compartment,
The loading rate calculation unit is
3. The luggage compartment detection system according to claim 1, further comprising: a luggage compartment detection unit that detects a range of the luggage compartment and a luggage information including information on a size of the luggage in the luggage compartment, and the luggage compartment detection unit detects the range of the luggage compartment and calculates the loading rate. The luggage compartment detection system of claim 3 , wherein the luggage information is detected from the image acquired by the acquisition unit using a trained model for luggage detection that has been machine-learned to detect features of the luggage. When the corner detection unit fails to correctly detect at least one of the plurality of corners,
The luggage compartment detection system according to claim 1 or 2, wherein the image is acquired again by the acquisition unit, or re-learning of the trained model for corner detection is performed. A luggage compartment detection program for detecting a luggage compartment range,
a function of acquiring an image including the entire luggage compartment by capturing an image of the entire luggage compartment in a predetermined capturing direction in a state in which the loaded luggage is visible;
a function of detecting at least one of the multiple corners of the luggage compartment from the acquired image using a trained model for corner detection that has been machine-learned using teacher image data in which the position of at least one of the multiple corners of the luggage compartment when viewed in the imaging direction is labeled for the image acquired for machine learning; and
A function of detecting the range of the luggage compartment using position information of the at least one corner detected;
The above is realized by a computer .
A cargo compartment detection program, wherein the trained model for corner detection is a trained model trained using the teacher image data in which the position of the lower area below the cargo compartment is labeled in addition to the position of the corner . A method for detecting a range of a luggage compartment using a luggage compartment detection system, comprising:
The luggage compartment detection system includes an acquisition unit, a corner detection unit, and a luggage compartment detection unit,
an acquisition step of acquiring an image including the entire luggage compartment by capturing an image of the entire luggage compartment in a predetermined imaging direction by the acquisition unit while the luggage is visible;
a corner detection process in which the corner detection unit detects at least one of the multiple corners of the luggage compartment from the image acquired by the acquisition unit using a trained model for corner detection trained by machine learning using teacher image data in which the position of at least one of the multiple corners of the luggage compartment when viewed in the imaging direction is labeled for the image acquired for machine learning;
a luggage compartment detection step of detecting a range of the luggage compartment by the luggage compartment detection unit using position information of the at least one corner detected by the corner detection unit;
Equipped with
A cargo compartment detection method, wherein the trained model for corner detection is a trained model trained using the teacher image data in which the position of the lower area below the cargo compartment is labeled in addition to the position of the corner .

Images