JP7625892B2 - Information processing device, information processing method, information processing system, and program - Google Patents

Description

The present invention relates to an information processing device, an information processing method, an information processing system, and a program.

In recent years, technological developments have been progressing to support the delivery of mail and other packages. Mail delivery services handle mail of various types and sizes. Among these packages, there are packages whose addresses cannot be read by machine and packages that do not meet the standard size, and these packages must be sorted manually. Furthermore, packages that are not of a standard size may not fit within the frame of the shelves prepared for sorting. As a result, there have been limitations to improving the efficiency of operations.

Patent Document 1 discloses a system in which a marker attached to a box prepared for carrying luggage is read by a camera to determine the size and storage position of the marker, thereby identifying the position of the box stored on a shelf, and projecting information corresponding to the box onto the shelf using a projector.

JP 2020-132375 A

However, the configuration of Patent Document 1 assumes that boxes with markers attached in advance are registered in the system, and packages stored in the boxes are then removed. For this reason, it is necessary to prepare boxes with markers attached in advance. This makes it difficult to store packages of a wide variety of shapes and sizes on shelves in real time, such as in mail sorting work. In addition, when packages must be stored in prepared boxes, there is a problem in that there is a limit to the size of the items that can be stored, as the storage area is fixed by the size of the box.

In view of the above, the present invention aims to improve the efficiency of luggage sorting operations.

In order to solve the above problem, the present invention is an information processing device that includes a determination unit that determines a storage location and storage area of addressed luggage from a first image captured of a shelf on which the luggage is placed , and a projection control unit that projects a second image onto the shelf that displays the address of the luggage for each luggage based on the address acquired from the luggage, the storage location, and luggage information that associates the storage area, wherein the determination unit determines the storage area of the luggage to be stored in the storage area based on address information of luggage placed in an adjacent storage area, and the projection control unit projects the second image onto the storage area on the shelf corresponding to the luggage, with a width of the second image corresponding to the width of the storage area, and further the projection control unit highlights and displays the storage area corresponding to luggage having address information identical to the acquired address .

The present invention is also an information processing method in which an information processing device executes the steps of: determining a storage position and storage area of addressed luggage from a first image captured of a shelf on which the luggage is placed ; and projecting a second image onto the shelf, displaying the address of the luggage for each luggage based on luggage information correlating the address acquired from the luggage, the storage position, and the storage area; in the determining step, a storage area for the luggage to be stored in the storage area is determined based on address information of luggage placed in an adjacent storage area; in the projecting step, the second image is projected onto a storage area on the shelf corresponding to the luggage for each luggage, with a width of the second image corresponding to the width of the storage area; and further, in the projecting step, the storage area corresponding to luggage having address information identical to the acquired address is highlighted and displayed .

The present invention also provides an information processing system comprising an information processing device, a camera, and a projection device, wherein the information processing device comprises a determination unit that determines a storage position and storage area of a piece of luggage with an address from a first image captured by the camera of a shelf on which the piece of luggage is placed, and a projection control unit that causes the projection device to project onto the shelf a second image that displays the address of the piece of luggage for each piece of luggage based on luggage information that associates the address, the storage position, and the storage area obtained from the piece of luggage, wherein the determination unit determines the storage area of the piece of luggage to be stored in the storage area based on address information of the piece of luggage placed in an adjacent storage area, and the projection control unit projects the second image onto the storage area on the shelf corresponding to the piece of luggage, with a width of the second image corresponding to the width of the storage area for each piece of luggage, and further the projection control unit highlights and displays the storage area corresponding to the piece of luggage having the same address information as the obtained address .

The present invention is also a program that causes a computer of an information processing device to function as a determination unit that determines a storage position and storage area of addressed luggage from a first image of a shelf on which the luggage is placed , and a projection control unit that projects a second image onto the shelf, displaying the address of the luggage for each luggage based on luggage information that associates the address acquired from the luggage, the storage position, and the storage area, causes the determination unit to determine the storage area of the luggage to be stored in the storage area based on address information of the luggage placed in an adjacent storage area, causes the projection control unit to project the second image onto the storage area on the shelf corresponding to the luggage for each luggage, with a width of the second image corresponding to the width of the storage area, and further causes the projection control unit to highlight and display the storage area corresponding to the luggage having the same address information as the acquired address .

The present invention can improve the efficiency of luggage sorting operations.

1 is an overall configuration diagram of a support system according to a first embodiment. FIG. 13 is a flowchart of a process when storing luggage. FIG. 13 is an explanatory diagram of a storage determination. FIG. 4A is an explanatory diagram of the projection of a region image, and FIG. 4B is an explanatory diagram of the update of a storage region. 1A and 1B are diagrams for explaining storage area integration, in which FIG. 1A shows the storage area before integration and FIG. 13 is a flowchart of a process for integrating storage areas. FIG. 13 is an explanatory diagram for storing long and narrow luggage. 13 is a flowchart of a process when sorting luggage. FIG. 13 is an explanatory diagram of a case where an address to be sorted is selected. 13 is a flowchart of a storage location determination process. FIG. 13 is an explanatory diagram for a case where the hand position determination is NG. FIG. 13 is an explanatory diagram of a case where the hand position determination is OK. FIG. 13 is an overall configuration diagram of a support system according to a second embodiment. This is an explanatory diagram showing how the overall shape of a shelf can be grasped using AR glasses. FIG. 11 is an explanatory diagram of a display of a region image according to the second embodiment.

The support system of each embodiment displays information on a sorting shelf used to prepare packages for delivery and manages storage locations for packages. Packages are, for example, mail, but are not limited to this.
The support system is a specific example of an "information processing system" in the claims.
In describing each embodiment, the front of the shelf is referred to as the "front", and the up, down, left, right and rear directions are as shown in the figures.

[First embodiment]
<Configuration>
1, a support system 100 of the first embodiment is provided for a shelf 1. The support system 100 includes two cameras 2, a reading device 3, a tablet terminal 4, a projector 6, and a processing device 8. The camera 2, the reading device 3, the tablet terminal 4, the projector 6, and the processing device 8 are connected to each other so as to be able to communicate with each other via wire or wirelessly.

The shelf 1 is a sorting shelf for storing luggage 20. As shown in Figures 1 and 2, the shelf 1 includes a plurality of plates 11 on which the luggage 20 is placed, posts 12 for holding the plates 11 at a predetermined height, and an arm 13 on which the camera 2 is attached.
The package 20 is wrapped and has a destination address written on it (a label with the destination address written on it is attached to the package). The destination address may be written, for example, in a bar code or in text format.

A label 5 is affixed to the front surface of the plate 11. The label 5 is a mark that identifies the size and position of the luggage 20 stored on the shelf 1. For example, the pattern of the label 5 can be, but is not limited to, a pattern in which two or more colors are alternately arranged at intervals of 1 cm (centimeter).

Camera 2 is attached to the front end of arm 13, and the lens is directed toward the front surface of shelf 1. Thus, camera 2 can capture an image of the entire front surface of shelf 1. In particular, camera 2 can capture an image of label 5 affixed to the front surface of plate 11.
The image captured by the camera 2 is a specific example of a "first image" in the claims.

The reading device 3 is a device used by a worker to read the destination address written on the package 20. The reading device 3 can be, for example, a camera or a barcode reader, but is not limited to these.

The tablet terminal 4 is a mobile terminal used by the worker, into which the worker inputs specific information and into which specific information is displayed to the worker. For example, the worker can input an address from the tablet terminal 4, and perform operations when storing the luggage 20 and when sorting the luggage 20.

The projector 6 projects a predetermined image onto the front surface of the shelf 1. The projector 6 is a specific example of a "projection device" in the claims.
The mirror 7 is a light reflector that adjusts the projection location of the projector 6 .
1 and 2, for example, the projector 6 is disposed on the topmost shelf 1 and in front of the shelf 1. The mirror 7 is disposed substantially above the projector 6. When the projector 6 projects a predetermined image onto the mirror 7, the predetermined image is projected onto the front surface of the shelf 1, the front surface of the plate 11, by reflection from the mirror 7. The predetermined image may be projected onto the remaining area of the front surface of the plate 11 excluding the area of the label 5 affixed to the front surface of the plate 11. Furthermore, the predetermined image may be projected so as to overlap the label 5, as long as the visibility of the image itself is good.
It is also possible for the projector 6 to directly project a predetermined image onto the front surface of the shelf 1 without using the mirror 7 .

The processing device 8 controls the camera 2 and projector 6, as well as the information display on the tablet terminal 4.

The tablet terminal 4 and the processing device 8 are configured as a computer having a CPU (Central Processing unit), memory, a storage means (storage unit, storage device) such as a hard disk, and a network interface. Various functions of this computer are realized by the CPU executing programs loaded into the memory.

The processing device 8 includes a determination unit 81, a calculation unit 82, a projection control unit 83, and a notification unit 84. The processing device 8 also stores a parcel information DB 85.

The judgment unit 81 makes various judgments based on the images captured by the camera 2 when a worker stores luggage 20 on the shelf 1 or removes luggage 20 from the shelf 1.

The calculation unit 82 calculates the storage area for the luggage 20 stored on the shelf 1 based on the label 5 in the image captured by the camera 2. The storage area is the area occupied by the luggage placed on the board 11. For example, the calculation unit 82 can set the dimensions of the storage area to be the same as the width dimension (placing width) of the luggage 20. The calculation unit 82 can also calculate one storage area for multiple luggage 20. The calculation unit 82 can also calculate the storage area for luggage 20 that has already been stored on the shelf 1. Furthermore, by making the interval between the colors used in the label 5 smaller, the calculation accuracy of the storage area for the stored luggage 20 by the calculation unit 82 can be improved.

The projection control unit 83 controls the projection of the projector 6 and generates and displays the image to be projected on the front of the shelf 1.

The notification unit 84 issues a predetermined notification to the worker. For example, the notification unit 84 issues a notification to alert the worker to an error in storing or retrieving the luggage 20. The notification unit 84 can also issue a notification by, for example, a notification means (not shown) arranged in the work area or via the tablet terminal 4.

The luggage information DB 85 is a database that registers information about the luggage 20 stored on the shelf 1 as luggage information. The luggage information can include, but is not limited to, for example, the address of the destination written on the luggage 20 (hereinafter, may be referred to as the "luggage address"), the storage position of the luggage 20 stored on the shelf 1, and the storage area of the luggage 20 stored on the shelf 1. The storage position can indicate, for example, which position on which level of the board 11 the luggage is placed.
Furthermore, the package information DB 85 can register package information for each package 20 .
The projection control unit 83 can include the address of the destination of the package information in the image projected on the front surface of the shelf 1. The address projected on the front surface of the shelf 1 may be called a "shelf address."

<Processing>
Next, the processing by the processing device 8 will be described.

(When luggage is stored)
A case where a worker stores a baggage 20 on the shelf 1 will be described.

As shown in FIG. 3, the processing device 8 acquires the address of the luggage 20 to be stored (step A1). Specifically, the worker uses the reading device 3 to photograph the address written on the luggage 20 or to read the barcode written on the luggage 20, so that the reading device 3 reads the luggage address and transmits it to the processing device 8. The worker also uses the tablet terminal 4, which displays a list of addresses, to select a desired address, and the tablet terminal 4 transmits the selected address to the processing device 8 as the luggage address. The worker attempts to store the luggage 20 in a desired position on the shelf 1. The desired position may be an empty position, or may be a position where other luggage 20 is already stored (due to stacking). The camera 2 photographs the luggage 20 to be stored.

Next, the processing device 8 acquires a real-time image of the worker storing the luggage from the camera 2 (step A2). As shown in FIG. 4, when the worker stores the luggage 20, part of the label 5 is hidden by the luggage 20. The camera 2 can capture an image of the partially hidden label 5. The processing device 8 can acquire an image of the partially hidden label 5 from the camera 2.

Next, the determination unit 81 of the processing device 8 performs a storage determination process (step A3). Specifically, the determination unit 81 identifies the storage position of the luggage 20 from the image of the partially hidden label 5. The calculation unit 82 can also calculate the width dimension of the luggage 20 from the image of the partially hidden label 5. Thus, the determination unit 81 identifies a storage area for the luggage 20 that has the same dimensions as the calculated width dimension.

Next, the determination unit 81 determines whether the identified storage location has already been registered (step A4). Specifically, the determination unit 81 refers to the luggage information DB 85 to determine whether there is another piece of luggage 20 that is located in the same position as the identified storage location. If it has not been registered (No in step A4), the identified storage location is originally vacant, and the determination unit 81 creates a new storage area for the luggage 20 to be stored (step A5). At this time, the dimensions of the created storage area are the dimensions calculated by the calculation unit 82, and can be the same as the width dimension of the luggage 20 to be stored.

Next, the determination unit 81 registers the package information about the package 20 to be stored in the package information DB 85 (step A6). The package information includes the destination address of the package 20, the storage location of the package 20, and the storage area of the package 20.

Next, the projection control unit 83 of the processing device 8 generates an area image including the shelf address (step A7). Specifically, the projection control unit 83 generates an area image to be projected onto the front surface of the shelf 1, and includes the destination address included in the package information registered in step A6 in the generated area image.
The area image is a specific example of a "second image" in the claims.

Next, the projection control unit 83 executes a projection display process using the projector 6 (step A8). Specifically, the projection control unit 83 controls the projector 6 to display an area image on the front surface of the shelf 1 in a storage area included in the luggage information of the luggage 20 to be stored. As shown in FIG. 5(a), the area image 30 is displayed on the front surface of the plate 11 of the shelf 1 in the storage area for the luggage 20 (having dimensions the same as the width dimension of the luggage 20). This ends the process of FIG. 3.

On the other hand, if the identified storage location has not been registered (Yes in step A4), other luggage 20 has been placed at the identified storage location, and luggage is about to be stacked. Here, the determination unit 81 determines whether the luggage address of the luggage 20 to be stored matches the shelf address of the other luggage 20 that has already been placed (step A9). The shelf address of the other luggage 20 that has already been placed is an address included in the luggage information of the other luggage stored in the luggage information DB 85, and is an address displayed in the storage area of the other luggage 20 that has already been placed by the projector 6.

If there is no match (No in step A9), the notification unit 84 performs error processing (step A10). Specifically, the notification unit 84 issues a warning to prohibit the placement of the luggage 20 to be stored in the identified storage position determined in step A4. After that, when the worker stores the luggage 20 in another position on the shelf 1, the process returns to step A2 and continues the process in FIG. 3.

On the other hand, if there is a match (Yes in step A9), the determination unit 81 determines whether the luggage 20 to be stored is larger than the storage area of the other luggage 20 that has already been placed (step A11). The storage area of the other luggage 20 that has already been placed is obtained from the luggage information of the other luggage registered in the luggage information DB 85. If the luggage 20 to be stored is larger (Yes in step A11), the determination unit 81 updates the storage area for the luggage 20 to be stored and the other luggage 20 that has already been placed (step A12). Specifically, the storage area of the other luggage 20 that has already been placed is expanded to be the same as the width dimension of the luggage 20 to be stored.

As shown in Figure 5 (b), when a larger piece of luggage 20 to be stored is stacked on top of luggage 20 that has already been placed (see the second board 11 from the bottom and the third board 11 from the bottom), the dimensions of the storage area for the already placed luggage 20 are enlarged to be the same as the storage area for the luggage 20 to be stored, and an area image 30 corresponding to the enlarged storage area is projected.
The stacked luggage 20 are specific examples of the "first luggage" and the "second luggage" in the claims, respectively.
After that, the process returns to step A6, and the determination unit 81 registers the baggage information about the baggage 20 to be stored in the baggage information DB 85, and also registers the baggage information including the expanded storage area about the other baggage 20 that has already been placed in the baggage information DB 85. Then, the process of FIG. 3 is continued.

On the other hand, if the luggage 20 to be stored is not large (No in step A11), the determination unit 81 does not process any other luggage 20 that has already been placed, and returns to step A6, where it registers luggage information about the luggage 20 to be stored in the luggage information DB 85. Then, it continues the processing in FIG. 3.

As shown in FIG. 5(b), when a smaller piece of luggage 20 to be stored is stacked on top of an already placed piece of luggage 20 (see the bottom board 11), an area image 30 corresponding to the original storage area is projected without enlarging the dimensions of the storage area for the already placed luggage 20.
The process of FIG. 3 is executed for each piece of luggage 20 to be stored.

(Storage area integration)
When storage areas for multiple packages 20 with the same address are adjacent to each other, the processing device 8 can consolidate the storage areas into one.
As shown in Fig. 6(a), a storage area is set for each of the two pieces of luggage 20 placed on the bottom board 11 of the shelf 1, and an area image 30 is projected onto the storage area. If the distance between the two pieces of luggage 20 is sufficiently small, the processing device merges the two storage areas into one. As a result, as shown in Fig. 6(b), one storage area is set for the two pieces of luggage 20, 20 on the bottom level, and one area image 30 is projected onto the storage area.

On the other hand, the space between the two pieces of luggage 20 placed on the board 11 on the first level from the bottom of the shelf 1 is large enough that the storage areas are not consolidated.
By consolidating storage areas, the luggage information for the group of luggage 20 stored on shelf 1 can be simplified.
The packages 20 that are the subject of storage area consolidation are specific examples of the "third package" and the "fourth package" in the claims.

The process of integrating storage areas when storing new luggage 20 to be stored on shelf 1 will be described. The processing device 8 refers to the registered luggage information of the luggage 20 to be stored (see step A6 in FIG. 3) and knows the storage location of the luggage 20 to be stored.

As shown in FIG. 7, the determination unit 81 of the processing device 8 refers to the luggage information DB 85 to check the storage area of the luggage 20 to be stored and the storage areas adjacent to it (step B1). "Adjacent storage areas" refers to the storage areas of luggage 20 on the same level of the plate 11 on which the luggage 20 to be stored is placed. Note that the check in step B1 can be performed for each level of the shelf 1.

Next, the determination unit 81 of the processing device 8 determines whether the address (baggage address) of the luggage 20 to be stored is the same as the address (shelf address) of another luggage 20 having an adjacent storage area (step B2). If they are not the same (No in step B2), the storage areas are not integrated, and the projection control unit 83 of the processing device 8 performs projection display processing of a new area image for the luggage 20 to be stored following step A8 (Figure 3) (step B3). Thereafter, the processing in Figure 7 ends.

On the other hand, if the address is the same (Yes in step B2), the determination unit 81 determines whether the distance between the storage area of the luggage to be stored and the adjacent storage area is equal to or less than a predetermined value (a sufficiently small value) (step B4). If it is not equal to or less than the predetermined value (No in step B4), the process proceeds to step B3, where a projection display process of a new area image is performed.

On the other hand, if it is equal to or less than the predetermined value (Yes in step B4), the determination unit 81 merges the storage area of the luggage 20 to be stored with the adjacent storage area (step B5). Note that the luggage information of the luggage 20 to be stored and the luggage information of other luggage 20 having adjacent storage areas are updated for the merged storage area.

Next, the projection control unit 83 performs a projection display process of one area image 30 onto the integrated storage area (step B6). After that, the process of FIG. 7 ends.

(When storing unstable luggage)
For example, if the luggage 20 has an elongated shape and is unstable when placed on the plate 11 of the shelf 1, the storage area for the luggage 20 may not be determined, which may result in a loss of reliability of the luggage information. In this case, it is advisable to prepare a pair of partition plates 40, 40 that can be attached to the shelf 1, as shown in FIG.

The partition plates 40, 40 can partially divide the space of one level of the shelf 1. By storing the luggage 20 in the partial space between the partition plates 40, 40, the luggage 20 can be leaned against the partition plates 40, 40, and the luggage 20 can be stably placed on the plate 11 of the shelf 1. In addition, marks 41, 41 are attached to the front of the partition plates 40, 40. The processing device 8 can grasp the position of the partition plates 40, 40 by capturing the marks 41, 41 in the image captured by the camera 2. As shown in FIG. 8, it is preferable that the marks 41, 41 overlap the label 5 when the partition plates 40, 40 are attached to the shelf 1. This makes it easier to ensure consistency between the judgment of the luggage 20 based on the marks 41, 41 and the judgment of the luggage 20 based on the label 5.

The determination unit 81 of the processing device 8 can grasp the positions of the partitions 40, 40, and can therefore recognize the storage area between the partitions 40, 40. Thus, the determination unit 81 can determine that the storage area of the luggage 20 placed between the partitions 40, 40 is the storage area between the partitions 40, 40. The storage area included in the luggage information of the luggage 20 can be the storage area between the partitions 40, 40. Since the luggage 20 is placed stably between the partitions 40, 40, the storage area of the luggage 20 can be determined, and the reliability of the luggage information can be ensured.

The partition plates 40, 40 are movable, and the storage area between the partition plates 40, 40 can be changed as appropriate.
Furthermore, as shown in FIG. 8, a plurality of packages 20 having the same address can be placed side by side in the storage area between the partition plates 40, 40.

(When sorting luggage)
A case where a worker takes out baggage 20 from shelf 1 as part of a sorting operation will be described.

As shown in FIG. 9, the processing device 8 acquires the addresses of the items to be sorted (step C1). Specifically, the worker selects the desired address using the tablet terminal 4 that displays a list of addresses, and the tablet terminal 4 transmits the selected address to the processing device 8.

The projection control unit 83 of the processing device 8 performs a highlighted projection display process of the acquired address (step C2). Specifically, the projection control unit 83 causes the projector 6 to light up and display the acquired address on the front of the shelf 1 in the storage area of the luggage 20 stored on the shelf 1 that has the same shelf address as the address acquired in step C1. This lighted display may be appropriately highlighted to make it easier for the worker to understand.

10, for example, assume that the worker selects "XX 1-chome" from the tablet terminal 4. In this case, the projector 6 can light up and display an area image 30 with the description "XX 1-chome" in the storage area of the baggage 20 whose shelf address is "XX 1-chome."
Furthermore, the worker can select multiple addresses from the tablet terminal 4, and the projector 6 can light up and display the selected multiple addresses on the front surface of the shelf 1. After this, the worker picks up the packages 20 from the addresses that are lighted up.

Next, the processing device 8 obtains a real-time image of the worker removing the luggage from the camera 2 (step C3). When the worker removes the luggage 20 that is indicated as being lit, the camera 2 can capture the worker's hand that has entered the storage area for the luggage 20. The camera 2 can also capture the label 5 that is partially hidden by the hand.

Next, the determination unit 81 of the processing device 8 performs a removal position determination process (step C4). Specifically, the determination unit 81 determines whether the removal position of the luggage 20 is correct or not from the image of the label 5 partially hidden by the hand.

If the removal position is not correct (No in step C5), the notification unit 84 of the processing device 8 performs error processing (step C6). Specifically, the notification unit 84 issues a warning to the worker that he or she is removing the wrong luggage 20. If the worker then attempts to remove another luggage 20 stored on the shelf 1, the process returns to step C2 and continues the process in FIG. 9.

On the other hand, if the removal position is correct (Yes in step C5), the worker has properly removed the corresponding luggage 20. At this point, the processing device 8 deletes the luggage information of the removed luggage 20 from the luggage information DB 85 (step C7). The camera 2 can capture an image of the empty state of the shelf 1 after the luggage 20 has been removed. The determination unit 81 of the processing device 8 determines that removal of the luggage 20 has been completed from the captured image, and deletes the luggage information.
If there are multiple packages 20 to be removed, steps C2 to C7 are repeated.

After the worker has removed all of the relevant packages 20 (packages 20 with illuminated addresses), the processing device 8 acquires a sorting end operation from the worker (step C8). Specifically, when the worker selects the sorting end button on the tablet terminal 4, the processing device 8 acquires a sorting end signal and ends the processing of FIG. 9.

(Storage location determination)
When a worker stores or retrieves luggage 20, the processing device 8 can determine a specific space on the shelf 1 as a storage location and determine whether the worker has put his or her hand in or taken out of that storage location.

As shown in FIG. 11, the processing device 8 acquires a storage location designated by, for example, a worker (step D1). Specifically, the worker can designate a storage location by touching a desired location on the entire image of the shelf 1 displayed on the tablet terminal 4. The designated storage location is transmitted to the processing device 8. After this, the worker puts his or her hand into the desired storage location on the actual shelf 1.

Next, the processing device 8 acquires a real-time image of the worker inserting his/her hand from the camera 2 (step D2). The camera 2 can capture an image of the inserted hand and the label 5 that is partially hidden by the hand.

Next, the determination unit 81 of the processing device 8 performs hand position determination processing (step D3). Specifically, the determination unit 81 determines whether or not the hand insertion position matches the specified storage location from the image of the label 5 partially hidden by the hand.

If it does not match the specified storage location (No in step D4), the notification unit 84 of the processing device 8 performs error processing (step D5). Specifically, the notification unit 84 issues a warning to the worker that he has inserted his hand in the wrong position (for example, an announcement that "the storage location is wrong"). If the worker then tries to insert his hand in another location on the shelf 1, the process returns to step D2 and continues the process in FIG. 11.

For example, as shown in FIG. 12, when a worker retrieves a package 20 with address A (shelf address displayed in area image 30) in a specified storage location, the worker may approach the package 20 next to address B (shelf address displayed in area image 30) as shown in FIG. 12(a), or approach a location far away from the specified storage location as shown in FIG. 12(b). In this case, it will not match the specified storage location, and the notification unit 84 will perform error processing.

On the other hand, if it matches the specified storage location (Yes in step D4), it means that the worker has stored the luggage 20 in the specified storage location or removed the luggage 20 from the specified storage location.

For example, as shown in FIG. 13, when a worker retrieves a package 20 with address A (shelf address displayed in area image 30) in a specified storage location, the worker may bring his/her hand close to the package 20 with address A as shown in FIG. 12(a), or bring his/her hand close to a location slightly away from the specified storage location as shown in FIG. 12(b). In this case, the specified storage location is considered to match.

Then, the processing device 8 updates the luggage information of the target luggage 20 in the luggage information DB 85 (step D6). Specifically, when the luggage 20 is stored, the luggage address, storage area, and the luggage width dimension calculated by the calculation unit 82 are associated and registered for the target luggage 20. When the luggage 20 is removed, the luggage information of the target luggage 20 is deleted from the luggage information DB. This ends the processing of FIG. 11.

According to the first embodiment, the processing device 8 can display to the worker an image display in which the storage location and address of the luggage 20 are associated. Therefore, when sorting, it is possible to assist the worker in picking up the luggage 20, and it is possible to improve the efficiency of the luggage sorting work.
In addition, since the storage area is determined based on the size of the luggage 20, a sorting operation can be realized that is not dependent on the size or quantity of the luggage 20.
In addition, since the storage area can be updated as the luggage 20 is piled up, the efficiency of managing luggage 20 with the same address can be improved.
In addition, by consolidating the storage areas for adjacent packages 20, the efficiency of managing packages 20 with the same address can be improved.
Furthermore, by providing the partition plate 40, it is possible to reliably store luggage 20 having an unstable shape when placed on the shelf 1, and the efficiency of managing the luggage 20 can be improved.
Furthermore, by using the label 5, it is possible to reliably determine whether the package 20 has been stored on the shelf 1. In particular, since the package 20 can be stored in any vacant location, the area of the shelf 1 can be effectively utilized, and space can be saved.
Furthermore, by using the label 5, it is possible to reliably determine whether the package 20 has been removed from the shelf 1.
Furthermore, by using a projection device such as the projector 6, an image display correlating the storage location and address of the luggage 20 can be displayed in an easy-to-understand manner to the worker.

[Second embodiment]
In describing the second embodiment, differences from the first embodiment will be mainly described, and overlapping descriptions will be omitted.
In the support system 200 of the second embodiment, an augmented reality (AR) glass 9 is used as a projection device instead of the projector 6 and the mirror 7. Also, a plurality of cords 51 are attached to the front surface of the shelf 1.

The AR glasses 9 are wearable terminals used by workers and can display specified information to the worker wearing the AR glasses 9. The camera 2, the reading device 3, the tablet terminal 4, the processing device 8, and the AR glasses 9 (a computing device (not shown) provided in the AR glasses 9) are connected to each other so as to be able to communicate with each other via wire or wirelessly. The AR glasses 9 include a camera 91, a prism 92, and a lens 93.

The camera 91 is attached to the AR glasses 9. The camera 91 can capture an image of the foreground that is equivalent to the field of vision of a worker.
The prism 92 is a light guiding means for displaying a predetermined image on the lens 93 .
The lens 93 is a display medium on which information output by the arithmetic unit of the AR glasses 9 is projected via the prism 92. The information to be displayed is equivalent to the image projected by the projector 6 in the first embodiment.

The codes 51 are affixed to the front of the support posts 12 of the shelf 1 and to the left and right of each of the labels 5. The codes 51 may be, for example, but are not limited to, QR codes. Each of the codes 51 includes information describing its location on the shelf 1.
The processing device 8 can control the AR glasses 9 and the camera 91. However, if the AR glasses 9 have sufficient calculation and control functions, the control by the processing device 8 may not be necessary.

15 , the camera 91 captures an image of the code 51 and reads the information of the code 51, thereby being able to grasp the overall shape of the shelf 1. Thus, the AR glasses 9 can grasp each position and each area of the shelf 1 by analyzing the image captured by the camera 91.
In addition, the processing device 8 can transmit the luggage information registered in the luggage information DB 85 to the AR glasses 9. As a result, as shown in Fig. 16, when the worker looks at the shelf 1, the AR glasses 9 can project an area image 30 indicating the shelf address and storage area for each luggage 20 stored on the shelf 1 onto the lens 93 on which the shelf 1 is displayed, in the same manner as in the first embodiment.
By replacing the projection by the projector 6 with the projection by the AR glasses 9, the various processes described in the first embodiment can also be executed in the second embodiment.

According to the second embodiment, the same effects as those of the first embodiment can be obtained.
Furthermore, by using the AR glasses 9, the projector 6 is no longer necessary, and the task of installing the projector 6 and the mirror 7 on the shelf 1 is no longer necessary, allowing the support system 100 to be reduced in size.
In addition, the accuracy of grasping the storage area can be improved by using the camera 91 of the AR glasses 9. In addition, the camera 91 can read the address of the baggage 20 instead of the reading device 3.

<Modification>
(a) The width of the area image 30 for each piece of luggage 20 stored on the shelf 1 does not need to be the same as the width of the storage area for the luggage 20, and may be larger or smaller than the width of the storage area. In other words, the width of the area image 30 may be a width corresponding to the width of the storage area for the luggage 20.
(b) Instead of using the label 5, an application for measuring the dimensions of an object in a captured image may be installed in the processing device 8. By using such an application, the width dimension of the baggage 20 in the image can be measured and the storage area can be specified.
(c) Means that can be realized in software can be realized in hardware, and means that can be realized in hardware can be realized in software.

100, 200 Support system 1 Shelf 2 Camera 3 Reading device 4 Tablet terminal 4
5 Label 51 Code 6 Projector 7 Mirror 8 Processing device 81 Determination unit 82 Calculation unit 83 Projection control unit 84 Notification unit 85 Baggage information DB
9 AR glass 91 Camera 92 Prism 93 Lens

Claims (11)

a determination unit that determines a storage location and a storage area of the addressed package from a first image of a shelf on which the addressed package is placed;
a projection control unit that projects a second image onto the shelf, the second image displaying the address of each of the packages based on package information that associates the address acquired from the package, the storage location , and the storage area;
Equipped with
The determination unit determines a storage area for the luggage to be stored in the storage area based on address information of luggage placed in the adjacent storage area,
the projection control unit projects, for each of the packages, the second image onto a storage area of the shelf corresponding to the package with a width of the second image corresponding to a width of the storage area;
Furthermore, the projection control unit displays the storage area corresponding to the package having the same address information as the acquired address in an emphasized manner.
Information processing device. The load includes a first load and a second load;
2. The information processing device according to claim 1, wherein when the second luggage is stacked on the first luggage, when the loading width of the second luggage is larger than the loading width of the first luggage, the projection control unit updates the width of the second image for the first luggage to the width of the second image for the second luggage. The luggage includes a third luggage and a fourth luggage,
3 . The information processing device according to claim 1 , wherein when a distance between the third piece of luggage and the fourth piece of luggage is equal to or less than a predetermined value, the projection control unit integrates the second image for the third piece of luggage and the second image for the fourth piece of luggage. A pair of partition plates are provided to divide the space of the shelf to form partial spaces;
The information processing device according to claim 1 , wherein the determination unit determines a storage area for the baggage placed in the partial space as a storage area corresponding to the partial space. providing a label on the front surface of the shelf that allows the dimensions of the luggage to be measured;
The information processing device according to claim 1 , wherein the determination unit determines the storage area for the luggage from the first image captured when the luggage is stored, the first image showing how the label is covered by the luggage. providing a label on the front surface of the shelf that allows the dimensions of the luggage to be measured;
The information processing device according to any one of claims 1 to 5, wherein the determination unit determines whether the removal position of the luggage specified by the worker is correct or not from the first image that captures the degree to which the label is hidden by the worker's hand when the luggage is removed. The information processing device according to any one of claims 1 to 6, wherein the projection control unit projects the second image onto the shelf using a projector attached to the shelf. The information processing device according to any one of claims 1 to 6, wherein the projection control unit projects the second image onto a lens of AR glasses worn by a worker. An information processing device,
determining a storage location and a storage area of the addressed package from a first image of a shelf on which the addressed package is placed ;
a step of projecting a second image onto the shelf, the second image displaying the address of each of the packages based on package information that associates the address acquired from the package, the storage location, and the storage area;
In the determining step, a storage area for the luggage to be stored in the storage area is determined based on address information of luggage placed in the adjacent storage area;
In the projecting step, the second image is projected onto a storage area of the shelf corresponding to the luggage with a width of the second image corresponding to a width of the storage area for each luggage;
Furthermore, in the projecting step, the storage area corresponding to the package having the same address information as the acquired address is highlighted and displayed.
Information processing methods. The device includes an information processing device, a camera, and a projection device,
The information processing device,
a determination unit that determines a storage location and a storage area of the package from a first image of a shelf on which the package with an address is placed, the first image being captured by the camera;
a projection control unit that causes the projection device to project onto the shelf a second image that displays the address of the package for each package based on package information that associates the address acquired from the package, the storage location, and the storage area ;
Equipped with
The determination unit determines a storage area for the luggage to be stored in the storage area based on address information of luggage placed in the adjacent storage area,
the projection control unit projects, for each of the packages, the second image onto a storage area of the shelf corresponding to the package with a width of the second image corresponding to a width of the storage area;
Furthermore, the projection control unit displays the storage area corresponding to the package having the same address information as the acquired address in an emphasized manner.
Information processing system. The computer of the information processing device,
a determination unit that determines a storage position and a storage area of the addressed package from a first image of a shelf on which the addressed package is placed;
a projection control unit that projects a second image, which displays the address of the package for each package, on the shelf based on package information that associates the address acquired from the package, the storage location, and the storage area;
causing the determination unit to determine a storage area for a package to be stored in the storage area based on address information of packages placed in adjacent storage areas;
cause the projection control unit to project, for each of the packages, the second image onto a storage area of the shelf corresponding to the package with a width of the second image corresponding to a width of the storage area;
and further causing the projection control unit to highlight and display the storage area corresponding to the package having the same address information as the acquired address.
program.

Images