JP7287015B2 - Merchandise management system and merchandise management method - Google Patents

Description

The present invention relates to a merchandise management system and merchandise management method capable of automatically detecting erroneous return of merchandise placed on a merchandise shelf.

In stores such as convenience stores and supermarkets, products are often displayed in showcases in which a plurality of product shelves are arranged in multiple stages. Stores need to manage the replenishable number of products arranged on product shelves and replenish the products in a timely and appropriate manner.

Patent Document 1 describes a system that uses a human sensor and a weight sensor to determine a product that has been taken out or a product that has been placed based on detection timing. In addition, Patent Document 2 describes a product situation analysis device that detects the disappearance of an object from the object placement area and the return of the object from the product placement area based on image information obtained by a camera. It is

JP 2018-206372 A JP 2014-149687 A

By the way, the products placed on the product shelf of the showcase are placed at a position specified in advance. Since the clerk had to go around the store and visually confirm the erroneously returned product, there was a problem that the work load on the clerk was heavy.

It should be noted that erroneous return is a phenomenon in which product removal and return are combined, and it is difficult to detect erroneous return only by simple product detection.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a merchandise management system and merchandise management method capable of automatically detecting erroneous return of merchandise placed on a merchandise shelf.

In order to solve the above-described problems and achieve the object, a product management system according to the present invention provides a product shelf in which products of the same product type are placed in rows extending in the depth direction from the front top position of the product shelf. and a camera sensor for acquiring a front image of the product placed at the front front position of each row of the product shelf; a front identification unit that generates product front information that identifies the product type and front position of the product; a panel sensor that is arranged on the product shelf and acquires a bottom image of the product placed on the product shelf; a product shelf identifying unit for generating product shelf information identifying the product size, the number of products in each row, and the product shelf position of the product for each product shelf based on the information and the bottom image; merchandise management having an erroneous return detection unit for detecting an erroneous return in which the merchandise is returned to a position other than the designated position based on merchandise placement information relating to the merchandise placement, the merchandise front information, and the merchandise shelf information and a part.

Further, in the merchandise management system according to the present invention, in the above invention, the erroneous return detection unit detects when the merchandise type of the current merchandise at the front position is different from the previously designated merchandise type based on the merchandise front information. , based on the product shelf information, if the size of the product in the same column is not the same as the size of the product specified in advance, or if the product number is 0 in the column based on the product shelf information, 1 or more products is placed, it is detected as the presence of the erroneous return.

Further, in the merchandise management system according to the present invention, in the above invention, the erroneous return detection unit determines that the merchandise type of the current merchandise in the front position is the merchandise type of the previously designated merchandise based on the merchandise front information. If different, the product type of the product, the product type of the different product, and the front position are generated as the erroneous return information.

Further, in the merchandise management system according to the present invention, in the above invention, the erroneous return detection unit detects when the size of the merchandise in the same row is not the same as the size of the merchandise designated in advance based on the merchandise shelf information. , the product type and the front position of products having different product sizes are generated as erroneous return information.

Further, in the product management system according to the present invention, in the above invention, the erroneous return detection unit detects, based on the product shelf information, when one or more products are placed in a row with a product number of 0, The product type and the front position of the product in the row on which one or more products are placed are generated as the erroneous return information.

In the product management system according to the present invention, in the above invention, when the erroneous return detection unit detects the erroneous return, the product management unit transmits the erroneous return information to a terminal connected to the product management system. It is characterized by comprising an output processing unit that outputs a notification to the device.

Further, a product management method according to the present invention is a product management method for managing products on a product shelf in which products of the same product type are placed in rows extending in the depth direction from the front front position of the product shelf, the product management method comprising: A front image acquiring step of acquiring a front image of the product placed at the front front position of each row of the product shelf; and a product whose product type and front position of the product on the front of the product shelf are identified based on the front image. a front surface identification step of generating front surface information; a bottom surface image acquiring step of acquiring a bottom surface image of the product placed on the product shelf and placed on the product shelf; a product shelf identification step of generating product shelf information identifying the size of the product, the number of products in each row, and the position of the product on the shelf for each product shelf; and an erroneous return detection step of detecting an erroneous return in which the product is returned to a position other than the designated position based on the information, the product front information and the product shelf information.

According to the present invention, erroneous return of a product placed on a product shelf is automatically detected, so it is possible to reduce the work burden on the store clerk.

FIG. 1 is a block diagram showing the configuration of a commodity management system that is an embodiment of the present invention. FIG. 2 is a schematic diagram showing the configuration of showcases arranged in a store. FIG. 3 is a diagram showing an example of a state in which the same product type is placed on each row of the product shelf. FIG. 4 is a diagram showing an imaging region of a front image captured by a camera sensor. FIG. 5 is a perspective view showing the configuration of a product shelf on which a panel sensor is arranged. FIG. 6 is a diagram showing an example of a bottom image. FIG. 7 is a flow chart showing a procedure for managing erroneous product returns by the product management system. FIG. 8 is a flow chart showing the erroneous return detection processing procedure in step S105. FIG. 9 is a diagram showing an example of a case where the product type of the current product in the front position is different from the previously specified product type. FIG. 10 is a diagram showing an example of a case where the size of products in the same column is not the same as the size of a product specified in advance. FIG. 11 is a diagram showing an example of a case where one or more products are placed in a row with zero products. FIG. 12 is a diagram showing an example of a case where it is determined that there is no erroneous return. FIG. 13 is a diagram showing an example of notification output of erroneous return information to a tablet terminal.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments for carrying out the present invention will be described with reference to the accompanying drawings.

<System configuration>
FIG. 1 is a block diagram showing the configuration of a commodity management system 1 that is an embodiment of the present invention. Moreover, FIG. 2 is a schematic diagram showing the structure of the showcase 2 arranged in the store. As shown in FIG. 1, in the product management system 1, a product type identification controller 10 as a front identification unit, a panel sensor controller 20 as a product shelf identification unit, and a product management controller 30 are connected to a network N1. The PC terminal 40 and the tablet terminal 50 are connected to the network N1 via the network N2.

A camera sensor 11 and a storage unit 12 are connected to the product type identification controller 10 . The camera sensor 11 is an imaging device that acquires a front image D1 of the showcase 2 . The storage unit 12 stores a front image D1, product feature information D2, and product front information D3.

The product type identification controller 10 generates product front information D3 identifying the product type and front position (XY plane) of the product in each row of the product shelf based on the front image D1. At this time, the product type identification controller 10 identifies the product type of the front image D1 by referring to the product feature information D2, which is the feature information of the front image for each product type. The product type identification controller 10 sends the generated product front information D3 to the panel sensor controller 20 . The product characteristic information D2 is information indicating the product packaging state including, for example, the color, pattern, design, and shape of the product.

As shown in FIG. 2, in the showcase 2, products P are placed on a plurality of product shelves 2a to 2f via panel sensors S1 to S6, respectively. As shown in FIG. 3, on each of the product shelves 2a to 2f, products P of the same product type are placed in each row R extending in the depth direction (+Z direction) from the top position of the front surface.

As shown in FIG. 2, the camera sensor 11 is attached to the ceiling 101, for example, and acquires front images D1 of the product shelves 2a to 2f. As shown in FIG. 3, when products P exist in each row R, all products P appear in the front. As shown in FIG. 4, the camera sensor 11 acquires an image of the front area E of the showcase 2 as a front image D1. An area E is an area where the products P on all the product shelves 2a to 2f can be imaged. Note that the front image D1 may be a set of front images divided for each of the product shelves 2a to 2f.

In FIG. 1, the panel sensor controller 20 is connected via a hub 21 to the panel sensors S (S1 to S6) arranged on the product shelf and the storage unit 22. FIG. As shown in FIG. 2, the panel sensors S1-S6 are arranged on the upper surface of each product shelf 2a-2f. For example, as shown in FIG. 5, a panel sensor S1 is arranged on the upper surface of the product shelf 2a.

The panel sensor S is a capacitance type panel sensor that detects changes in the capacitance of the product P placed on the upper surface of the panel sensor S in a matrix. 6 is acquired as the bottom image D4 of the product. FIG. 6 shows the bottom image D4 when the PET bottle is placed, and a circular image corresponding to the shape of the bottom of the PET bottle is formed in the area where the PET bottle is placed.

In FIG. 1, the storage unit 22 stores product front information D3, bottom image D4, and product shelf information D5.

Based on the product front information D3 and the bottom image D4, the panel sensor controller 20 determines the product size for each product shelf 2a to 2f, the number of products for each product type in each row R, the product shelf position (XZ The product shelf information D5 that identifies the plane) is generated.

As for the number of products, template matching processing is performed on the bottom image D4 using template matching information for each product type indicated by the product front information D3, and the number of portions determined to match in the template matching processing is detected as the number of products.

The product management controller 30 has an erroneous return detection unit 30a and an output processing unit 30b, and is connected to a storage unit 31. FIG. The storage unit 31 stores product placement information D6, which is the placement information of products designated in advance, and erroneous return information D7, which is the result of erroneous return detection. The product management controller 30 also manages the number of excess or deficiency of products based on the product front information D3 and the product shelf information D5.

The erroneous return detection unit 30a detects erroneous return where the product is returned to a position other than the designated position based on the product placement information D6, the product front information D3, and the product shelf information D5, and stores it as erroneous return information D7. . When the erroneous return detection unit 30a detects an erroneous return, the output processing unit 30b notifies and outputs the erroneous return information D7 to the PC terminal 40, the tablet terminal 50, and the like.

When the panel sensor controller 20 does not detect any product in the area where the leading product of each row R is arranged, the panel sensor controller 20 notifies the product management controller 30 that there is no leading product in the row R for which no product was detected. Upon receipt of this notification, the product management controller 30 notifies, for example, the tablet terminal 50 of an advance product delivery instruction for the row R in which no product has been detected. Further, the product management controller 30 notifies the tablet terminal 50 of a product advance instruction when one or more products are detected in the row R in which no product is detected.

<Product return management process>
Next, a procedure for managing erroneous product returns by the product management system 1 will be described with reference to the flowchart shown in FIG. As shown in FIG. 7, the product management controller 30 first acquires the product arrangement information D6 specified in advance in the storage unit 31 (step S101). After that, the product type identification controller 10 generates product front information D3 (step S102). Furthermore, the panel sensor controller 20 generates product shelf information D5 (step S103).

The product management controller 30 refers to the product placement information D6, the product front information and the product shelf information, and determines whether or not there is a product placement different from the product placement information D6 (step S104). Different product placement means that there is no product at the position indicated by the product placement information D6, there is a different product, or there is a product at a position where there is no product, and erroneous return is not detected at this point.

If there is a product layout that differs from the product layout information D6 (step S104, Yes), an erroneous return detection process is performed by the erroneous return detector 30a (step S105). As a result, it is determined whether or not it is determined that there is an erroneous return (step S106). If an erroneous return is detected (step S106, Yes), the output processing unit 30b notifies and outputs the erroneous return information D7 to the PC terminal 40 and the tablet terminal 50 (step S107), and proceeds to step S108.

On the other hand, if there is no product placement that differs from the product placement information D6 (step S104, No), or if an erroneous return is not detected (step S106, No), the process proceeds directly to step S108. In step S108, it is determined whether or not there is an end instruction (step S108), and if there is no end instruction (step S108, No), the process proceeds to step S101, repeats the above-described series of processes, and ends. If there is (step S108, Yes), this process ends.

<Error return detection processing>
FIG. 8 is a flow chart showing the erroneous return detection processing procedure in step S105. First, based on the product front information D3, the erroneous return detection unit 30a determines whether or not the product type of the current product in the front position is different from the previously designated product type (step S201). When the product type of the current product in the front position is different from the product type specified in advance (step S201, Yes), it is determined that there is an erroneous return, erroneous return information D7 is generated (step S204), and step S105. return to

On the other hand, if the product type of the current product in the front position is the same as the previously specified product type (step S201, No), the size of the product in the same row is determined based on the product shelf information D5. It is determined whether or not the size is the same as the size of the product specified in advance (step S202). If the size of the product in the same column is not the same as the size of the product specified in advance (step S202, No), it is determined that there is an erroneous return, erroneous return information D7 is generated (step S204), and step S105. return to

If the size of the product in the same column is the same as the size of the product specified in advance (step S202, Yes), then one or more products are added to the column where the number of products is 0 based on the product shelf information D5. It is determined whether or not it is placed (step S203). If 1 or more products are placed in the row of 0 products (step S203, Yes), it is determined that there is an erroneous return, erroneous return information D7 is generated (step S204), and the process proceeds to step S105. return.

On the other hand, if 1 or more products are not placed in the row of 0 products (step S203, No), it is determined that there is no erroneous return (step S205), and the process returns to step S105.

<Example when the product type of the current product in the front position is different from the product type specified in advance>
Here, an example in which it is determined that there is an erroneous return in step S201 will be described with reference to FIG. In this determination, the product front information D3 is referred to.

FIG. 9(a) is a schematic diagram corresponding to the product shelf information D5 on the XZ plane showing an example where products are arranged in three rows. Corresponds to information D3. FIG. 9A shows a state in which three products A are placed on row R1, three products B are placed on row R2, and no products are placed on row R3. Product B has a larger bottom image than product A.

FIG. 9(b) shows the placement state ST10 before the detection of erroneous return, in which three products A1, A2, and A3 are placed on row R1, three products B are placed on row R2, and three products B are placed on row R3. is not loaded with goods. On the other hand, in the placement state ST11 after the erroneous return detection, the product A3 placed on the front surface of the row R1 disappears, and the product A3' corresponding to the product A3 is placed on the front surface of the row R3. Row R3 is a row with no products, and the presence of product A3' in row R3 means that products of different product types are placed in row R3, and product A3' is an erroneously returned product. , it is determined that there is an erroneous return.

In addition, in the placement state ST12 after detection of an erroneous return, all the products A1, A2 and A3 on the row R1 are gone, and the products A1', A2' and A3' corresponding to the products A1, A2 and A3 are placed on the row R3. are placed. Row R3 is a row with no products, and the presence of product A3' in row R3 means that products of different product types are placed in row R3, and product A3' is an erroneously returned product. , it is determined that there is an erroneous return.

In FIG. 9(c), the placement state ST20 before the detection of an erroneous return is such that three products A1, A2, and A3 are placed on the row R1, and two products B are placed on the row R2 other than the front. ing. On the other hand, in the placement state ST21 after the erroneous return detection, the product A3' corresponding to the product A3 is placed on the front surface of the row R2. Row R2 is a row in which product B is placed, and product A3' is of a product type different from product B, so product A3' is an erroneously returned product and is determined to be erroneously returned.

In addition, in the placement state ST22 after the erroneous return detection, the product A1' corresponding to the product A1 other than the front of the row R1 is placed on the front of the row R2. Row R2 is a row in which product B is placed, and product A1' is of a product type different from product B, so product A1' is an erroneously returned product and is determined to have been erroneously returned.

In FIG. 9(d), the placement state ST30 before the detection of erroneous return is such that three products A1, A2, and A3 are placed on row R1, one product B is placed on the front of row R2, and row R3 is placed. is not loaded with goods. On the other hand, in the placement state ST31 after the detection of erroneous return, the product A1' corresponding to the product A1 is arranged in front of the row R3. Since the product A1' is of a product type different from "no product", the product A1' is an erroneously returned product and is determined to have an erroneous return.

<An example of the case where the size of the products in the same row is not the same as the size of the product specified in advance>
Here, an example in which it is determined that there is an erroneous return in step S202 will be described with reference to FIG. In this determination, product shelf information D5 is referred to.

In FIG. 10, three products A1, A2, and A3 are placed in row R, and product B is placed in front of row R2 in placement state ST40 before erroneous return detection. On the other hand, in the placement state ST41 after the erroneous return detection, the products A1' and A2' corresponding to the products A1 and A2 other than the front surface of the row R1 are placed in positions other than the front surface of the row R2. Since the products A1' and A2' are different in size from the product B in the row R2, the products A1' and A2' are returned by mistake and are determined to be returned by mistake.

In the placement state ST42 after the erroneous return detection, the product A3' corresponding to the product A3 on the front of the row R1 is placed on a position other than the front of the row R2. Since product A3' has a size different from that of product B in column R2, product A3' is an erroneously returned product and is determined to have an erroneous return.

<Example of a case where 1 or more products are placed in a row with 0 products>
Here, an example in which it is determined that there is an erroneous return in step S203 will be described with reference to FIG. In this determination, product shelf information D5 is referred to.

In FIG. 11, the placement state ST50 before detection of erroneous return is such that three products A1, A2, and A3 are placed on row R1, three products B are placed on row R2, and products are placed on row R3. not placed.
On the other hand, in the placement state ST51 after the erroneous return detection, the products A1' and A2' corresponding to the products A1 and A2 other than the front surface of the row R1 are placed in positions other than the front surface of the row R3. Row R3 is a row in which the number of products is 0, and 1 or more products A1′ and A2′ are placed in row R3. is determined.

In the placement state ST52 after the erroneous return detection, the product A3' corresponding to the product A3 on the front of the row R1 is placed on a position other than the front of the row R3. Row R3 is a row in which the number of products is 0, and one or more products A3' are placed in row R3. Therefore, product A3' is an erroneously returned product, and is determined to have been erroneously returned.

<Example of a case where it is determined that there is no erroneous return>
Here, an example in which it is determined that there is no erroneous return will be described with reference to FIG. In FIG. 12, in the placement state ST60 before the detection of erroneous return, three products A1, A2 and A3 are placed on row R1, three products B1, B2 and B3 are placed on row R2, and three products B1, B2 and B3 are placed on row R3. , the product is not placed. On the other hand, in the placed state ST61 after the detection of the erroneous return, the product A1 on the front side of the row R1 has been taken out and is gone. In this case, since the determination conditions of steps S201 to S203 are not met, it is determined that there is no erroneous return.

In the placed state ST62 after the detection of the erroneous return, the commodity A1 other than the front side of the row R1 is removed and removed. In this case, since the determination conditions of steps S201 to S203 are not met, it is determined that there is no erroneous return.

In the placed state ST63 after the erroneous return detection, all the commodities A1, A2 and A3 in the row R1 are removed and removed. In this case, since the determination conditions of steps S201 to S203 are not met, it is determined that there is no erroneous return.

<Example of error return information notification output>
FIG. 13 is a diagram showing an example of notification output of the erroneous return information D7 to the tablet terminal 50. As shown in FIG. As shown in FIG. 13, the notification output displays erroneous return information consisting of the erroneously returned product (product type) and the arrangement position of the erroneously returned product (front position; shelf level and X direction position). A store clerk carrying this tablet terminal 50 can refer to this notification output and quickly and easily return the erroneously returned product to its original position.

It should be noted that each configuration illustrated in the above-described embodiment and modified examples is functionally schematic, and does not necessarily need to be physically configured as illustrated. In other words, the form of dispersion/integration of each device and component is not limited to the illustrated one, and all or part of them can be functionally or physically distributed/integrated in arbitrary units according to various usage conditions. can be configured

Reference Signs List 1 product management system 2 showcase 2a to 2f product shelf 10 product type identification controller 11 camera sensor 12, 22, 31 storage unit 20 panel sensor controller 21 hub 30 product management controller 30a erroneous return detection unit 30b output processing unit 40 PC terminal 50 Tablet terminal 101 Ceiling A, A1-A3, A1'-A3', B, B1, B2, B3 Product D1 Front image D2 Product feature information D3 Product front information D4 Bottom image D5 Product shelf information D6 Product placement information D7 Erroneous return information E Area N1, N2 Network P Merchandise R, R1~R3 Row S, S1~S6 Panel sensor ST10~ST12, ST20~ST22, ST30, ST31, ST40~ST42, ST50~ST52, ST60~ST63 Mounting state

Claims (7)

A product management system for managing products on product shelves in which products of the same product type are placed in rows extending in the depth direction from the front front position of the product shelf,
a camera sensor for acquiring a front image of the product placed at the front top position of each row of the product shelf;
a front identification unit that generates product front information identifying product types and front positions of products on the front of the product shelf based on the front image;
a panel sensor arranged on the product shelf and acquiring a bottom image of the product placed on the product shelf;
a product shelf identification unit for generating product shelf information identifying the product size, the number of products in each row, and the product shelf position of the product for each product shelf based on the product front information and the bottom image; ,
An erroneous return detection unit for detecting an erroneous return in which the product is returned to a position other than the specified position based on the product placement information, the product front information, and the product shelf information regarding the product placement specified in advance. the product management department,
A merchandise management system characterized by comprising: The erroneous return detection unit determines the size of the product on the same row based on the product shelf information when the product type of the current product on the front position is different from the product type specified in advance based on the product front information. If the size is not the same as the size of the product specified in advance, or if 1 or more products are placed in the row with the number of products being 0 based on the product shelf information, it is detected as the presence of the erroneous return. The merchandise management system according to claim 1, characterized by: The erroneous return detection unit detects the product type and the front surface of the product different from the product type when the product type of the current product at the front position is different from the product type of the product specified in advance based on the product front information. 3. The commodity management system according to claim 2, wherein the position and position are generated as the erroneous return information. Based on the product shelf information, the erroneous return detection unit determines the product type and the front position of the product whose size is not the same when the size of the product in the same row is not the same as the size of the product specified in advance. is generated as the erroneous return information. Based on the product shelf information, when one or more products are placed in a row with a product number of 0, the erroneous return detection unit determines the product type of the product in the row where one or more products are placed. 3. The product management system according to claim 2, wherein the front position and the position of the product are generated as the erroneous return information. The product management unit is characterized by comprising an output processing unit that, when the erroneous return detection unit detects the erroneous return, notifies and outputs the erroneous return information to a terminal device connected to the product management system. The product management system according to any one of claims 3-5. A product type identification controller connected to a camera sensor for acquiring a front image of the product placed at the front front position of each row of the product shelf; and products placed on the product shelf and placed on the product shelf. A panel sensor controller connected with a panel sensor for acquiring a bottom image of the product shelf and a product management controller storing product arrangement information relating to the arrangement of products designated in advance are connected to each other, and the product management controller is connected to the product shelf. A product management method for managing products on the product shelf in which products of the same product type are placed in rows extending in the depth direction from the top position of the front face,
The product type identification controller acquires the front image of the product by the camera sensor , and based on the front image, generates product front information identifying the product type and front position of the product on the front surface of the product shelf ,
The panel sensor controller acquires the bottom image of the product by the panel sensor , and based on the product front information and the bottom image, the size of the product, the number of products in each row, and the product for each product shelf. Generate product shelf information that identifies the product shelf position of
A merchandise management method , wherein the merchandise management controller detects an erroneous return of the merchandise to a position other than a designated position based on the merchandise placement information, the merchandise front information, and the merchandise shelf information. .

Images