JP6239537B2 - Product management apparatus and program - Google Patents

Description

  Embodiments described herein relate generally to a product management apparatus and a program.

  Traditionally, there is a sushi restaurant called Kaiten Sushi. In such a store, sushi as a product is placed on a plate and placed on a rotating lane provided in contact with the kitchen or the audience seat, thereby circulating the sushi to the audience seat. Conventionally, a technology has been proposed for managing products on a conveyance path by attaching an RFID tag that records the type of product placed on the dish to each dish and reading it with a reading device installed on the conveyance path. Yes.

  By the way, in the above-mentioned store, a fraudulent act may be performed with respect to the goods on a rotation lane. Such fraudulent acts include, for example, an act of returning an empty plate onto the rotating lane, an act of picking up only a product (sushi) from the plate on the rotating lane, and the like. In order to detect such an illegal act, the store clerk monitors the behavior of each customer, but the labor is large and an oversight may occur. In addition, in the above-described conventional technology related to merchandise management, it is not possible to detect the customer's fraud, so the above problem cannot be solved.

  The problem to be solved by the present invention is to provide a merchandise management apparatus and a program capable of detecting an illegal act of a customer in a store such as a sushi restaurant.

The merchandise management apparatus according to the embodiment is a merchandise management apparatus that manages merchandise conveyed while being placed on a conveyance body, and captures captured images from an imaging apparatus that images the conveyance path of the conveyance body. Means, first detection means for detecting a first area representing the carrier from a captured image captured by the capture means, and second detection means for detecting a second area representing the product from the first area. , Third detection means for detecting the first area moved in another direction different from the conveyance direction of the conveyance path, the second detection means for the first area detected by the first detection means, and the And a fourth detection means for detecting fraud based on a detection result of the third detection means, wherein the fourth detection means moves the third detection means from outside the conveyance path into the conveyance path. 1 region is detected, and the second detecting means Determining if the first region can not detect the second region, and to detect fraudulent activity.

FIG. 1 is a diagram schematically illustrating a store system according to the embodiment. FIG. 2 is a block diagram illustrating an example of a hardware configuration of the commodity management apparatus. FIG. 3 is a diagram schematically illustrating an example of the data configuration of the first dictionary file. FIG. 4 is a diagram schematically illustrating an example of the data configuration of the second dictionary file. FIG. 5 is a diagram schematically illustrating an example of the data configuration of the product management file. FIG. 6 is a block diagram illustrating an example of a functional configuration of the commodity management apparatus. FIG. 7 is a flowchart illustrating a procedure of product management processing performed by the product management apparatus.

  Below, the merchandise management apparatus and program used in a sushi restaurant (store) called “conveyor sushi” will be described with reference to the drawings.

  FIG. 1 is a diagram schematically illustrating a store system 1 according to the present embodiment. As shown in FIG. 1, the store system 1 includes a transport device 10, a camera 20, a notification device 30, and a product management device 40. Here, the camera 20, the notification device 30, and the product management device 40 are communicably connected via a communication line (not shown).

  The transport apparatus 10 is a transport apparatus for transporting sushi that is a product in the main store. The transport apparatus 10 includes a circulation-type transport path (rotation lane) 10a formed in a substantially U shape. A part of the rotating lane 10a is disposed in the backyard R1, and the other part is disposed in the passenger cabin R2.

  In the backyard R1, mainly sushi and the like that are commodities are manufactured. In the backyard R1, the store clerk places the manufactured sushi on the dish D serving as a transporter, and places the dish D on the rotating lane 10a. In the guest room R2, a passenger seat SE such as a counter table is provided along the rotation lane 10a. The transport device 10 circulates and delivers sushi to the guest seat SE by transporting the plate placed on the rotating lane 10a in the direction of the arrow in the figure. In addition, about a conveyance of the conveying apparatus 10, a well-known technique shall be used. The shape of the rotation lane 10a is not limited to that shown in FIG.

  The camera 20 is an imaging device including an imaging element such as a color CCD (Charge Coupled Device) or a color CMOS (Complementary Metal Oxide Semiconductor). The camera 20 is provided for each passenger seat SE and images a predetermined area on the rotation lane 10a facing the passenger seat SE from above. The camera 20 captures a moving image at a frame rate of, for example, 30 fps and transmits the captured image to the product management apparatus 40 as a captured image. Note that the number of cameras 20 is not particularly limited. It is assumed that the captured image generated by each camera 20 is accompanied by a seat identifier that can identify the corresponding passenger seat SE.

  The notification device 30 is a device that is provided, for example, in the backyard R1 or the like and notifies the store clerk. Specifically, the notification device 30 operates according to the control of the product management device 40 to notify the store clerk that there is an inappropriate product. Note that the notification method of the notification device 30 is not particularly limited. For example, it is good also as a structure which alert | reports by the output of a message or an image by making the alerting | reporting apparatus 30 into an electric bulletin board and a display apparatus. Moreover, it is good also as a structure which alert | reports by the output of an audio | voice by making the alerting | reporting apparatus 30 into audio | voice output apparatuses, such as a buzzer and a speaker.

  The merchandise management apparatus 40 is an information processing apparatus such as a PC (Personal Computer) provided in the backyard R1 or the like. The merchandise management device 40 manages the state of merchandise (such as sushi) conveyed to each passenger seat SE based on the captured image captured by the camera 20. Hereinafter, the merchandise management apparatus 40 will be described.

  FIG. 2 is a block diagram illustrating an example of a hardware configuration of the merchandise management apparatus 40. The merchandise management apparatus 40 includes a microcomputer 41 as an information processing unit that executes information processing. The microcomputer 41 mainly includes a CPU (Central Processing Unit) 42, a ROM (Read Only Memory) 43, and a RAM (Random Access Memory) 44. The ROM 43 stores various programs that can be executed by the CPU 42. The RAM 44 is a main storage device of the product management device 40.

  A communication unit 45 is connected to the CPU 42 of the product management apparatus 40. The communication unit 45 performs data communication with the camera 20 and the notification device 30 under the control of the CPU 42.

  In addition, a storage unit 46 such as an HDD or an SSD is connected to the CPU 42 of the merchandise management apparatus 40. The storage unit 46 stores various programs and various files that can be executed by the CPU 42. In addition, the storage unit 46 stores a first dictionary file F1, a second dictionary file F2, and a product management file F3.

  FIG. 3 is a diagram schematically illustrating an example of the data configuration of the first dictionary file F1. As shown in FIG. 3, the first dictionary file F1 stores the price and the dish feature amount in association with the dish identifier. The dish identifier is a unique identifier assigned to each dish type. In general, in a sushi restaurant, prices are ranked for each product, and products are transported using dishes with different patterns for each price. Therefore, an identifier that can identify the type is stored in the dish identifier of the first dictionary file F1 for each type of dish used for transporting the product. The price corresponding to the type of dish (dish identifier) is stored in the price of the first dictionary file F1. Further, the dish feature amount of the first dictionary file F1 is data obtained by parameterizing the appearance characteristics of each dish such as the shape of the dish and the pattern of the dish.

  FIG. 4 is a diagram schematically illustrating an example of the data configuration of the second dictionary file F2. As shown in FIG. 4, the second dictionary file F2 stores the product name (product name), the dish identifier, the first product feature value, the second product feature value, and the like of the product in association with the product identifier of each product. .

  Here, the product identifier is a unique identifier assigned to each product type. The dish identifier is a dish identifier of a dish on which the product is placed. The first product feature amount is data obtained by parameterizing features on the appearance (surface) of foods constituting the product, such as the color, gloss, pattern, and unevenness of the product. This first product feature amount is mainly used to specify the type of product. The second product feature amount is data obtained by parameterizing features on the outer shape such as the shape, layout, and topping of the product when the product is placed on the plate. The second product feature amount is mainly used to determine whether or not the product is placed on the dish with a predetermined outer shape (appropriate outer shape). It is assumed that the first product feature amount and the second product feature amount are created based on products in an appropriate state.

  FIG. 5 is a diagram schematically illustrating an example of the data configuration of the product management file F3. The merchandise management file F3 stores merchandise information, a cheating flag, a cheating image, and the like in association with the seat identifier.

  The seat identifier is a unique identifier assigned to each passenger seat SE. The product information has a product identifier and date / time information. Here, the product information stores the product identifier of the product acquired from the rotation lane 10a and the date / time information indicating the date / time when the acquisition was performed. The cheating flag is flag information for identifying whether or not cheating has been performed. In addition, the captured image of the camera 20 when an illegal act is detected is stored in the illegal act image.

  In the product management unit 416, which will be described later, every time a customer acquires a dish (product) from the rotating lane 10a, product information including the product identifier of the product is registered in association with the seat identifier of the passenger seat SE where the customer sits. In addition, when the merchandise management unit 416 detects an illegal act on the product, the merchandise management unit 416 sets a cheating flag in association with the product information of the product. In addition, the merchandise management unit 416 registers the captured image when the fraud is detected in association with the merchandise information of the merchandise as the fraud image. Note that the merchandise information, the fraudulent action flag, and the fraudulent image associated with the seat identifier are cleared from the merchandise management file F3 upon completion of the accounting for the customer at the corresponding passenger seat SE.

  Next, the functional configuration of the merchandise management apparatus 40 will be described with reference to FIG. FIG. 6 is a block diagram illustrating an example of a functional configuration of the product management device 40. The product management apparatus 40 cooperates with the CPU 42 and a program stored in the ROM 43 or the storage unit 46 to obtain an image capture unit 411, a dish detection unit 412, a product identification unit 413, an outer shape determination unit 414, and an operation detection unit 415. The product management unit 416 and the notification processing unit 417 are provided as functional units.

  The image capturing unit 411 is a functional unit corresponding to the capturing unit of the present embodiment. The image capturing unit 411 captures a captured image captured by the camera 20 via the communication unit 45. Specifically, the image capturing unit 411 sequentially captures captured images captured by the camera 20 via a communication line (not shown) and stores the captured images on the RAM 44. It is assumed that the captured image captured by the camera 20 is accompanied by the seat identifier of the camera 20.

  The dish detection unit 412 is a functional unit corresponding to the first detection unit and the second detection unit of the present embodiment. The dish detection unit 412 detects a region representing a dish or a product from the captured image captured by the image capturing unit 411 by using a known technique such as a pattern matching technique. Here, the detection method of the dish detection unit 412 is not particularly limited.

  For example, the dish detection unit 412 compares the contour line extracted from the captured image with the shape defined in the dish feature value of the first dictionary file F1. Then, the dish detection unit 412 detects an area in the captured image that matches the shape of the dish feature value as a dish area (first area). Further, the dish detection unit 412 compares the pattern represented in the dish area with the pattern defined in the dish feature amount of the first dictionary file F1, and identifies a matching pattern. Then, the dish detection unit 412 detects the remaining area from which the matched pattern portion is removed from the dish area as a product area (second area). In addition, the dish detection unit 412 identifies the dish identifier of the pattern of the dish feature amount that matches the pattern of the dish area as the dish type.

  The product specifying unit 413 is a functional unit corresponding to the specifying unit of the present embodiment. The product specifying unit 413 specifies the product included in the captured image from the captured image captured by the image capturing unit 411. Specifically, the product specifying unit 413 extracts data representing the appearance (surface) feature from the product region detected by the dish detection unit 412 as the first product feature amount. The product specifying unit 413 compares the first product feature amount extracted from the product region with the first product feature amount of each registered product registered in the second dictionary file F2, and calculates the similarity. Here, the degree of similarity quantitatively indicates how similar the product in the captured image is to each registered product. The product specifying unit 413 calculates the degree of similarity by comparing the first product feature amount of each registered product associated with the plate identifier based on the plate identifier specified by the plate detection unit 412. Also good. Then, the product specifying unit 413 specifies (recognizes) a registered product having a highest similarity that exceeds a predetermined threshold among the calculated similarities as a product item represented in the product area.

Recognizing an object included in an image in this way is called generic object recognition. Regarding general object recognition, for example, various recognition techniques are described in the following documents.
Keiji Yanai, “Current Status and Future of General Object Recognition”, IPSJ Journal, Vol. 48, no. SIG16 [Search January 6, 2015], Internet <URL: http://mm.cs.uec.ac.jp/IPSJ-TCVIM-Yanai.pdf>

Further, techniques for performing general object recognition by dividing an image into regions for each object are described in the following documents.
Jamie Shotton et al., “Semantic Texton Forests for Image Categorization and Segmentation”, [Search January 6, 2015], Internet <URL: http://jamie.shotton.org/work/publications/cvpr08.pdf>

  Note that the method of calculating the similarity is not particularly limited. For example, the similarity may be calculated as an absolute evaluation or a relative evaluation. When calculating the similarity as an absolute evaluation, the first product feature amount of the product in the captured image is compared with each first product feature amount registered in the second dictionary file F2 on a one-to-one basis. The similarity derived as a result of the above may be adopted as it is. Further, when calculating the similarity as a relative evaluation, the total of the similarities may be calculated to be 100%.

  The outer shape determination unit 414 is a functional unit corresponding to the determination unit of the present embodiment. The outer shape determination unit 414 determines whether or not the outer shape of the product included in the captured image is appropriate by using a known technique such as a pattern matching technique. Specifically, the outer shape determination unit 414 uses, as the second product feature amount, features on the outer shape such as the shape, layout, and topping of the products existing in the region from the dish region and the product region detected by the dish detection unit 412. Extract. In addition, the outer shape determination unit 414 reads the second product feature amount of the registered product specified by the product specifying unit 413 from the second dictionary file F2. Then, the outer shape determination unit 414 compares the second product feature amount extracted from the dish region with the second product feature amount read from the second dictionary file F2, thereby determining whether or not the outer shape of the product matches. judge. Here, when the determination result of the outer shape determination unit 414 indicates a match, it means that the outer shape of the product is in an appropriate state.

  The operation detection unit 415 is a functional unit corresponding to the third detection unit of the present embodiment. The motion detection unit 415 detects a customer's motion performed on a dish or product from a plurality of consecutive captured images captured by the image capture unit 411 using a known technique such as an inter-frame difference method. To do. Specifically, the motion detection unit 415 detects the region when the plate region or the product region detected by the plate detection unit 412 moves in another direction different from the conveyance direction of the rotation lane 10a.

  For example, the motion detection unit 415 detects that the dish area on the rotation lane 10a detected by the dish detection unit 412 has moved from the rotation lane 10a to the outside of the rotation lane 10a as the first operation. In addition, as a second operation, the operation detection unit 415 detects that the dish area outside the rotation lane 10a detected by the dish detection unit 412 has moved from outside the rotation lane 10a into the rotation lane 10a. Here, the first operation corresponds to an operation in which the customer acquires the dish from the rotation lane 10a. Further, the second operation corresponds to an operation in which the customer returns the dish onto the rotation lane 10a.

  In addition, as a third operation, the motion detection unit 415 indicates that, out of the set of the dish area and the product area detected by the dish detection unit 412, only the product area moves from the rotation lane 10 a to the outside of the rotation lane 10 a. Detect. The third operation corresponds to an operation (injustice) in which the customer acquires only the product from the dish on the rotation lane 10a.

  The merchandise management unit 416 is a functional unit corresponding to the fourth detection unit of the present embodiment. The product management unit 416 registers product information in the product management file F3 and cancels registration based on the processing results of the dish detection unit 412 to the operation detection unit 415.

  For example, when the motion detection unit 415 detects the first motion, the product management unit 416 registers product information using the product identifier specified by the product specification unit 413 from the dish area (product region) to be operated. . More specifically, when the first operation is detected, the product management unit 416 includes a product identifier specified by the product specifying unit 413 from the detection source dish region (product region) and the current date and time information. Generate information. Then, the product management unit 416 registers the generated product information in the product management file F3 in association with the seat identifier of the captured image in which the first operation is detected.

  In addition, when the operation detection unit 415 detects the second operation, the product management unit 416 determines whether an illegal act has been performed based on the processing results of the product specification unit 413, the outer shape determination unit 414, and the like. For example, the merchandise management unit 416 determines that an illegal act has been detected in the case of the following processing results (first processing result, second processing result, and third processing result).

  As a 1st process result, the case where the dish detection part 412 cannot detect a goods area is mentioned about the dish area | region where 2nd operation | movement was detected. In addition, as a second processing result, there is a case where the product specifying unit 413 cannot specify a product for the plate region where the second operation is detected, even though the plate detection unit 412 detects the product region. . A possible cause of these processing results is that the dish in which all or part of the product has been removed has been returned to the rotating lane 10a. Therefore, when the first or second processing result is obtained, the merchandise management unit 416 determines that an illegal act has been detected, and records the illegal act.

  More specifically, the merchandise management unit 416 refers to the second dictionary file F2 and identifies the merchandise identifier corresponding to the dish identifier of the dish area where the second action is detected as the target identifier. In addition, the product management unit 416 refers to the product management file F3 and searches for product information including the target identifier from the product information registered in association with the seat identifier of the captured image in which the second operation is detected. The merchandise management unit 416 sets a fraudulent action flag in association with one of the searched merchandise information, for example, the merchandise information with the latest date information. Further, the merchandise management unit 416 registers the captured image in which the second operation is detected in the merchandise management file F3 as a cheating image in association with the merchandise information for which the fraud flag is set. Note that product information for which the fraudulent flag is set is excluded from the search range of the target identifier.

  In addition, as a third processing result, there is a case where the determination result of the outer shape determination unit 414 shows a difference even though the product specifying unit 413 has specified the product for the dish area where the second operation is detected. It is done. A possible reason for this processing result is that the dish in which a part of the product has been removed has been returned to the rotating lane 10a. Therefore, the merchandise management unit 416 determines that an illegal act has been detected when the second processing result is obtained, and records the illegal act.

  More specifically, the product management unit 416 specifies the product identifier of the product specified by the product specifying unit 413 as the target identifier. The merchandise management unit 416 searches the merchandise information including the target identifier from the merchandise information registered in association with the seat identifier of the captured image in which the second operation is detected, with reference to the merchandise management file F3. Then, the merchandise management unit 416 sets the fraud flag and registers the fraudulent action image in one of the retrieved merchandise information as described above.

  In addition, as a processing result other than the first to third processing results described above, the product specifying unit 413 specifies a product for the product region in the dish region where the second operation is detected, and the outer shape determination unit 414 The case where it determines with a match is mentioned. In this case, the merchandise management unit 416 determines that the dish once taken from the rotation lane 10a is returned to the rotation lane 10a as it is, and performs a process of canceling the registered merchandise information. More specifically, the merchandise management unit 416 searches the merchandise information registered in association with the seat identifier of the captured image of the detection source that includes the merchandise identifier specified by the merchandise specifying unit 413. Then, the merchandise management unit 416 deletes one of the searched merchandise information, for example, transaction information with the latest date and time information from the merchandise management file F3.

  In addition, when the motion detection unit 415 detects the third motion for the product region in the captured image, the product management unit 416 determines that the cheating is detected, and records the cheating. More specifically, the product specifying unit 413 generates product information including the product identifier specified by the product specifying unit 413 and the current date and time information, and associates the product information with the seat identifier of the captured image in which the third operation is detected. Is registered in the product management file F3. Then, the merchandise management unit 416 sets a fraud flag and registers a fraudulent image for the merchandise information registered in the merchandise management file F3. In the case of the third operation, the dish is not picked up from the rotation lane 10a, so that the information indicating that the operation is the third operation is added to the product information so that the dish can be distinguished from other illegal acts. It is good.

  The notification processing unit 417 is a functional unit that controls the notification device 30 via a communication line (not shown) to perform notification. Specifically, when the merchandise management unit 416 determines that the merchandise management unit 416 determines that the fraud has occurred, the notification processing unit 417 informs the store clerk of the seat identifier of the seat SE (captured image) in which the fraud is detected by operating the notification device 30. To do. Note that the notification processing unit 417 may be configured to switch a message to be notified and a notification method according to the determination result of the product management unit 416. Thereby, the store clerk can determine the type of fraudulent activity by confirming the notification contents, so that convenience can be improved.

  Hereinafter, the product management process performed by the product management device 40 will be described with reference to FIG. Here, FIG. 7 is a flowchart showing the procedure of the merchandise management process performed by the merchandise management apparatus 40. As a premise of this processing, it is assumed that the product identification unit 413, the outer shape determination unit 414, and the operation detection unit 415 are operating in the background.

  First, the image capturing unit 411 sequentially captures captured images captured by the camera 20 (step S11). The dish detection unit 412 stands by until a dish area is detected from the captured image captured in step S11 (step S12; No). When the dish detection unit 412 detects the dish area (step S12; Yes), the dish type is specified and the product area is detected based on the dish area.

  Subsequently, the product management unit 416 determines whether or not the operation detection unit 415 has detected any operation. Here, when the motion detection unit 415 detects the first motion (step S13; Yes), the product management unit 416 generates product information including the product identifier specified from the detection source dish region (product region). To do. Then, the merchandise management unit 416 registers the generated merchandise information in the merchandise management file F3 in association with the seat identifier of the captured image in which the first operation is detected (step S14), and returns the process to step S12.

  Moreover, when the operation | movement detection part 415 detects a 2nd operation | movement (step S13; No-> step S15; Yes), the merchandise management part 416 transfers to step S16. In step S16, the merchandise management unit 416 determines whether or not the dish detection unit 412 has detected the merchandise area from the dish area in which the second operation is detected (step S16). When the dish detection unit 412 cannot detect the product area (step S16; No), the product management unit 416 determines that an illegal act has been detected, and proceeds to step S22.

  When the dish detection unit 412 can detect the product region (step S16; Yes), the product management unit 416 determines whether or not the product specification unit 413 can specify the product from the detected product region (step S17). . When the product identification unit 413 cannot identify the product (step S17; No), the product management unit 416 determines that an illegal act has been detected, and proceeds to step S22.

  When the product identification unit 413 can identify the product (step S17; Yes), the product management unit 416 determines whether the determination result of the outer shape determination unit 414 for the specified product is a match indicating that the product is in the proper shape. Is determined (step S18). When the determination result of the outer shape determination unit 414 indicates a difference (step S18; No), the product management unit 416 determines that an illegal act has been detected, and proceeds to step S22.

  When the determination result of the outer shape determination unit 414 indicates a match (step S18; Yes), the merchandise management unit 416 determines that the dish once taken has been returned to the rotation lane 10a normally. Then, the merchandise management unit 416 cancels the registered merchandise information (step S19), and then returns the process to step S12.

  Moreover, when the operation | movement detection part 415 detects 3rd operation | movement (step S13; No-> step S15; No-> step S20; Yes), the merchandise management part 416 determines with having detected the fraudulent act. Next, the merchandise management unit 416 generates merchandise information including the merchandise identifier specified from the detection source dish area (commodity area). Then, the merchandise management unit 416 registers the generated merchandise information in the merchandise management file F3 in association with the seat identifier of the captured image in which the third operation is detected (step S21), and proceeds to step S22. In addition, when the operation | movement detection part 415 does not detect any operation | movement (step S13; No-> step S15; No-> step S20; No), the goods management part 416 shall return a process to step S12.

  In subsequent step S22, the merchandise management unit 416 records fraud by setting a fraud flag and registering a fraud image in the merchandise information of the merchandise determined to be fraud (step S22). And the alerting | reporting process part 417 alert | reports the seat identifier by which the fraudulent act was performed by operating the alerting | reporting apparatus 30 (step S23), and returns a process to step S12.

  As described above, the product management apparatus 40 according to the present embodiment detects the state of the product (dish) and the customer's movement from the captured image obtained by imaging the rotation lane 10a, and based on the detection result, the presence / absence of fraud Determine. Thereby, since the customer's fraudulent act on the product can be detected, it becomes possible to save the labor of the store staff involved in the monitoring, and the product can be managed efficiently.

  In addition, when the merchandise management device 40 detects an illegal act, the merchandise management device 40 cooperates with the notification device 30 to notify the store clerk of the seat identifier of the passenger seat SE where the illegal act has been performed. Thereby, since it is possible to notify the store clerk that the cheating has been performed and the customer seat SE where the cheating has been performed, it is possible to improve the convenience for confirming the status of the cheating.

  In addition, when the merchandise management device 40 detects a fraud, the fraudulent act is recorded by registering the fraud flag and the fraud image in the merchandise management file F3 in association with the corresponding seat identifier (product information). Do. Thereby, since the status confirmation of fraudulent activity can be performed efficiently, it is possible to improve the efficiency of operations related to product management. In particular, by referring to the fraudulent image, it is possible to check the products subject to the fraudulent act and the situation when the fraudulent act is performed. it can.

  Further, when the product management device 40 detects an operation in which a customer acquires a product from the rotation lane 10a, the product management device 40 registers product information including the product identifier of the product in the product management file F3 in association with the corresponding seat identifier. Thus, by referring to the product information of each seat identifier, it is possible to easily confirm the breakdown of the product acquired by the customer, so that the accounting process can be performed efficiently. For example, the dish identifier corresponding to the product identifier included in the product information is specified from the second dictionary file F2, and the price corresponding to the identifier is specified from the first dictionary file F1. And the total amount of the goods which the customer acquired can be calculated by adding the price specified from all the product information. Further, by referring to the product information of each seat identifier, it is possible to easily confirm the order in which the products are acquired and the time interval, so that it is possible to efficiently perform analysis and the like related to store management.

  As mentioned above, although embodiment of this invention was described, the said embodiment was shown as an example and is not intending limiting the range of invention. The above embodiments can be implemented in various other forms, and various omissions, replacements, changes, additions, combinations, and the like can be made without departing from the scope of the invention. Moreover, the said embodiment and its deformation | transformation are included in the range of the invention, the summary, and the invention described in the claim, and its equal range.

  For example, in the merchandise management apparatus 40 according to the above-described embodiment, the notification is performed when an illegal act is detected. However, the notification may not be performed.

  Moreover, in the said embodiment, although it was set as the structure which provides the camera 20 for every passenger seat SE, it is good not only as this but the structure which provides one camera 20 with respect to several passenger seat SE. In the case of adopting this configuration, a detection area corresponding to the position of each passenger seat SE is predetermined in the captured image captured by the camera 20, and a seat identifier of each passenger seat SE is attached to the corresponding detection area. And The merchandise management unit 416 of the merchandise management apparatus 40 registers merchandise information and records fraud based on the dish identifier of the dish area detected by the dish detection unit 412 and the detection area to which the merchandise area belongs.

  Moreover, in the said embodiment, although rotating sushi was mentioned as an example as an example of a store, you may apply to other stores which handle goods other than this, not only this. In addition, it does not ask whether a conveyance path is a circulation type.

  In addition, the program executed by each device of the above embodiment is provided by being incorporated in advance in a storage medium (ROM or storage unit) included in each device, but is not limited thereto. For example, the program may be provided in a form that can be installed or executed and stored (recorded) in a computer-readable storage medium (recording medium). Examples of the storage medium include a CD-ROM, a flexible disk (FD), a CD-R, and a DVD (Digital Versatile Disk). Further, the storage medium is not limited to a medium independent of a computer or an embedded system. For example, the storage medium includes a storage medium that is stored or temporarily stored by downloading a program transmitted via a LAN, the Internet, or the like.

  The program executed by each device of each of the above embodiments may be provided or distributed via a network such as the Internet.

DESCRIPTION OF SYMBOLS 1 Store system 10 Conveyance apparatus 10a Rotation lane 20 Camera 30 Notification apparatus 40 Merchandise management apparatus 411 Image taking-in part 412 Dish detection part 413 Merchandise specification part 414 Shape determination part 415 Operation | movement detection part 416 Merchandise management part 417 Notification processing part

JP 2006-20881 A

Claims (5)

A product management device for managing products transported in a state of being placed on a transport body,
Capture means for capturing a captured image from an imaging device that captures an image on the transport path of the transport body;
First detection means for detecting a first region representing the carrier from the captured image captured by the capture means;
Second detection means for detecting a second region representing the product from the first region;
Third detection means for detecting the first region moved in another direction different from the transport direction of the transport path;
Fourth detection means for detecting fraud based on the detection results of the second detection means and the third detection means for the first region detected by the first detection means;
Equipped with a,
The fourth detection means detects the first area where the third detection means has moved from outside the conveyance path into the conveyance path, and the second detection means cannot detect the second area from the first area. And a product management device that determines that fraud has been detected . Further comprising a specifying means for specifying the product represented in the second area,
The fourth detection means detects the first area where the third detection means has moved from outside the conveyance path into the conveyance path, and the second detection means detects the second area from the first area, The merchandise management apparatus according to claim 1 , wherein when the identifying unit cannot identify the merchandise represented in the second area, it is determined that an illegal act has been detected. A determination means for determining whether the outer shape of the product represented in the second area matches a predetermined appropriate outer shape;
The fourth detection means detects the first area where the third detection means has moved from outside the conveyance path into the conveyance path, and the second detection means detects the second area from the first area, The merchandise management apparatus according to claim 1 , wherein when the determination unit determines that the second area is different, it determines that an illegal act has been detected. The third detection means detects the second region moved in the other direction from the captured image,
The fourth detection means detects the second area where the third detection means has moved from the inside of the conveyance path to the outside of the conveyance path, and does not detect the movement of the first area as a detection source of the second area. The merchandise management apparatus according to any one of claims 1 to 3 , wherein it is determined that fraud has been detected. A computer that manages the products that are transported while being placed on the transport body,
Capture means for capturing a captured image from an imaging device that captures an image on the transport path of the transport body;
First detection means for detecting a first region representing the carrier from the captured image captured by the capture means;
Second detection means for detecting a second region representing the product from the first region;
Third detection means for detecting the first region moved in another direction different from the transport direction of the transport path;
Fourth detection means for detecting fraud based on the detection results of the second detection means and the third detection means for the first region detected by the first detection means;
To function ,
The fourth detection means detects the first area where the third detection means has moved from outside the conveyance path into the conveyance path, and the second detection means cannot detect the second area from the first area. And a program that determines that fraud has been detected .

Images