JP7510862B2 - Product reading device and product reading program - Google Patents

Description

Embodiments of the present invention relate to a product reading device and a product reading program.

Conventionally, supermarkets and other stores have sold products at discounts by attaching discount labels to fresh produce and other products that display the discount amount or discount rate. Discount labels are used to promote sales of products that are nearing closing time or their sell-by date in order to reduce waste of fresh produce. It is not determined in advance which products and when discount labels will be attached. For this reason, the POS terminal that processes the payment for products needs to reliably recognize the display content of the discount labels affixed to the products when processing the payment for the products.

The POS terminal is equipped with a product reader for reading product information. The product reader recognizes the barcode indicating the product code and the discount label from an image captured by an imaging element such as a CCD image sensor. The product reader outputs the product code identified from the barcode and the discount information indicated by the discount label to the POS terminal. The POS terminal performs a payment process including a discount on the product based on the received product code and discount information.

Conventional product reading devices read the barcode first and then the discount label in order to associate the product with the discount label. In this case, in order to quickly read the discount label and the barcode in a specified order, the operator must be aware of the discount label attached to the product when operating the device. For example, in a self-checkout register operated by a general customer, there is a problem that it is difficult to quickly read the discount label and the barcode in a specified order.

JP 2012-113689 A

The problem that this invention aims to solve is to provide a product reading device and a product reading program that are highly convenient and allow greater freedom when reading information from products.

According to an embodiment, a commodity reading device has a memory, an imaging device, an operation device, and a processor. The memory holds information. The imaging device captures an image of a code obtained by encoding commodity identification information attached to a commodity and a label on which additional information is written. The operation device includes a confirmation key that instructs the end of a reading process for one commodity. The processor stores in the memory the commodity identification information indicated by the code extracted from an image captured by the imaging device, stores in the memory a recognition result of the additional information written on the label extracted from the image captured by the imaging device, and, when an instruction input to the confirmation key of the operation device is obtained as a confirmation trigger , outputs the commodity identification information and the additional information stored in the memory as commodity information for one commodity.

FIG. 1 is a diagram showing the external appearance of a checkout device including a commodity reading device according to this embodiment. FIG. 2 is a block diagram showing an example of the configuration of a control system in a checkout device including a commodity reading device according to this embodiment. FIG. 3 is a flowchart for explaining a first operation example of the checkout device including the commodity reading device according to this embodiment. FIG. 4 is a flowchart for explaining a second operation example of the checkout device including the commodity reading device according to this embodiment. FIG. 5 is a flowchart for explaining a third operation example of the checkout device including the commodity reading device according to this embodiment.

Hereinafter, the present embodiment will be described with reference to the drawings.
First, a configuration example of the checkout device 2 according to the embodiment will be described.
FIG. 1 is a diagram showing the appearance of a checkout device 2 according to this embodiment.
The checkout device 2 is an example of a device (system) including a product reading device. The checkout device 2 has a computer that executes programs as a control device. The checkout device 2 executes various programs to execute product reading (registration) processing and transaction processing. The product reading device serving as the checkout device 2 according to the embodiment executes the product reading processing by a computer executing a product reading program.

The checkout device 2 shown in FIG. 1 is a self-checkout device (self-register) in which product registration processing and accounting processing are performed by the operation of a customer who purchases a product. However, the product reading device according to the embodiment is not limited to being applied to a self-register. For example, the product reading device according to the embodiment may be applied to a checkout device (POS system) operated by a staff member (store clerk).

As shown in FIG. 2, the checkout device 2 comprises a main body 10, and a first product placement table 11 and a second product placement table 12 arranged on the left and right sides of the main body 10. In this embodiment, the first product placement table 11 is a product placement area for placing products before the reading process. The second product placement table 12 is intended to place products after the reading process (registration process). However, products before the reading process may be placed in a shopping cart (hereinafter, cart) and placed in a predetermined position (product placement area).

The first product placement stand 11 is disposed on the right side of the main body 10 when facing the main body 10. For example, a basket (container) containing products (products before the reading process) that a customer wishes to purchase is placed on the first product placement stand 11. The first product placement stand 11 may also be used as a platform on which products (products before the reading process) that a customer wishes to purchase are placed directly.

The second product placement stand 12 is disposed on the left side of the main body 10 when facing the main body 10. The second product placement stand 12 is a position to which products that have been read (registered) are moved. For example, products that have been removed from the first product placement stand 11 and have been read (registered) are placed on the second product placement stand 12. The second product placement stand 12 may also be used to place a product storage unit such as a shopping bag or take-out basket that contains the products that have been read (registered). In this case, the products that have been read (registered) are stored (placed) in the product storage unit placed on the second product placement stand 12.

A scale 29 is provided on the second product placement table 12. The scale 29 detects that a product has been placed on the second product placement table 12 (that a product has been stored in a product storage section placed on the second product placement table 12). The scale 29 not only detects that the weight of the product placed on the table has increased, but also changes the measurement value depending on the operation of placing a product on the table (or storing a product in the product storage section). For example, the scale 29 detects a change in the measurement value in response to the customer's operation of placing a product on the second product placement table 12, even if the product is lightweight.

Furthermore, in the configuration example shown in FIG. 1, the second product placement stand 12 has a temporary placement stand 12a provided on the upper part via a support. The temporary placement stand 12a is used, for example, to temporarily place products before placing them in a shopping bag. The temporary placement stand 12a has two holding arms 12b spaced apart on the left and right. The holding arms 12b engage the product storage bag (shopping bag) serving as the product storage section in an open state.

The scale 29 also functions to measure the weight applied to the support on which the temporary placement table 12a and the holding arm 12b are mounted. The scale 29 can also measure the weight of the product on the temporary placement table 12a and the weight of the product placed in the product storage bag held by the holding arm 12b. With this function, the scale 29 can also detect the change in weight caused by placing a product on the temporary placement table 12a or placing a product in the product storage bag held by the holding arm 12b. In other words, the scale 29 measures the change in weight applied to the second product placement table 12, the temporary placement table 12a, and the holding arm 12b.

The main body 10 is provided with a display 21, a card reader/writer (RW) 22, a printer 23, a speaker 24, a scanner 25, a hand scanner 26, and a cash processing machine 27. A camera 28 is provided on the top of the main body 10 via a support. Furthermore, a computer that realizes a control unit 30 as shown in FIG. 2, which will be described later, is provided within the main body 10. The display 21, the card reader/writer (RW) 22, the printer 23, the speaker 24, the scanner 25, the hand scanner 26, the cash processing machine 27, the camera 28, and the scales 29 are electrically connected to the control unit 30 provided within the main body 10.

The display 21 is a touch panel including a display device 21a and a touch sensor 21b. The display device 21a displays operation guides, icons (touch keys) that can be selected by the customer, or warning messages as alerts. The display device 21a is, for example, an LCD (Liquid Crystal Display).

The touch sensor 21b is an example of an operation device. The touch sensor 21b detects the position of a person's touch on the display screen (operation screen) of the display device 21a. The touch sensor 21b sends information indicating the detected touch position to a processor 40 in the control unit 30, which will be described later and will perform processing to identify the information.

For example, the display device 21a displays a GUI (Graphical User Interface) for inputting various information and selecting instructions such as a payment method (cash, card) by a customer's touch operation. The display device 21a also displays a guidance screen, an information input screen, a product registration screen, a checkout screen, and the like. The guidance screen is a screen that displays guidance (messages) to inform the customer of the operation method. The information input screen is, for example, an operation screen for inputting information on a product that does not have a barcode. The product registration screen is a screen that displays information on a registered product. The product registration screen displays, for example, information on a product identified by a barcode read by the scanner 25 or the hand scanner 26, additional information such as a discount amount applied to the product, or a product specified by key operation. The checkout screen is a screen that displays the checkout amount (price, discount amount, tax rate) for each product, the total checkout amount for the products to be purchased (registered products), the deposit amount, the amount of change, and the like in the accounting process in which the customer pays for the products.

The card reader/writer (RW) 22 processes cards. The card reader/writer 22 has a card insertion slot. The card reader/writer 22 has the function of reading data recorded on a card and the function of writing data to a card. The card reader/writer 49 may be a magnetic card that records magnetic information, or a contact or non-contact IC card. The card reader/writer 22 may also have a mechanism for processing multiple types of cards.

For example, the card reader/writer 22 processes cards such as credit cards or membership cards for settlement that are inserted into the card insertion slot. Examples of cards for settlement (payment) include credit cards, debit cards, electronic money cards, and prepaid cards. Examples of membership cards include member's cards that record member information, or point cards that record points that can be used as a service.

The printer 23 prints receipts. The printer 23 has a receipt issuing port. The printer 23 issues receipts by printing various characters, images, and the like on receipt paper. The printer 23 may be, for example, a thermal printer or a dot-matrix printer. For example, when a transaction process is completed, the printer 23 creates a receipt on which the transaction details (transaction details) are printed and ejects the created receipt from the receipt issuing port.
The speaker 24 outputs sounds such as a warning sound as an alert or a voice message.

Scanner 25 and hand scanner 26 are composed of imaging devices having imaging elements such as CCD image sensors. Scanner 25 and hand scanner 26 capture images showing product identification information that identifies the product, and images showing additional information such as discounts applied to the product.

For example, the scanner 25 and the hand scanner 26 capture an image of a code (barcode) indicating the product identification information as an image indicating the product identification information. Here, the code attached to the product is displayed on the surface of the product and is information (optical code) that allows the product identification information to be optically read. For example, the code may be a barcode or a two-dimensional barcode.

Furthermore, scanner 25 and hand scanner 26 capture images of labels (additional information, discount labels) that contain additional information such as discounts applied to products as images showing additional information. Labels attached to products are, for example, affixed to the surface of a specific product (product packaging) by a store staff member at a time determined by the store selling the product. The label may be a discount label (discount sticker) that contains discount information such as the discount amount or discount rate for the product. The label may also be a label that indicates the tax rate applied to the product.

The scanner 25 captures an image including the product's code and label at a predetermined imaging position (reading position). For example, the scanner 25 captures an image including the code and label of a product that is held up by a customer so that the code and label attached to the product face each other. From the image captured by the scanner 25, the product identification information indicated by the code and the additional information (discount information) written on the label are recognized.

The hand scanner 26 is handheld and operated by the customer. When the hand scanner 26 is brought close to the code and discount label attached to the product, it takes an image including the code and discount label. The product identification information indicated by the barcode and the discount information (additional information) indicated by the discount label are also recognized from the image taken by the hand scanner 26.

The scanner 25 and the hand scanner 26 may capture an image including both an image of the barcode (code) attached to the product and an image of the additional information label (discount label) attached to the product. The scanner 25 and the hand scanner 26 may also capture an image including the barcode (code) attached to the product and an image of the additional information label (discount label) attached to the product in sequence.

The cash processing machine 27 processes cash. The cash processing machine 27 has a banknote input port, a banknote discharge port, a coin input port, and a coin discharge port. The cash processing machine 27 processes banknotes inserted into the banknote input port 27a. The cash processing machine 27 discharges banknotes as change from the banknote discharge port 27b. The cash processing machine 27 accepts and processes coins inserted into the coin input port 27c. The cash processing machine 27 discharges coins as change from the coin discharge port 27d.

Camera 28 is set to capture an image of a shooting range including a work area where customers perform tasks (operations) such as scanning products and processing the transaction for products to be checked out. It is installed to capture an image of the area where customers perform tasks (operations) related to scanning products and processing the transaction.

The camera 28 captures images of the customer's actions (work) in reading (registering) products and processing the transaction. For example, the camera 28 is installed so that the first product placement table 11 and the second product placement table 12 are included in the range of its image capture. The camera 28 installed in this manner can capture images of the period from when a product is placed on the first product placement table 11 to when it is placed on the second product placement table 12. The images captured by the camera 28 are temporarily stored in the RAM 42 and then saved in the data memory 43 or the like.

Next, the configuration of a control system in the checkout device 2 according to this embodiment will be described.
FIG. 2 is a block diagram showing an example of the configuration of a control system in the checkout device 2 according to this embodiment.
2, the main body 10 of the checkout device 2 has a control unit 30. The control unit 30 is realized by, for example, a computer. A display 21, a card reader/writer 22, a printer 23, a speaker 24, a scanner 25, a hand scanner 26, a cash processing machine 27, a camera 28, a scale 29, and the like are connected to the control unit 30. The control unit 30 also has a processor 40, a read-only memory (ROM) 41, a random-access memory (RAM) 42, a data memory 43, and various interfaces (IFs) 44 to 53.

The processor 40 executes various processes by executing programs. The processor 40 is, for example, a CPU (central processing unit). The processor 40 realizes various processing functions by executing programs stored in the ROM 41 or the data memory 43. For example, the processor 40 executes a program for product reading (registration) processing (product reading program) to execute a product reading (registration) process that reads (registers) product information for each product. The processor 40 executes a program for accounting processing to execute an accounting process that settles the settlement amount based on the product information of the target product obtained by the product reading process.

The processor 40 also functions as an image analysis device that performs image analysis processing by executing a program for image analysis. As part of the image analysis processing, the processor 40 analyzes the movements of the person (customer or store clerk) operating the checkout device 2 from the image acquired from the camera 28. For example, the processor 40 executes a program for motion recognition to perform processing to recognize the movements of the person in the work area corresponding to the checkout device 2. In this embodiment, the processor 40 detects that the person has stored the product that has been read in the product storage section set on the second product placement platform 12 (moved the product for which reading processing has been completed).

The image analysis process as described above may be performed by an image analysis device serving as an external device connected to the processor 40 via the communication IF 53 or the like. In this case, the processor 40 may supply the image captured by the camera 28 to the image analysis device connected via the communication IF 53, and obtain the results of the image analysis from the image analysis device.

The ROM 41 is a non-volatile memory that stores programs and data. For example, the ROM 41 stores programs such as an operating system, middleware, and applications. The ROM 41 may also store data that the processor 40 references when executing various processes.
The RAM 42 stores working data, such as data referenced by the processor 40 when the processor 40 executes various processes, or data to be used temporarily.

The data memory 43 is a memory that stores data. The data memory 43 is a rewritable non-volatile memory. The data memory 43 is composed of, for example, an EEPROM (registered trademark) (electric erasable programmable read-only memory), a hard disk drive (HDD), or a solid state drive (SSD). The data memory 43 stores data used when the processor 40 executes various processes, or data generated by the processes by the processor 40. The data memory 43 may also store various programs executed by the processor 40. For example, the data memory 43 may store a program for product reading processing (product reading program), a program for accounting processing, or a program for image analysis (program for action recognition).

The RAM 42 or data memory 43 also functions as a cache memory in the product reading process of the first, second or third operation example described below. The RAM 42 or data memory 43 as a cache memory stores (caches) information about a product (product information) including product identification information and discount information (additional information) for one product that is the subject of the reading process.

The RAM 42 or data memory 43 may also be configured to record, for example, product information on each product that has been read (registered) as list-format data. The RAM 42 or data memory 43 also stores images captured by the camera 28. The data memory 43 may also be configured to store, for example, video data in which timestamp data indicating the time of capture has been added to the images captured by the camera 28.

The processor 40 connects to the scanner 25 via the scanner IF 44. The processor 40 acquires the image (scanned image) read by the scanner 25 via the scanner IF 44. The processor 40 also connects to the hand scanner 26 via the scanner IF 45. The processor 40 acquires the captured image (scanned image) captured by the hand scanner 26 via the scanner IF 45.

The processor 40 extracts the barcode and discount label contained in the image captured by the scanner 25 or hand scanner 26. The processor 40 obtains the product identification information indicated by the barcode by decoding the barcode extracted from the captured image. The processor 40 also obtains discount information (additional information) by recognizing the information displayed on the discount label from the image of the discount label extracted from the captured image.

The processor 40 is connected to the display 21 via the display IF 46. The processor 40 controls the display content displayed by the display device 21a of the display 21 via the display IF 46. For example, the processor 40 displays a message on the display 21 as an alert or indicator to notify the user that a product that has not yet been read (unregistered) is present.

The processor 40 also has a touch sensor 21b as an operation device. The processor 40 acquires information indicating the touch position detected by the touch sensor 21b of the display 21 via the display IF 46. The processor 40 detects input to each icon (touch key) displayed on the display 21 based on the information indicating the touch position detected by the touch sensor 21b. For example, the processor 40 displays a confirmation key on the display 21 to instruct the end of reading of each product, and detects the input of an instruction to the confirmation key by the touch sensor 21b. The checkout device 2 may also be provided with a hard key (button) that functions as a confirmation key as an operation device.

The processor 40 is connected to the card reader/writer 22 via the card IF 47. The processor 40 controls processing of the card by the card reader/writer 22 via the card IF 47. For example, the processor 40 executes a credit card settlement process processed by the card reader/writer 22.
The processor 40 is connected to the printer 23 via the printer IF 48. The processor 40 controls the printer 23 via the printer IF 48. For example, the processor 40 causes the printer 23 to issue a receipt indicating the result of the transaction.
The processor 40 is connected to the speaker 24 via the audio IF 49. The processor 40 controls the sound output from the speaker 24 via the audio IF 49. For example, the processor 40 may output from the speaker 24 an alert indicating that discount information displayed by the discount label has been read.

The processor 40 connects to the cash processing machine 27 via the deposit/withdrawal IF 50. The processor 40 controls the cash processing machine 27 via the deposit/withdrawal IF 50. For example, the processor 40 causes the cash processing machine 27 to count the cash deposited by the customer, and acquires information indicating the amount of the deposit counted by the cash processing machine 27. The processor 40 also controls the cash processing machine 27 to dispense change calculated from the amount deposited and the total amount (product price) of registered products.

The processor 40 connects to the camera 28 via the camera IF 51. The processor 40 acquires images captured by the camera 28 via the camera IF 51. For example, the processor 40 acquires images captured by the camera 28 including an image of the inside of a basket (container) on the first product placement table 11 when transitioning to checkout processing. The processor 40 also acquires images captured by the camera 28 continuously (e.g., at a predetermined interval) during the registration process including an image of the inside of a basket (container) on the first product placement table 11.

The processor 40 is connected to the scale 29 via the scale IF 52. The processor 40 acquires information indicating the weight value detected (measured) by the scale 29 via the scale IF 52. The processor 40 may also acquire information indicating that the weight measured by the scale 29 has changed via the scale IF 52.

The processor 40 connects to an external device via the communication IF 53. For example, the processor 40 connects to an attendant terminal as an external device via the communication IF 53. The attendant terminal is a device that displays information indicating the usage status, such as the product reading (registration) status or the transaction processing execution status, of the checkout device 2. The processor 40 transmits information indicating the operation status to the attendant terminal 5 via the communication IF 53.

The processor 40 may also be connected to a computer such as a management server as an external device via the communication IF 53. For example, the processor 40 may transmit product information determined in the reading (registration) process or information indicating products that have been checked out in the checkout process to the management server via the communication IF 53. The processor 40 may also appropriately acquire information about products from a store computer with which it communicates via the communication IF 53.

Next, a first, second and third operational examples of the checkout device 2 configured as above will be described.
First, a first operation example of a process including a product reading (registration) process in the checkout device 2 as a product reading device according to the embodiment will be described.
FIG. 3 is a flowchart for explaining a first operation example of a product reading (registration) process in the checkout device 2 serving as the product reading device according to the embodiment.
Here, it is assumed that a customer approaches the checkout device 2 with a product to be purchased. The customer sets the product to be purchased (product before registration) in a predetermined position. For example, the customer places a basket (container) containing the product to be purchased (target product) on the first product placement table 11, which serves as a product placement area. The customer may also place the target product directly on the first product placement table 11. The customer may also place a cart containing the target product in a predetermined position, which serves as a product placement area.

After placing the target product in the product placement area, the customer instructs the start of the product scanning process (or checkout process) by performing a predetermined operation. For example, the customer instructs the start of the product scanning process by touching a start key displayed on the display 21. The processor 40 determines the start of the product scanning process by detecting, via the touch sensor 21b, that the start key instructing the start of the product scanning process has been touched.

The start of the product reading process may also be recognized by the processor 40 in response to the customer's movements. For example, the processor 40 may recognize the movements of a person (customer) who has entered the work area by analyzing images captured by the camera 28, and may determine the start of the product reading process in response to the recognized movement of the person.

The processor 40 of the checkout device 2 starts the product reading process to read the product identification information and additional information (discount information) from the target product in response to an instruction to start the product reading process. In the product reading process, the processor 40 performs a process to obtain the product identification information and discount information for each individual product (ACTS 10 to 14).

In the first operation example, the processor 40 assumes that the customer inputs the confirmation key when the reading of the product identification information and discount information for one product is completed. The processor 40 performs a process of acquiring the product identification information and discount information (ACTs 10 to 14) until the confirmation key is input, indicating that the reading (registration) of each product is completed (ACT 15, NO).

That is, the processor 40 acquires an image of each product captured by the scanner 25 (or the hand scanner 26) in response to the customer's operation (ACT 10). The processor 40 performs a process to acquire product identification information and discount information based on the captured image acquired from the scanner 25 (ACT 11 to 14).

For example, the processor 40 extracts an image of a barcode indicating product identification information and an image of a discount label indicating discount information from the photographed image obtained by the scanner 25. If the processor 40 can extract an image of a barcode from the photographed image, the processor 40 reads (decodes) the barcode extracted from the photographed image. The processor 40 determines whether the information obtained by decoding the barcode is valid product identification information. If valid product identification information is obtained from the barcode extracted from the photographed image, the processor 40 determines that the product identification information of the product has been obtained.

When the processor 40 determines that the product identification information of the product has been obtained (ACT 11, YES), the processor 40 saves the product identification information obtained from the barcode in the captured image in a cache memory (ACT 12). For example, the processor 40 stores the product identification information obtained from the captured image of the product in the RAM 42 or data memory 43 as a cache memory. As a result, the processor 40 holds the product identification information indicated by the barcode attached to the product in the cache memory.

When product identification information different from the product identification information stored in the cache memory is acquired, that is, when product identification information of a different product is acquired, the processor 40 determines that product identification information of the next product has been acquired. In this case, the processor 40 may determine that there is a product reading error, or may determine that the information already stored in the cache memory (product identification information and discount information) is one piece of product information.

In the former case (when acquiring product identification information of a different product results in a product reading error), the operation may be, for example, to notify the user of the product reading error and prompt the user to enter the Confirm key or to try again to read the product. Such an operation can be achieved by the processor 40 displaying on the display 21 a reading error and a guide to enter the Confirm key or to try again to read the product. By operating in this manner, the user can reliably confirm that reading of the product identification information and discount information for one product has been completed by entering the Confirm key.

In the latter case (when acquiring product identification information of another product, the information already held in the cache memory is confirmed as information for one product), the operation may be to output the held information (product identification information and discount information) as one piece of product information, and to newly hold the acquired product identification information in the memory. For example, the processor 40 outputs the product identification information and discount information already held in the cache memory as one piece of product information (registers it as product information). After outputting the information held in the memory as one piece of product information, the processor 40 holds the acquired product identification information as information for the next product in the cache memory. This allows the checkout device 2 to operate as finishing reading the product identification information and discount information for one product when it reads the product identification information (barcode) of the next product.

Furthermore, when processor 40 extracts an image of a discount label from the captured image, it recognizes the information written on the extracted discount label. Processor 40 acquires the information written on the discount label by performing character recognition on the image of the discount label, for example. Processor 40 determines whether the information acquired by recognizing the discount label is valid discount information (additional information). When the information acquired as a result of recognizing the discount label extracted from the captured image is valid discount information, processor 40 determines that it has acquired the discount information attached to the product.

When the processor 40 determines that valid discount information has been acquired from the recognition result of the discount label attached to the product (ACT 13, YES), the processor 40 stores the discount information acquired from the discount label in the captured image in a cache memory (ACT 14). For example, the processor 40 stores the discount information acquired from the captured image of the product in the RAM 42 or the data memory 43 as a cache memory. This allows the processor 40 to store the discount information indicated by the discount label attached to the product in the cache memory.

When discount information different from discount information already stored in the cache memory is acquired, the processor 40 determines that a discount label other than the previously read label has been read. In this case, the processor 40 may determine that there is a discount label reading error, or may store the discount information indicated by the different (second) discount label for the product in the cache memory.

In the former case (when the reading of another discount label is deemed an error), the operation may be, for example, to notify the user of a read error and prompt the user to enter the Confirm key or to try reading the product again. Such an operation can be achieved by the processor 40 displaying on the display 21 a message indicating an error in reading the discount information and a guide to entering the Confirm key or to try reading the product again. By operating in this manner, the user can reliably confirm that reading of one discount label for one product has been completed by entering the Confirm key.

In the latter case (when the recognition result of another discount label is added to the cache), for example, the processor 40 may store the obtained recognition result of the discount label in the cache memory as another (second) discount information for one product. With this type of operation, even if multiple additional information labels (multiple discount labels, or a discount label and another additional information label) are attached to one product, the information written on the multiple labels can be applied to the calculation of the settlement amount.

The processor 40 also accepts input to the confirmation key by the customer while executing the processes of ACTs 10 to 14. For example, the processor 40 detects whether or not the confirmation key has been pressed using the touch sensor 21b as an operation device.

If there is no input to the Confirm key (ACT 15, NO) or no instruction to transition to checkout processing (ACT 17, NO), the processor 40 returns to ACT 10 and repeatedly executes the process of reading the product identification information and discount information for the product. That is, the processor 40 repeatedly executes the input of the product identification information indicated by the barcode and the discount information indicated by the discount label for one product until the Confirm key is input. As a result, the checkout device 2 acting as a product reading device can read the product identification information and discount information for one product in any order.

Furthermore, when the processor 40 detects input to the Confirm key (ACT 15, YES), it outputs the product identification information and discount information of the product stored in a memory such as the RAM 42 (ACT 16). In response to the input to the Confirm key, the processor 40 confirms (registers) the product identification information and discount information acquired from the photographed image of the product as the product information of the product (information for paying for the product). In other words, the checkout device 2 in the first operation example confirms and registers the product identification information and discount information for one product in response to the input to the Confirm key.

When the processor 40 outputs the product identification information and discount information for the product, it checks whether there is an instruction to proceed to checkout processing (ACT 17). If there is no instruction to proceed to checkout processing (S17, NO), the processor 40 returns to ACT 10 and executes the process of reading the product identification information and discount information for the next product.

When a customer determines that scanning (registration) of all products to be purchased is complete, the customer instructs the transition to checkout processing (end of product scanning (registration)). For example, the customer instructs the transition to checkout processing by operating the end scanning (registration) key, which is a touch key displayed on the display 21. In this case, the processor 40 determines the transition to checkout processing by detecting input to the end scanning (registration) key by the touch sensor 21b.

If a transition to checkout processing (or the end of product registration processing) is instructed (ACT 17, YES), the processor 40 executes checkout processing for all products whose product information has been registered (ACT 18). In the checkout processing, the processor 40 determines the settlement amount for each registered product from the product price based on the product identification information and the discount amount based on the discount information (additional information). In addition, if a tax rate or the like is specified as additional information, the processor 40 calculates the settlement amount for each product by adding the tax amount based on the tax rate.

After calculating the settlement amount for all items to be purchased, processor 40 calculates the total settlement amount for all items to be purchased. Processor 40 displays the calculated total amount on display 21 and performs processing to collect the total settlement amount for all items to be purchased from the customer. Once the total amount for all items to be purchased has been collected, processor 40 ends the accounting processing for the items to be purchased.

According to a checkout device (product reading device) to which the above-described first operation example is applied, the end of reading information on one product is confirmed when the confirmation key is entered. This allows the product reading device to read the code and label attached to one product in any order until the confirmation key is entered. As a result, the product reading device can reliably obtain not only the product identification information indicated by the barcode attached to each product, but also additional information (such as discount information) written on the attached label. Furthermore, an operator such as a customer can operate the product reading device to read the code and label attached to one product using a highly flexible operating procedure.

Next, a second operation example of the checkout device 2 as a commodity reading device according to the embodiment will be described.
FIG. 4 is a flowchart for explaining a second operation example of the process including the product reading (registration) process in the checkout device 2 as the product reading device according to the embodiment.
First, the processor 40 of the checkout device 2 starts the product reading process to read the product identification information and additional information (discount information) from the target product in response to an instruction to start the product reading process, similar to the first operation example. In the product reading process, the processor 40 performs a process to obtain the product identification information and discount information for each product (ACTS 20 to 24).

In the second operation example, the processor 40 detects that reading of the product identification information and discount information for one product has been completed based on the weight change measured by the scale 29. That is, the processor 40 performs a process of acquiring the product identification information and discount information for one product (ACT 20 to 24) until it detects that reading of one product has been completed based on the weight change measured by the scale 29 (ACT 25, NO).

When the product scanning process starts, the processor 40 acquires images of individual products captured by the scanner 25 (or the hand scanner 26) in response to the user's operation (ACT 20). The processor 40 performs a process to acquire product identification information and discount information based on the captured images acquired from the scanner 25 (ACT 21-24).

For example, the processor 40 extracts an image of a barcode indicating product identification information and an image of a discount label indicating discount information from the photographed image obtained by the scanner 25. If the processor 40 can extract an image of a barcode from the photographed image, the processor 40 reads (decodes) the barcode extracted from the photographed image. The processor 40 determines whether the information obtained by reading the barcode is valid product identification information. If valid product identification information is obtained from the barcode extracted from the photographed image, the processor 40 determines that the product identification information of the product has been obtained.

When the processor 40 determines that the product identification information of the product has been obtained (ACT 21, YES), the processor 40 saves the product identification information obtained from the barcode in the captured image in a cache memory (ACT 22). For example, the processor 40 stores the product identification information obtained from the captured image of the product in the RAM 42 or data memory 43 as a cache memory. As a result, the processor 40 holds the product identification information indicated by the barcode attached to the product in the cache memory.

When product identification information different from the product identification information stored in the cache memory is acquired, that is, when product identification information of a different product is acquired, the processor 40 determines that product identification information of the next product has been acquired. In this case, the processor 40 may determine that there is a product reading error, as in the first operation example, or may determine that the information already stored in the cache memory (product identification information and discount information) is one piece of product information.

Furthermore, when processor 40 extracts an image of a discount label from the captured image, it recognizes the information written on the extracted discount label. Processor 40 acquires the information written on the discount label by performing character recognition on the image of the discount label, for example. Processor 40 determines whether the information acquired by recognizing the discount label is valid discount information (additional information). When the information acquired as a result of recognizing the discount label extracted from the captured image is valid discount information, processor 40 determines that it has acquired the discount information attached to the product.

When the processor 40 determines that valid discount information has been acquired from the recognition result of the discount label attached to the product (ACT 23, YES), the processor 40 stores the discount information acquired from the discount label in the captured image in a cache memory (ACT 24). For example, the processor 40 stores the discount information acquired from the captured image of the product in the RAM 42 or the data memory 43 as a cache memory. This allows the processor 40 to store the discount information indicated by the discount label attached to the product in the cache memory.

When discount information different from the discount information already stored in the cache memory is acquired, the processor 40 determines that a discount label other than the previously read label has been read. In this case, the processor 40 may determine that there is a discount label reading error, as in the first operation example, or may store the additional (discount) information indicated by the other (second) label for the product in the cache memory.

In the second operation example, while executing the processes of ACT 20 to ACT 24, the processor 40 detects a change in weight based on the measurement value measured by the scale 29 (ACT 25). The processor 40 detects that one product has moved to the second product placement platform 12, which is the position after reading, based on the change in weight measured by the scale 29. In other words, the processor 40 detects the change in weight measured by the scale 29 as a confirmation trigger indicating that the reading process for one product has ended.

If the processor 40 does not detect a weight change that serves as a confirmation trigger (ACT 25, NO) or if there is no instruction to transition to checkout processing (ACT 27, NO), it returns to ACT 20 and repeatedly executes the process of reading the product identification information and discount information for the product. The processor 40 repeatedly inputs the product identification information indicated by the barcode and the discount information indicated by the discount label for one product until a weight change that indicates the product has moved to a position after reading is completed (second product placement table 12) is detected. As a result, the checkout device 2, which serves as a product reading device, can read the product identification information and discount information for one product in any order until a weight change that serves as a confirmation trigger is detected.

When the processor 40 detects a weight change as a confirmation trigger (ACT 25, YES), it outputs the product identification information and discount information (additional information) of the product stored in a cache memory such as the RAM 42 (ACT 26). In response to the detection of a weight change, the processor 40 outputs (registers) the product identification information and discount information acquired from the captured image of the product as product information of the product (information for paying for the product).

When the processor 40 outputs (registers) the product identification information and discount information for the product, it checks whether there is an instruction to proceed to checkout (ACT 27). If there is no instruction to proceed to checkout (S27, NO), the processor 40 returns to ACT 20 and executes the process of reading the product identification information and discount information for the next product.

Here, the customer (operator) who has determined that the reading (registration) of all products to be purchased has been completed instructs the transition to checkout processing (end of product reading (registration)). For example, the customer instructs the transition to checkout processing by operating the reading (registration) end key, which is a touch key displayed on the display 21. In this case, the processor 40 determines the transition to checkout processing by detecting input to the reading (registration) end key by the touch sensor 21b.

If a transition to checkout processing (or the end of product registration processing) is instructed (ACT 27, YES), the processor 40 executes checkout processing for all products whose product information has been registered (ACT 28). In the checkout processing, the processor 40 determines the settlement amount for each registered product from the product price based on the product identification information and the discount amount based on the discount information (additional information). In addition, if a tax rate or the like is specified as additional information, the processor 40 calculates the settlement amount for each product by adding the tax amount based on the tax rate.

After calculating the settlement amount for all items to be purchased, processor 40 calculates the total settlement amount for all items to be purchased. Processor 40 displays the calculated total amount on display 21 and performs processing to collect the total settlement amount for all items to be purchased from the customer. Once the total amount for all items to be purchased has been collected, processor 40 ends the accounting processing for the items to be purchased.

A checkout device (product reading device) to which the above-described second operation example is applied determines that reading of information on one product has ended when a weight change is detected at the second product placement position. This allows the product reading device of the second operation example to read the code and label attached to one product in any order until a weight change is detected at the second product placement position.

In other words, a product reading device to which the second operation example is applied can reliably obtain not only the product identification information indicated by the barcode, but also additional information (such as discount information) printed on a label that is optionally affixed to each product. Also, an operator such as a customer can use highly flexible operating procedures to cause the product reading device to read the code and label attached to the product. Furthermore, an operator such as a customer can move the product to the reading end position (second product placement position) to move on to the reading process for the next product.

Next, a third operation example of the checkout device 2 as a commodity reading device according to the embodiment will be described.
FIG. 5 is a flowchart for explaining a third operation example of a process including a product reading (registration) process in the checkout device 2 as a product reading device according to the embodiment.
First, the processor 40 of the checkout device 2 starts the product reading process to read the product identification information and additional information (discount information) from the target product in response to an instruction to start the product reading process, similar to the first operation example. In the product reading process, the processor 40 performs a process to obtain the product identification information and discount information for each product (ACTS 30 to 34).

In the third operation example, the processor 40 detects that reading of the product identification information and discount information for one product has been completed when the customer places the product in the product storage section (or moves the product to the product's position after reading has been completed (e.g., second product reading position 12)). That is, the processor 40 performs a process of acquiring the product identification information and discount information for one product (ACTs 30-34) until it detects that the person photographed by the camera 28 has placed the product in the product storage section (ACT 35, NO).

When the product scanning process starts, the processor 40 acquires images of individual products captured by the scanner 25 (or the hand scanner 26) in response to the user's operation (ACT 30). The processor 40 performs a process to acquire product identification information and discount information based on the captured images acquired from the scanner 25 (ACT 31 to 34).

For example, the processor 40 extracts an image of a barcode indicating product identification information and an image of a discount label indicating discount information from the photographed image obtained by the scanner 25. If the processor 40 can extract an image of a barcode from the photographed image, the processor 40 reads (decodes) the barcode extracted from the photographed image. The processor 40 determines whether the information obtained by reading the barcode is valid product identification information. If valid product identification information is obtained from the barcode extracted from the photographed image, the processor 40 determines that the product identification information of the product has been obtained.

When the processor 40 determines that the product identification information of the product has been obtained (ACT 31, YES), the processor 40 saves the product identification information obtained from the barcode in the captured image in a cache memory (ACT 32). For example, the processor 40 stores the product identification information obtained from the captured image of the product in the RAM 42 or data memory 43 as a cache memory. As a result, the processor 40 holds the product identification information indicated by the barcode attached to the product in the cache memory.

When acquiring product identification information different from the product identification information stored in the cache memory (when acquiring product identification information of a different product), the processor 40 determines that it has acquired product identification information of the next product. In this case, the processor 40 may determine that there is a product reading error, as in the first operation example, or may determine that the information already stored in the cache memory (product identification information and discount information) is one piece of product information.

Furthermore, when processor 40 extracts an image of a discount label from the captured image, it recognizes the information written on the extracted discount label. Processor 40 acquires the information written on the discount label by performing character recognition on the image of the discount label, for example. Processor 40 determines whether the information acquired by recognizing the discount label is valid discount information (additional information). When the information acquired as a result of recognizing the discount label extracted from the captured image is valid discount information, processor 40 determines that it has acquired the discount information attached to the product.

When the processor 40 determines that valid discount information has been acquired from the recognition result of the discount label attached to the product (ACT 33, YES), the processor 40 stores the discount information acquired from the discount label in the captured image in a cache memory (ACT 34). For example, the processor 40 stores the discount information acquired from the captured image of the product in the RAM 42 or data memory 43 as a cache memory. This allows the processor 40 to store the discount information indicated by the discount label attached to the product in the cache memory.

When discount information different from the discount information already stored in the cache memory is acquired, the processor 40 determines that a discount label other than the previously read label has been read. In this case, the processor 40 may determine that there is a discount label reading error, as in the first operation example, or may store the additional (discount) information indicated by the other (second) label for the product in the cache memory.

In the third operation example, while executing the processes of ACTs 30 to 34, the processor 40 detects an action (the action of a person placing a product in the product storage section or the action of a person moving the product to a position after reading is completed) from the image captured by the camera 28 as a confirmation trigger (ACT 35). The processor 40 continuously acquires images captured by the camera 28. The processor 40 executes an image analysis process to analyze the action of the person (customer) in the image captured by the camera 28. The processor 40 detects an action indicating that the reading process for the product has been completed from the image captured by the camera 28 as a confirmation trigger.

For example, the processor 40 can realize a process for recognizing a person's movements using a skeletal estimation technology such as OpenPose. For example, the processor 40 detects, as a confirmation trigger, the action of putting a product into a product storage section (e.g., a shopping bag (plastic bag), a take-out storage container (basket, bag, box, card, etc.)) for storing the product that the customer has set on the second product placement stand 12, etc. The processor 40 may also detect, as a confirmation trigger, the action of the customer moving a product to the second product placement stand 12.

If processor 40 does not detect a customer action that serves as a confirmation trigger (ACT 35, NO) or if there is no instruction to transition to checkout processing (ACT 37, NO), it returns to ACT 30 and repeatedly executes the process of reading the product identification information and discount information for that product. Processor 40 repeatedly inputs the product identification information indicated by the barcode and the discount information indicated by the discount label for one product until a weight change that indicates the product moving to the reading end position is detected. As a result, the checkout device 2, which serves as a product reading device, can read the product identification information and discount information for one product in any order until a weight change that serves as a confirmation trigger is detected.

When the processor 40 detects a customer action that serves as a confirmation trigger (ACT 35, YES), the processor 40 outputs the product identification information and discount information of the product stored in a cache memory such as the RAM 42 (ACT 36). In response to the customer action indicating the end of reading for one product, the processor 40 outputs (registers) the product identification information and discount information acquired from the captured image of the product as product information of the product (information for paying for the product).

When the processor 40 outputs (registers) the product identification information and discount information for the product, it checks whether there is an instruction to proceed to checkout (ACT 37). If there is no instruction to proceed to checkout (S37, NO), the processor 40 returns to ACT 30 and executes the process of reading the product identification information and discount information for the next product.

As in the first and second operation examples described above, the processor 40 determines whether to proceed to checkout by detecting the customer's input to the reading (registration) end key. If a command to proceed to checkout (or end of product registration process) is issued (ACT 37, YES), the processor 40 executes checkout for all products whose product information has been registered (ACT 38). In checkout, the processor 40 determines the settlement amount for each registered product from the product price based on the product identification information and the discount amount based on the discount information (additional information). In addition, if a tax rate or the like is specified as additional information, the processor 40 calculates the settlement amount for each product by adding the tax amount based on the tax rate.

After calculating the settlement amount for all items to be purchased, processor 40 calculates the total settlement amount for all items to be purchased. Processor 40 displays the calculated total amount on display 21 and performs a process to collect the total settlement amount for all items to be purchased from the customer. Once the total amount for all items to be purchased has been collected, processor 40 ends the accounting process for the items to be purchased.

A checkout device (product reading device) that applies the above third operation example determines that reading of information on a product has been completed when a person's movement indicating that reading of the product has been completed is detected. This allows the product reading device to read the code and label attached to a single product in any order until a person's movement indicating that reading of the single product has been completed is detected.

In other words, a product reading device to which the third operation example is applied can reliably obtain not only the product identification information indicated by the barcode, but also additional information (such as discount information) printed on a label that is optionally affixed to each product. Also, a customer or other operator can use highly flexible operating procedures to have the product reading device read the code and label attached to the product. Furthermore, when the product reading device detects an operation indicating that reading of one product has been completed, the customer or other operator can move on to the reading process of the next product without having to perform any special operations such as key input.

Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims.
The following is a summary of the contents of the claims as originally filed.
[1]
A memory for storing information;
an imaging device for imaging a code obtained by encoding product identification information attached to a product and a label on which additional information is written;
storing in the memory product identification information indicated by a code extracted from an image captured by the imaging device;
storing in the memory a recognition result of additional information written on a label extracted from an image captured by the imaging device;
a processor that outputs the product identification information and additional information stored in the memory as product information of one product when a confirmation trigger indicating the end of a reading process for one product is acquired;
A product reading device having the above structure.
[2]
The additional information is discount information for the price of the product or information indicating a tax rate for the product.
The product reading device according to [1].
[3]
The reading device further includes an operation device including a confirmation key for instructing the end of the reading process for one product,
The processor acquires an instruction input to the confirmation key of the operation device as the confirmation trigger.
The product reading device according to any one of [1] and [2].
[4]
The device further includes a scale for measuring a change in weight of the product as the product is moved to a predetermined position after the reading process is completed,
The processor acquires a change in weight measured by the scale as the determination trigger.
The product reading device according to any one of [1] and [2].
[5]
The device further includes a camera for capturing an image of a person performing a reading process on the target product,
The processor detects, as the determination trigger, an action of the person placing one product in a product storage unit that stores products, from an image captured by the camera.
The product reading device according to any one of [1] and [2].
[6]
On the computer,
Acquire an image from an imaging device that captures an image of a code in which product identification information attached to the product is encoded and a label on which additional information is written;
storing in a memory product identification information indicated by a code extracted from an image captured by the imaging device;
storing in the memory a recognition result of additional information written on a label extracted from an image captured by the imaging device;
when a confirmation trigger indicating the end of the reading process for one product is acquired, the product identification information and the additional information stored in the memory are output as product information for the one product.
A product reading program to carry out this function.

2...Checkout device (product reading device), 12...Second product placement platform, 21...Display, 21a...Display device, 21b...Touch sensor (operation device), 25...Scanner (imaging device), 26...Hand scanner (imaging device), 28...Camera, 29...Scale, 40...Processor, 41...ROM, 42...RAM, 43...Data memory.

Claims (3)

A memory for storing information;
an imaging device for imaging a code that encodes product identification information attached to a product and a label on which additional information is written;
an operation device including a confirmation key for instructing the end of the reading process for one product;
storing in the memory product identification information indicated by a code extracted from an image captured by the imaging device;
storing in the memory a recognition result of additional information written on a label extracted from an image captured by the imaging device;
a processor that outputs the product identification information and additional information stored in the memory as product information of one product when an instruction input to the confirmation key of the operation device is acquired as a confirmation trigger ;
A product reading device having the above structure. The additional information is discount information for the price of the product or information indicating a tax rate for the product.
The product recognition device according to claim 1 . On the computer,
Acquire an image from an imaging device that captures an image of a code in which product identification information attached to the product is encoded and a label on which additional information is written;
storing in a memory product identification information indicated by a code extracted from an image captured by the imaging device;
storing in the memory a recognition result of additional information written on a label extracted from an image captured by the imaging device;
when an input of an instruction to a confirmation key of an operation device including a confirmation key for instructing to end a reading process for one product is acquired as a confirmation trigger , the product identification information and the additional information stored in the memory are output as product information of the one product;
A product reading program to carry out this function.