JP7699801B2 - Product sales data processing device and program - Google Patents

Description

The present invention relates to a product sales data processing device and a program.

Conventionally, in various stores such as supermarkets and convenience stores, transactions are carried out using POS (Point Of Sales) terminals. The POS terminals perform product registration and payment processes. In addition to POS terminals in which a store clerk registers products and performs payment, there are also so-called self-checkouts that are operated by customers themselves to register products and perform payment (see, for example, Patent Document 1). Self-checkouts are equipped with a monitoring terminal that displays the operating status of each self-checkout, and a store clerk is stationed at the monitoring terminal to monitor the self-checkout.

Japanese Patent Application Publication No. 10-069574

However, with conventional technology, the burden on store staff assigned to the monitoring terminal can increase because they have to perform a variety of tasks, such as assisting customers with their operations, performing recovery work, and monitoring for fraud. Also, when operational assistance is required at multiple self-checkout registers at the same time, one customer ends up having to wait. As a result, conventional technology has the problem that transactions cannot always be carried out smoothly.

The present invention was made in consideration of these circumstances, and its purpose is to provide technology that allows transactions to be carried out smoothly at self-checkout registers.

In order to solve the above-mentioned problems, a product sales data processing device according to one aspect of the present invention includes a detection means for detecting a product present in an identifier reading area, a product registration means for registering product information based on the identifier read in the reading area, an image information acquisition means for acquiring image information capturing an image of a customer reading the identifier in the reading area, and a display control means for displaying on a display means a product list image based on the product information registered by the product registration means and an image based on the image information acquired by the image information acquisition means, wherein the display control means displays the product list image and the image in separate display areas if the product information is registered by the product registration means when a product is detected by the detection means, and displays the image in a superimposed manner on the product list image if the product information is not registered by the product registration means despite the product being detected by the detection means.

1 is a network configuration diagram of a POS (Point Of Sales) system according to an embodiment of the present invention. FIG. 2 is an explanatory diagram showing an example of installation of a POS terminal device 20. FIG. 2 is a block diagram showing the hardware configuration of the POS terminal device 20. 2 is an explanatory diagram showing an example of a reading area 400 of a scanner unit 206. FIG. 10 is a flowchart showing an example of a captured image display control process performed by the POS terminal 20. 11 is an explanatory diagram showing an example of a screen transition relating to display control of a captured image displayed on an operation display unit 205. FIG. 11 is an explanatory diagram showing an example of a screen transition relating to display control of a captured image displayed on an operation display unit 205. FIG. 10 is a flow chart showing an example of a captured image display control process performed by a POS terminal device 20 according to a first modified example. 13A to 13C are explanatory diagrams showing an example of a screen transition according to Modification 1 displayed on the operation display unit 205. 13A to 13C are explanatory diagrams showing an example of a screen transition according to Modification 2 displayed on the operation display unit 205. FIG. 13 is an explanatory diagram showing an example of a screen according to Modification 4 displayed on the operation display unit 205; FIG. 13 is an explanatory diagram showing an example of a screen according to Modification 5 that is displayed on the operation display unit 205;

(Embodiment)
Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.
Fig. 1 is a network configuration diagram of a POS (Point Of Sales) system according to an embodiment of the present invention. The POS system 1 can be introduced into various stores, such as supermarkets, home improvement centers, convenience stores, and various supply stores. As shown in Fig. 1, the POS system 1 includes a store controller 10, a transaction status management device 11, and three POS terminal devices 20-1, 20-2, and 20-3. Hereinafter, when there is no particular distinction between the POS terminal devices 20-1, 20-2, and 20-3, they will be collectively referred to as the POS terminal device 20.

Each device is a computer device equipped with a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), a communication unit, etc. Each device is connected by wire or wirelessly via a communication line 55 such as a LAN (Local Area Network) or a WAN (Wide Area Network).

The store controller 10 stores various information necessary for transactions, such as a product master. The product master is a file that stores product information such as product identification information (e.g., JAN (Japanese Article Number) code), product name, and sales price for each product. The product master also includes product identification information for products that require weighing (products to be weighed), product names, and unit prices of the products to be weighed. In addition to the product master, the store controller 10 also stores various information such as inventory status, sales history, and deposit and withdrawal records. The store controller 10 may also store customer membership information.

The transaction status management device 11 displays the status (processing status, etc.) of the POS terminal device 20 and controls the POS terminal device 20. In the POS system 1, a store clerk can be stationed at the transaction status management device 11 and monitor it. Here, various types of self-checkouts where customers perform registration operations and settlement operations are widespread in stores. For example, self-checkouts include semi-self-checkouts where a store clerk performs registration operations and a customer performs settlement operations, full self-checkouts where a customer performs registration operations and settlement operations, and self-checkouts using a mobile terminal where a customer performs registration operations using a mobile terminal and settlement operations at a settlement device. The transaction status management device 11 can monitor settlement at such self-checkouts. Note that in the case of a POS terminal device 20 where a store clerk is involved (performing registration operations), the transaction status management device 11 may not monitor the registration side. In this case, it is also possible to monitor only the settlement side, or not monitor the settlement side either.

The POS terminal device 20 is an example of a product sales data processing device. The POS terminal device 20 is a cash register device that executes a product registration process to register products and a settlement process to settle payments. In the product registration process, registration information is generated that indicates the products to be purchased by the customer (hereinafter sometimes referred to as "purchased products"). In the settlement process, settlement is performed based on the registration information generated in the product registration process, and the total price of the purchased products (hereinafter sometimes referred to as "purchase amount"). The POS terminal device 20 is a self-checkout that can perform product registration processing and settlement processing by customer operation. Note that although FIG. 1 shows multiple POS terminal devices 20, there may be only one.

The POS system 1 may also include a service counter terminal. The service counter terminal accepts requests for the delivery and return of products purchased by customers, ordering of products, and sale of gift certificates. For example, the service counter terminal receives and manages transaction information indicating the product and amount for each transaction at each POS terminal device 20 from each POS terminal device 20. The service counter terminal may also manage the transaction status of each POS terminal device 20.

The POS system 1 may also include a mobile terminal. The mobile terminal is, for example, a terminal device such as a mobile phone, smartphone, or tablet device carried by a store clerk. The mobile terminal may be lent to each store clerk, or may be owned by each store clerk. In response to a call request from the POS terminal device 20 (including the service counter terminal), the mobile terminal notifies the request or receives the call.

(Installation example of POS terminal device 20)
Fig. 2 is an explanatory diagram showing an example of the installation of the POS terminal 20. As shown in Fig. 2, a counter is placed on the right side of the POS terminal 20 as seen by the operator. Although not shown in the figure, a counter may also be placed on the left side of the POS terminal 20 in Fig. 2. In other words, counters may be placed on both sides of the POS terminal 20. A shopping basket can be placed on the counter.

The POS terminal 20 also includes a sign pole 220. The sign pole 220 is installed facing upward, and by emitting a predetermined lighting pattern, even store clerks who are not near the POS terminal 20 can recognize the lighting pattern. Specifically, the sign pole 220 includes a light-emitting unit such as a lamp or an LED (light emitting diode). The light-emitting unit can be lit in a predetermined lighting pattern. Depending on the lighting pattern of the sign pole 220, it can indicate that the POS terminal 20 is in use or that a store clerk has been called.

(Example of configuration of POS terminal device 20)
Next, the configuration of the POS terminal 20 will be described with reference to FIGS.
Fig. 3 is a block diagram showing the hardware configuration of the POS terminal device 20. In Fig. 2 and Fig. 3, the same parts are given the same reference numerals.

Below, an example of the configuration of the POS terminal 20 shown in FIG. 3 will be described with reference to FIG. 2. The POS terminal 20 comprises a CPU 201, a ROM 202, a RAM 203, a hard disk 204, an operation display unit 205, a scanner unit 206, a card payment unit 208, a change dispenser 209, a printing unit 213, an audio output unit 214, a communication unit 215, and a camera 216. These can communicate with each other via a bus.

The CPU 201 is a central processing unit that reads and executes various programs stored in the ROM 202 to control the operation of the POS terminal 20. The various programs include a product sales data processing program according to this embodiment.
The ROM 202 is a read-only memory, and stores various types of information used by the CPU 201, including various programs.

RAM 203 is a readable and writable memory that stores various information. For example, RAM 203 stores information acquired from the outside (such as a product master acquired from store controller 10) and information generated during processing. This information also includes settlement information generated during the settlement process for settling (paying) registered products (purchased products).

The hard disk 204 stores various information. For example, the hard disk 204 stores the image captured by the camera 216 (e.g., video). For example, the hard disk 204 may store various programs, such as a product sales data processing program executed by the CPU 201, instead of the ROM 202. Also, instead of the RAM 203, the hard disk 204 may store information acquired from the outside and information generated during processing.

The operation display unit 205 is a touch display (touch panel). The operation display unit 205 displays various information to the customer and accepts various operational inputs from the customer.

When the scanner unit 206 detects an item (e.g., a product) in a specified reading area (see FIG. 4), the reading function is turned on and various codes are optically read. When no more items are detected in the reading area, the reading function is turned off. The various codes read by the scanner unit 206 are, for example, barcodes (product codes, etc.) attached to products and codes (barcodes, two-dimensional codes, etc.) printed on receipts (registered trademark). The various codes also include gift certificates (gift certificates, coupons, complimentary coupons) and codes (barcodes, two-dimensional codes, etc.) printed on various cards (e.g., membership cards, point cards, etc.). The various codes also include store clerk codes attached to store clerk nametags.

Product registration includes registration by reading the barcode attached to the product, as well as registration using a preset key. A preset key is a button corresponding to the product (a button for ordering the product) and is displayed on the operation display unit 205. A product is registered by operating (pressing) the preset key.

The card payment unit 208 is a payment mechanism that uses various cards (credit cards, prepaid cards such as transportation IC cards, point cards, etc.). The card payment unit 208 is equipped with a card recognition unit (reading unit), a display unit, and an operation unit. The card recognition unit magnetically or electrically reads information from various cards. The read information is used to settle the purchase. The card recognition unit can also recognize information to be used in settling the purchase, for example, when using points held on a point card. The card recognition unit can also recognize information that is not used in settling the purchase, for example, when adding points to a point card. The card recognition unit can recognize contact cards (IC cards, magnetic cards) and non-contact cards (IC cards).

The change machine 209 (cash settlement unit) is a cash settlement mechanism, and has an input port for banknotes and coins, and an output port for banknotes and coins. The change machine 209 has a sensor that detects banknotes and coins inserted into the input port. The sensor detects that banknotes or coins have been inserted into the input port, and detects the number of banknotes by denomination. The change machine 209 calculates the amount inserted into the input port, calculates the change amount, which is the difference between the input amount and the purchase amount, and outputs the change from the output port. The change machine 209 faces the operator, and accepts various operations and cash insertions from customers.

The printing unit 213 issues various media (receipts, bills, etc.).

The audio output unit 214 outputs audio. For example, the audio output unit 214 outputs audio guidance and warning sounds. The audio guidance and warning sounds are related to product registration and payment.

The communication unit 215 is an interface that transmits and receives information with other devices (the store controller 10, the transaction status management device 11, and other POS terminal devices 20).

The camera 216 is attached to the upper right of the operation display unit 205. The camera 216 is a camera that continuously captures moving images or still images. A CCD (charge coupled device) camera or a CMOS (complementary metal oxide semiconductor) camera can be used as the camera 216. The camera 216 captures images of the product registration operation and checkout operation by the customer, as well as the products during the registration operation.

(Regarding the reading area 400 of the scanner unit 206)
Fig. 4 is an explanatory diagram showing an example of a reading area 400 of the scanner unit 206. In Fig. 4, the reading area 400 is an area in which an identifier of a product can be read. The identifier is, for example, a barcode, but may also be a two-dimensional code. The scanner unit 206 reads the barcode of a product present in the reading area 400. The barcode reading method is, for example, a laser method. However, the reading method is not limited to the laser method, and a CCD method or a pen method may also be used.

When a product is present within the reading area 400, the scanner unit 206 turns on its reading function. When the reading function is turned on, for example, a laser beam is directed at the label surface of the barcode and the reflected light is read by a sensor. On the other hand, when a product is no longer present within the reading area 400, the reading function is turned off. When the reading function is turned off, the laser beam is no longer emitted and the sensor no longer reads.

(Functional Configuration of POS Terminal Device 20)
In the POS system 1, for example, a store clerk is assigned to the transaction status management device 11, and the transaction status management device 11 monitors the transaction. The store clerk assigned to the transaction status management device 11 performs various tasks, such as providing operational support to customers, performing recovery work, and monitoring for fraud. This can increase the burden on the store clerk, and when operational support is required simultaneously on multiple POS terminal devices 20, one of the customers may have to wait. This can prevent transactions from being carried out smoothly.

The POS terminal device 20 according to this embodiment captures an image of the customer's product registration operations in the reading area 400 and controls the display of the captured image, thereby making it possible to encourage the customer to properly register products. The functional configuration of the POS terminal device 20 according to this embodiment will be described below.

The functional configuration of the POS terminal 20 will be described below. The POS terminal 20 comprises a product registration unit, an image capture information acquisition unit, and a display control unit. Each unit is realized by the CPU 201. That is, the CPU 201 executes a product sales data processing program stored in the ROM 202 to realize the functions of each unit. The POS terminal 20 also comprises a detection unit. The detection unit is realized by the scanner unit 206.

The detection unit detects an item (goods) present in the reading area 400. The detection unit is realized, for example, by the scanner unit 206, but may also be realized by the camera 216. That is, the camera 216 may detect an item present in the reading area 400. The detection unit may also be a dedicated device (for example, a proximity sensor) for detecting an item (goods) present in the reading area 400.

The product registration unit registers product information based on the barcode read by the scanner unit 206 in the reading area 400. Note that, below, reading by the scanner unit 206 may be referred to as scanning. The image information acquisition unit acquires image information capturing an image of the barcode reading operation (scanning operation) performed by the customer in the reading area 400.

The display control unit causes the display unit to display the product list image and the captured image. The product list image is an image based on product information registered by the product registration unit. Specifically, the product list image is an image in which products are listed in a list, and includes product names and prices. The product list image may be an image displayed in a display area showing the registered details, or may be an image that is generated separately as an image different from the registered details and displayed in a display area different from the display area showing the registered details. When products are registered by the product registration unit, the products indicated by the registered product information are added to the product list image.

The captured image is an image based on the imaging information acquired by the imaging information acquisition unit. Specifically, the captured image is a real-time image showing the scanning operation. The display means is, for example, the operation display unit 205. Furthermore, the captured image is a video, but may also be a still image.

The display control unit causes the product list image and the captured image to be displayed simultaneously on the operation display unit 205. Specifically, the display control unit causes the product list image to be displayed in one area, and the captured image to be displayed in another area. This makes it possible to appeal to customers that an image has been captured when the product is registered.

However, the display control unit does not have to display the product list image and the captured image simultaneously on the display unit. For example, the captured image may be displayed according to a setting. The captured image setting may be switched by an authorized operator (store clerk), or a switching button may be displayed on the product registration screen and the setting may be switched by an unauthorized operator (customer). In addition, when the captured image is not to be displayed, the display control unit may display a text image that reads "Recording."

The camera 216 is on and captures an image of the area including the reading area 400 even when the detection unit does not detect a product. The display control unit displays a real-time image captured by the camera 216 even when the detection unit does not detect a product. However, the display control unit may be configured to display a real-time image captured by the camera 216 when the detection unit detects a product. In this case, the camera 216 may be configured to be turned on when the detection unit detects a product.

(About superimposed display)
When the product registration unit does not register the product information even though the detection unit detects the product, that is, when the scanning operation fails, the display control unit performs a superimposed display in which the captured image is superimposed on the product list image. In the superimposed display, the captured image is displayed on top of the product list image, so that part or all of the product list image cannot be seen.

The display control unit also enlarges the captured image in the superimposed display. The enlarged display means displaying the captured image larger than before the superimposed display, and specifically means displaying the captured image using, for example, half or more of the display area of the operation display unit 205. When the captured image is enlarged, the product list image becomes more difficult to view.

The display control unit also redisplays, in a superimposed display, the captured image captured when the product information was not registered. In other words, the captured image displayed in the superimposed display is the recording when the scanning operation failed. As described above, the captured image displayed when not superimposed is a real-time image.

(How to cancel the overlay display)
When the product information is registered by performing a scan operation again after the superimposed display, the display control unit cancels the superimposed display. To be more specific, the scanner unit 206 can scan a barcode even when the superimposed display is performed. To be more specific, the scanner unit 206 can detect an item (product) present in the reading area 400 and can scan a barcode even when the superimposed display is performed. When the product registration unit registers the product information by performing a scan operation again, the display control unit cancels the superimposed display and displays the product list image and the captured image in the original display area.

If the scanning operation fails, the notification unit issues a notification. The notification is a notification indicating that the scanning operation has failed, or a notification encouraging the user to perform the scanning operation again. The notification may be in the form of a display using the operation display unit 205, or a sound using the sound output unit 214.

(One example of the display control process of the captured image performed by the POS terminal device 20)
Next, the captured image display control process performed by the POS terminal 20 in this embodiment will be described with reference to Fig. 5. Fig. 5 is a flow chart showing an example of the captured image display control process performed by the POS terminal 20. In Fig. 5, the POS terminal 20 judges whether product registration has started (step S501). The POS terminal 20 waits until product registration starts (step S501: NO). When product registration starts (step S501: YES), the POS terminal 20 displays a product list image and the captured image in separate areas (step S502).

The POS terminal 20 then determines whether the scanner unit 206 has detected a product (article) in the reading area 400 (step S503). The POS terminal 20 waits until it detects a product in the reading area 400 (step S503: NO). When it detects a product in the reading area 400 (step S503: YES), the POS terminal 20 starts scanning the barcode with the scanner unit 206 (step S504).

Then, the POS terminal 20 determines whether the barcode scan was successful (step S505). If the barcode scan was not successful (step S505: NO), the POS terminal 20 determines whether a product was not detected in the reading area 400 (step S506).

If the product in the reading area 400 is not undetected (step S506: NO), i.e., if the scanner unit 206 has detected a product in the reading area 400, the POS terminal 20 returns to step S505. If the product in the reading area 400 is undetected (step S506: YES), the POS terminal 20 ends the scan (step S507). The POS terminal 20 then acquires image information captured by the camera 216 when the scan operation fails (step S508). The image information captured when the scan operation fails is, specifically, image information captured from the start of scanning in step S504 to the end of scanning in step S506.

The POS terminal 20 then displays the captured image on the product list image (step S509). Next, the POS terminal 20 issues a notification to prompt the user to perform another scan (step S510) and returns to step S503.

If the barcode is successfully scanned in step S505 (step S505: YES), the POS terminal 20 ends the scan (step S511). The POS terminal 20 then registers the product identified from the barcode (step S512). The POS terminal 20 then determines whether the captured image is being superimposed on the product list image (step S513).

If the images are not being displayed in an overlapping manner (step S513: NO), the POS terminal 20 proceeds to step S515. If the images are being displayed in an overlapping manner (step S513: YES), the POS terminal 20 cancels the overlapping display (step S514), i.e., displays the product list image and the captured image in separate areas. The POS terminal 20 then determines whether the checkout start button has been pressed (step S515).

If the checkout start button is not pressed (step S515: NO), the POS terminal 20 returns to step S503. If the checkout start button is pressed (step S515: YES), the POS terminal 20 executes the checkout process (step S516) and ends the series of processes.

(Screen transition related to display control of captured image)
6 and 7 are explanatory diagrams showing an example of screen transition related to display control of captured images displayed on the operation display unit 205. Fig. 6 (A) shows a product registration screen 600 displayed on the operation display unit 205. The product registration screen 600 includes a product list image 610 and a captured image 620. The product list image 610 and the captured image 620 are displayed in different display areas. In addition, in the product registration screen 600, the display area of the product list image 610 is larger than that of the captured image 620. The product registration screen includes a preset key 630, a checkout start button 640, and a store clerk call button 650.

Product list image 610 shows a list of registered products. Product list image 610 shows the product name and price of each product. In the figure, two products (tea and daifuku mochi) are shown to have been registered. Captured image 620 is a real-time video captured by camera 216. In the figure, the captured image shows a scanning operation by a customer to register milk. When the barcode on the milk is successfully scanned and the registration of the milk is completed, the screen transitions to product registration screen 660 shown in FIG. 6 (B).

In the product registration screen 660 shown in FIG. 6(B), a product registration notification 670 indicates that milk has been registered. The product list image 610 also indicates that milk has been added, bringing the total number of registered products to three. In this way, the customer's scanning operation is constantly displayed on the captured image 620, and the registered products are displayed on the product list image 610.

Here, if scanning the barcode attached to the milk fails on the product registration screen 600 in FIG. 6(A), the screen transitions to the superimposed screen 700 shown in FIG. 7(A). The superimposed screen 700 shown in FIG. 7(A) shows a screen in which a captured image 620 is superimposed on a product list image 610. On the superimposed screen 700, the captured image 620 is an image (recorded image) when the scanning operation fails. Specifically, the captured image 620 shows the captured information when the scanning operation to register milk fails. The same image is repeatedly displayed on the captured image 620. Also, on the superimposed screen 700, the display area of the product list image 610 is almost hidden, and is a smaller display area than the captured image 620.

The superimposed screen 700 also displays a prompt notification 710 that prompts the user to perform another scan operation. If the user is successful in scanning the barcode on the milk by performing another scan operation, the superimposed screen 700 ends and the screen transitions to the product registration screen 660 shown in FIG. 7(B). The product registration screen 660 shown in FIG. 7(B) is the same screen as the product registration screen 660 shown in FIG. 6(B). Even if the user performs another scan operation to scan the barcode of a product other than milk, the superimposed screen 700 ends and the screen transitions to the product registration screen 660 shown in FIG. 7(B).

The superimposed screen is also terminated when the end button 720 is pressed on the superimposed screen 700 of FIG. 7(A). In this case, the screen returns to the product registration screen 600 of FIG. 6(A). The end button 720 may not be displayed on the superimposed screen 700. In other words, the superimposed screen 700 may not be terminated in response to a customer operation. In other words, the condition for canceling the superimposed screen 700 may be that product information is registered by a second scan operation in response to a customer operation.

As described above, the POS terminal device 20 according to this embodiment displays the captured image 620 on the product list image 610 in a superimposed manner (superimposed screen 700) when a product is detected in the reading area 400 but the product information is not registered. This allows the customer to focus on the captured image 620 when the scanning operation fails. Therefore, when the barcode scan fails, the customer can be prompted to rescan. In addition, for customers who intentionally commit fraud by causing the barcode scan to fail, the fact that they are being monitored can be displayed, which can have a deterrent effect on fraud. This reduces the burden on the store clerk at the transaction status management device 11 in providing operational support to customers and monitoring for fraud. Therefore, at the self-checkout, the store clerk can efficiently provide support to customers, allowing transactions to be carried out smoothly.

The POS terminal device 20 according to this embodiment also enlarges the captured image 620 in the superimposed display. This allows the customer to pay more attention to the captured image 620.

In addition, the POS terminal device 20 according to this embodiment is configured to cancel the superimposed display if the customer performs a scan operation to register product information after the superimposed display. This allows the user to return to the original screen and register the product again if the barcode scan is successful.

The POS terminal device 20 according to this embodiment is also configured to redisplay the captured image that was displayed when product information was not registered in a superimposed display. This allows a recording of the scanning operation when a barcode scan fails to be displayed, making it possible to appeal to customers that the operation is being recorded. This can therefore improve the deterrent effect against fraud.

(Modification of the embodiment)
Next, modified examples of the embodiment will be described. In each of the following modified examples, the contents described in the above-mentioned embodiment will be omitted as appropriate. In addition, each of the following modified examples and the above-mentioned embodiment can be combined with each other.

(Variation 1)
First, a first modified example of the embodiment will be described. In the above-described embodiment, a configuration has been described in which the condition for canceling the superimposed display is that product information is registered by performing a scan operation again. In the first modified example, in addition to or instead of such a configuration, a configuration will be described in which the condition for canceling the superimposed display is that the product registered by performing a scan operation again matches the product captured when the scan operation failed.

In the first modified example, the imaging information acquisition unit acquires first imaging information and second imaging information. The first imaging information is imaging information captured when product information is not registered, i.e., imaging information captured when the scanning operation fails. The second imaging information is imaging information captured during a second scanning operation, i.e., imaging information captured when the scanning operation is successful after the second scanning operation.

The display control unit cancels the superimposed display according to the degree of similarity based on the first imaging information and the second imaging information. The degree of similarity is obtained, for example, by performing image analysis on the image of the first imaging information and the image of the second imaging information. Here, the POS terminal device 20 according to the first modified example includes a match determination unit. The match determination unit extracts images showing products from the first imaging information and the second imaging information, respectively. Furthermore, the match determination unit extracts feature amounts from each of the extracted images, compares the extracted feature amounts, and calculates the similarity between the two. The feature amounts include shape, pattern, and color.

When the calculated similarity is equal to or greater than a predetermined threshold, the match determination unit determines that the products match. On the other hand, when the calculated similarity is less than the predetermined threshold, the match determination unit determines that the products do not match (mismatch).

The display control unit cancels the superimposed display when the match determination unit determines that there is a match, and continues the superimposed display when the match determination unit determines that there is a mismatch. Furthermore, when the match determination results in a mismatch, the notification unit notifies the user of this fact.

Furthermore, if the match determination indicates a match, the product registration unit registers the product information obtained by rescanning as is. On the other hand, if the match determination indicates a mismatch, the product registration unit receives an instruction from the customer as to whether or not to register the product information obtained by rescanning, and registers the product information according to the received result.

Note that the product match determination does not need to be performed precisely, and it is sufficient to be able to determine whether or not there is a rough match. For example, even if the feature is color only, it is possible to determine whether or not there is a rough match. Specifically, for example, it is possible to perform a match determination by comparing the color most frequently obtained from the product image when the scanning operation has failed with the color most frequently obtained from the product image when the scanning operation has been successful. In this way, by performing a match determination using color only as a feature, the match determination can be simplified.

(Example of display control process of captured images performed by the POS terminal device 20 according to the first modification)
Next, the captured image display control process performed by the POS terminal 20 according to the first modification will be described with reference to Fig. 8. Fig. 8 is a flow chart showing an example of the captured image display control process performed by the POS terminal 20 according to the first modification.

In FIG. 8, the POS terminal 20 determines whether product registration has started (step S801). The POS terminal 20 waits until product registration starts (step S801: NO). When product registration starts (step S801: YES), the POS terminal 20 displays the product list image and the captured image in separate areas (step S802).

The POS terminal 20 then determines whether the scanner unit 206 has detected a product (article) in the reading area 400 (step S803). The POS terminal 20 waits until it detects a product in the reading area 400 (step S803: NO). If it detects a product in the reading area 400 (step S803: YES), the POS terminal 20 starts scanning the barcode with the scanner unit 206 (step S804).

Then, the POS terminal 20 determines whether the barcode scan was successful (step S805). If the barcode scan was not successful (step S805: NO), the POS terminal 20 determines whether a product was not detected in the reading area 400 (step S806).

If the product in the reading area 400 is not undetected (step S806: NO), i.e., if the scanner unit 206 has detected a product in the reading area 400, the POS terminal 20 returns to step S805. If the product in the reading area 400 is undetected (step S806: YES), the POS terminal 20 ends the scan (step S807). The POS terminal 20 then acquires image information (first image information) captured by the camera 216 when the scan operation failed (step S808). The first image information is, specifically, image information captured from the start of scanning in step S804 to the end of scanning in step S806.

The POS terminal 20 then displays the captured image on the product list image (step S809). Next, the POS terminal 20 issues a notification to prompt the user to perform another scan (step S810) and returns to step S803.

If the barcode scan is successful in step S805 (step S805: YES), the POS terminal 20 ends the scan (step S811). The POS terminal 20 then determines whether the captured image is being superimposed on the product list image (step S812).

If the superimposed display is not in progress (step S812: NO), the POS terminal 20 proceeds to step S819. If the superimposed display is in progress (step S812: YES), the POS terminal 20 acquires image information (second image information) captured by the camera 216 when the scan operation is successful (step S813). Specifically, the second image information is image information captured from the start of scanning in step S804, through step S805: YES, to the end of scanning in step S811.

The POS terminal 20 then determines the degree of similarity based on the second image information acquired in step S813 and the first image information acquired in step S808 (step S814). The POS terminal 20 then determines whether the products match using the images of each product extracted from the first image information and the second image information (step S815).

If the products do not match (step S815: NO), the POS terminal 20 displays a confirmation screen and registers the product according to the customer's instructions (step S816). Note that the POS terminal 20 does not register the product depending on the customer's instructions. If the products match (step S815: YES), the POS terminal 20 registers the product identified by the barcode (step S817).

The POS terminal 20 then cancels the superimposed display (step S818), i.e., displays the product list image and the captured image in separate areas. The POS terminal 20 then determines whether the checkout start button has been pressed (step S819). If the checkout start button has not been pressed (step S819: NO), the POS terminal 20 returns to step S803. If the checkout start button has been pressed (step S819: YES), the POS terminal 20 executes the checkout process (step S820) and ends the series of processes.

(Screen Transition According to Modification 1)
Next, Fig. 9 is an explanatory diagram showing an example of a screen transition according to the first modified example displayed on the operation display unit 205. Fig. 9(A) shows a screen similar to the superimposed screen 700 shown in Fig. 7(A). The superimposed screen 700 in Fig. 9(A) shows imaging information when a scan operation for the product "milk" has failed. If a second scan operation succeeds in scanning the barcode attached to the product "coffee", the screen transitions to a confirmation screen 900 shown in Fig. 9(B).

The confirmation screen 900 displays a notification display 910. The notification display 910 indicates that the product "milk" when the scanning operation fails is different from the product "coffee" when the scanning operation is successful. The notification display 910 also includes a "Yes" button to accept that the product is to be registered, and a "No" button to accept that the product is not to be registered. When the "Yes" button is pressed, the product registration of coffee is completed, and the screen returns to the product registration screen. When the "No" button is pressed, the product registration of coffee is not performed, and the screen returns to the product registration screen. Note that, if the barcode attached to the product "milk" is successfully scanned by performing a second scan operation on the superimposed screen 700 in FIG. 9(A), the screen transitions to the product registration screen 660 shown in FIG. 7(B).

According to the first variant example, the superimposed display (superimposed screen 700) is cancelled according to the degree of similarity based on the image information when the scan operation fails and the image information when the second scan operation is successful. This makes it possible to prevent the superimposed display from being cancelled when the product scanned immediately before and the product scanned again are not the same, i.e., when a different product is scanned after the first scan operation fails. This makes it possible to prevent a transaction from being completed without registering a product that has failed to be scanned. This can therefore further improve the deterrent effect against fraudulent activities.

(Another aspect of Modification 1)
Other aspects of the first modification are described below. In the first modification, regardless of the result of the match determination, the customer may be asked whether or not the two products match, and the products may be registered according to the result of the acceptance. For example, the POS terminal device 20 may notify the customer "Is this the same product as before?" regardless of the result of the match determination, and accept "Yes" or "No." The POS terminal device 20 registers the product information if it accepts "Yes," and does not register the product information if it accepts "No." The POS terminal device 20 may also cancel the superimposed screen 700 after accepting "Yes" or "No."

The product registration unit may also register the product information obtained by rescanning as is, regardless of the result of the match determination.

In addition, if the match determination indicates a mismatch, the superimposed screen 700 may not be released. The release may require confirmation from a store clerk. Specifically, confirmation from the store clerk means a release operation by the store clerk. For example, if the match determination indicates a mismatch, a store clerk may be called, and the customer may not be able to perform any operations until the store clerk's release operation is accepted. Note that if the customer is not able to perform any operations, the superimposed screen 700 may continue to display an image of the failed scan operation, and may display a message indicating that a store clerk is being called.

The store clerk may be called by lighting up the sign pole 220 or by sounding an alarm. The transaction status management device 11 may also notify the call. Furthermore, the store clerk may be able to cancel the call from the transaction status management device 11. In other words, the store clerk may be able to perform the cancel operation remotely.

(Variation 2)
Next, a second modification of the embodiment will be described. In the above-described embodiment, a case where a product with a barcode is registered has been described. In the second modification, in addition to or instead of the above configuration, a case where a product without a barcode is registered will be described. Fig. 10 is an explanatory diagram showing an example of a screen transition related to the second modification displayed on the operation display unit 205.

Products without barcodes include, for example, vegetables, fruits, prepared foods, and fresh fish, and are registered using the preset key 630 (see FIG. 6A). Now, suppose that an operator (e.g., a customer) picks up a product without a barcode and places it in the reading area 400. In this case, the scanner unit 206 detects the presence of the product (e.g., a carrot). Because the product does not have a barcode, when the product leaves the reading area 400, the POS terminal device 20 determines that the scan operation has failed. For this reason, as shown in FIG. 10A, the captured image 620 shown on the superimposed screen 1000 displays an image of the failed scan operation (a recording of a carrot).

In the second modification, as shown in FIG. 10(A), preset keys 1010 are displayed on the superimposed screen 1000. The preset keys 1010 include buttons corresponding to, for example, vegetables, fruits, side dishes, and fresh fish. For example, when the preset key 1010 indicating vegetables is pressed, the screen transitions to the screen shown in FIG. 10(B).

Figure 10 (B) shows a product selection screen 1020 corresponding to the vegetable preset key 1010. The product selection screen 1020 includes multiple product selection buttons. When the button indicating "carrot" is selected from among the product selection buttons, product registration is completed, the superimposed screen 1000 is released, and the screen returns to the product registration screen.

According to the second modification, even if it is determined that the scanning operation of a product without a barcode has failed, the product can be easily registered using the preset key 1010. In addition, since the captured image 620 displays an image of the failed scanning operation, it is possible to obtain a deterrent effect against fraudulent acts against products without a barcode.

(Variation 3)
Next, a third modification of the embodiment will be described. In the above-described embodiment, the captured image (recorded image) when the scanning operation fails is used as the captured image displayed in the superimposed display. In the third modification, in addition to or instead of such a configuration, a configuration will be described in which a real-time captured image is used as the captured image displayed in the superimposed display.

In the third modification, a real-time image is simply displayed on the superimposed screen 700. Even in this way, the captured image 620 is displayed superimposed on the product list image 610, which can provide a fraud prevention effect. Furthermore, even if the captured image 620 is a real-time image on the superimposed screen 700, it can be enlarged to further emphasize that surveillance is taking place. Furthermore, in the third modification, since recording is not required, there is no need to provide a storage device or the like. This makes it possible to carry out smooth transactions with a simple configuration.

(Variation 4)
Next, a fourth modified example of the embodiment will be described. In the above-described embodiment, only the captured image (recorded image) when the scanning operation fails is used as the captured image displayed in the superimposed display. In the fourth modified example, a configuration will be described in which the recorded image and a real-time captured image are used as the captured image displayed in the superimposed display.

In the fourth variation, a real-time image (an image when a second scan operation is performed) and an image (recorded image) captured when the scan operation fails are displayed side by side on the superimposed screen 700. These images are superimposed on the product list image.

(Example of a screen according to Modification 4)
Fig. 11 is an explanatory diagram showing an example of a screen related to the modified example 4 displayed on the operation display unit 205. In Fig. 11, an overlapping parallel screen 1100 shows a screen in which a captured image 620 is displayed overlapping a product list image 610. The captured image 620 includes a first captured image 620a and a second captured image 620b. The first captured image 620a is an image (recorded) when a scanning operation fails, that is, an image at the time of the first scanning operation. The second captured image 620b is an image of a second scanning operation (a real-time scanning operation), that is, an image at the time of the second scanning operation.

The condition for canceling the superimposed parallel screen 1100 is that product information is registered by performing a scan operation again. In other words, it is possible to cancel the superimposed parallel screen 1100 even if the same product is not scanned. Also, an end button 720 (see FIG. 7(A)) may be displayed on the superimposed parallel screen 1100 so that the superimposed parallel screen 1100 can be canceled in response to pressing the button.

As described above, in the fourth modification, the first captured image 620a and the second captured image 620b are displayed side by side. This allows the customer to easily recognize that the products targeted by the first and second scanning operations are different. This makes it possible to encourage the customer to rescan the same product.

A screen similar to the superimposed parallel screen 1100 is displayed not only on the POS terminal device 20 but also on the transaction status management device 11. This allows the store clerk at the transaction status management device 11 to easily understand that the products targeted by the first scanning operation and the second scanning operation are different.

Here, the store clerk performs various tasks (for example, spreading out and stacking bags to make bundles of bags for easy use by customers, being called when customers purchase shopping bags, moving around multiple customers to assist them in operating the device, etc.). For this reason, the clerk is often unable to stare at the transaction status management device 11.

According to variant example 4, by displaying the superimposed parallel screen 1100 on the transaction status management device 11, it is possible to know from the form of the product (size, color, shape, etc.) whether the same item will be scanned or a different item will be scanned in the next scan operation. In addition, because the first captured image 620a and the second captured image 620b are displayed side by side, even if there is no store clerk near the transaction status management device 11 and the store clerk is looking at the operation display unit 205 from a distance, it is easy to know that a second scan operation is being performed. Therefore, the store clerk can take prompt action to prevent fraud.

(Variation 5)
Next, a fifth modified example of the embodiment will be described. In the fourth modified example described above, a configuration is described in which a match determination is not performed between a captured image (recorded image) displayed in a superimposed display when a scanning operation has failed and a real-time captured image. In the fifth modified example, a configuration is described in which a match determination is performed between a captured image (recorded image) displayed in a superimposed display when a scanning operation has failed and a real-time captured image.

In the fifth variation, the POS terminal device 20 includes a match determination unit. As described above, the match determination unit determines whether the products targeted by each scan operation match based on the image information captured when the scan operation fails and the image information captured in a second scan operation.

(Example of a screen according to Modification 5)
Fig. 12 is an explanatory diagram showing an example of a screen related to Modification 5 displayed on the operation display unit 205. In Fig. 12, a superimposed parallel screen 1200 shows a screen in which the captured images 620 (the first captured image 620a and the second captured image 620b) are superimposed on a product list image 610. The superimposed parallel screen 1200 also includes a discrepancy notification 1210. The discrepancy notification 1210 is displayed when a result of a mismatch is obtained in the match determination by the match determination unit.

The discrepancy notification 1210 is a notification that indicates that the match determination by the match determination unit has resulted in a mismatch and that a store clerk will be called. The background image of the discrepancy notification 1210 is displayed in a different color from the normal background image (for example, see the superimposed parallel screen 1100 in FIG. 11). This makes it possible to prompt the customer to rescan the same product and register the product.

On the other hand, if the match determination unit obtains a result of a match, the superimposed parallel screen 1200 is released. In other words, the condition for releasing the superimposed parallel screen 1200 is that the product registered by a second scan operation matches the product captured when the scan operation failed.

According to variant example 5, if a different product is scanned after a failed scan operation, the superimposed display can be prevented from being released. This can prevent a transaction from being completed without registering a product for which a scan operation has failed. This can further improve the deterrent effect against fraudulent activities.

The superimposed parallel screen 1200 is also displayed on the transaction status management device 11. This allows the store clerk at the transaction status management device 11 to easily understand that the products targeted by the first and second scan operations are different. In particular, the first captured image 620a and the second captured image 620b are displayed side by side, and further, the background image of the discrepancy notification 1210 is displayed in a different display mode from the normal background image, so that even if there is no store clerk near the transaction status management device 11 and the store clerk is looking at the operation display unit 205 from a distance, the store clerk can easily understand that a second scan operation is being performed. Therefore, the store clerk can quickly take action to prevent fraud.

The embodiments will be summarized below.
[Title of the Invention] Product sales data processing device and program [Technical field] The present invention relates to a product sales data processing device and program.
[Background technology]
Conventionally, in various stores such as supermarkets and convenience stores, transactions are carried out using POS (Point Of Sales) terminals. The POS terminals perform product registration and payment processes. In addition to POS terminals in which a store clerk registers products and performs payment, there are also so-called self-checkouts that are operated by customers themselves to register products and perform payment (see, for example, Patent Document 1). Self-checkouts are provided with a monitoring terminal that displays the operating status of each self-checkout, and a store clerk is stationed at the monitoring terminal to monitor the self-checkout.
[Prior art documents]
[Patent Documents]
[Patent Document 1] JP-A-10-069574 [Summary of the Invention]
[Problem to be solved by the invention]
However, in the conventional technology, the salesperson at the monitoring terminal must perform various tasks, such as assisting customers with operations, performing recovery work, and monitoring for fraud, which can increase the salesperson's burden. Also, when operation assistance is required at multiple self-checkouts at the same time, one of the customers must wait. For this reason, the conventional technology has the problem that transactions cannot be carried out smoothly.
The present invention has been made in consideration of the above circumstances, and has an object to provide a technique that enables transactions to be carried out smoothly at self-checkout registers.

[Means for solving the problems]
(1) In order to solve the above-mentioned problems, a product sales data processing device that is one aspect of the present invention comprises a detection means for detecting a product present in an identifier reading area, a product registration means for registering product information based on the identifier read in the reading area, an image information acquisition means for acquiring image information capturing an image of a customer reading the identifier in the reading area, and a display control means for causing a display means to display a product list image based on the product information registered by the product registration means and an image based on the image information acquired by the image information acquisition means, wherein when a product is detected by the detection means, if product information is registered by the product registration means, the display control means displays the product list image and the image in separate display areas, and when a product is detected by the detection means but product information is not registered by the product registration means, the display control means performs a superimposed display of the image on the product list image.
According to the above configuration, when a scanning operation fails, the customer can be directed to the captured image 620. Therefore, when a barcode scan fails, the customer can be prompted to rescan. In addition, for customers who intentionally commit fraud by failing to scan a barcode, it is possible to convey that the customer is being monitored, which can have a deterrent effect on fraud. This reduces the burden on the store clerk assigned to the transaction status management device 11 in providing operational support to customers and monitoring for fraud. Therefore, at the self-checkout, the store clerk can efficiently provide support to customers, allowing transactions to be carried out smoothly.

(2) In the above configuration (1), the display control means may be configured to display the captured image in the superimposed display at a larger size than the captured image when the product list image and the captured image are displayed in separate display areas.
According to the above configuration, it is possible to draw the customer's attention to the captured image 620 more.

(3) In the configuration of (1) or (2) above, the display control means may be configured to cancel the superimposed display when product information is registered by performing a reading operation after the superimposed display is performed.
According to the above configuration, if the barcode is successfully scanned, the user can return to the original screen and register the product again.

(4) In any of the configurations (1) to (3) above, the image information acquisition means may acquire first image information captured when product information is not registered and second image information captured during a reading operation after the superimposed display is performed, and the display control means may cancel the superimposed display depending on a similarity based on the first image information and the second image information.
According to the above configuration, when the product scanned just before and the product scanned again are not the same, i.e., when a different product is scanned after a failed scan, the superimposed display can be prevented from being released. This can prevent a transaction from being completed without registering the product that was unsuccessfully scanned. This can further improve the deterrent effect against fraudulent activities.

(5) In any of the configurations (1) to (4) above, the display control means may be configured to display, in the superimposed display, a first image information captured when no product information was registered and a second image information captured during a reading operation after the superimposed display are performed, side by side.
According to the above configuration, if the products targeted by the first and second scanning operations are different, the customer can easily recognize that the products are different. This makes it possible to encourage the customer to rescan the same product. In addition, since the first and second captured image information are displayed side by side, even if there is no store clerk near the transaction status management device 11 and the store clerk is looking at the operation display unit 205 from a distance, the store clerk can easily understand that a second scan operation is being performed. Therefore, the store clerk can quickly take action to prevent fraud.

In addition, (6) is added. (6) In any of the configurations (1) to (5) above, the display control means may cause the captured image, which is displayed when the product information is not registered, to be displayed again in the superimposed display.
According to the above configuration, a recording of the scanning operation when a barcode scan fails is displayed, so that it is possible to appeal to customers that the recording is being made, thereby improving the effect of deterring fraud.

(7) In order to solve the above-mentioned problems, another aspect of the present invention is a program that causes a computer used in a product sales data processing device to function as a detection means for detecting products present in an identifier reading area, a product registration means for registering product information based on the identifier read in the reading area, an image information acquisition means for acquiring image information capturing an image of a customer reading the identifier in the reading area, and a display control means for causing a display means to display a product list image based on the product information registered by the product registration means and an image based on the image information acquired by the image information acquisition means, wherein when a product is detected by the detection means, if product information is registered by the product registration means, the display control means displays the product list image and the image in separate display areas, and when a product is detected by the detection means but product information is not registered by the product registration means, the display control means performs a superimposed display of the image on the product list image.
According to the above configuration, when a scanning operation fails, the customer can be directed to the captured image 620. Therefore, when a barcode scan fails, the customer can be prompted to rescan. In addition, for customers who intentionally commit fraud by failing to scan a barcode, it is possible to convey that the customer is being monitored, which can have a deterrent effect in preventing fraud. This can reduce the burden on the store clerk assigned to the transaction status management device 11 in providing operational support to customers and monitoring for fraud. Therefore, at the self-checkout, the store clerk can efficiently provide support to customers, allowing transactions to be carried out smoothly.

Note that all or part of each function (input/output, storage, processing (including judgment)) of the POS terminal device 20 described above may be realized in a device other than the device described as the entity that executes the function.

Specifically, in the above description, the POS terminal device 20 is configured to include a product registration unit, an image capture information acquisition unit, and a display control unit. All or part of these functional units may be included in another computer device. For example, all or part of these functional units may be included in the store controller 10, the transaction status management device 11, or other computer devices. Furthermore, the number of computer devices that include these functional units is not limited to multiple, and may be one. For example, one computer device may include all of these functional units.

Specifically, for example, instead of the POS terminal device 20, the store controller 10 or the transaction status management device 11 may perform a display control process that superimposes the captured image on the product list image.

In relation to the above, the POS terminal device 20 may function as a so-called thin client specialized for the input/output interface when it comes to registering products and settling accounts. In other words, the POS terminal device 20 may accept various inputs (operation by an operator, detection by a device such as a scanner), send input information (operation information, scan information, etc.) to a cloud server, receive the results of processing by the cloud server based on the input information (update screen information, device control information, etc.), and perform various outputs (display on a display unit, control of a device).

The program for implementing the POS terminal device 20 described above may be recorded on a computer-readable recording medium, and the program may be read into a computer system for execution. The term "computer system" as used herein includes hardware such as an OS and peripheral devices. The term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into a computer system. The term "computer-readable recording medium" also includes storage devices that hold a program for a certain period of time, such as volatile memory (RAM) inside a computer system that becomes a server or client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. The program may be transmitted from a computer system that stores the program in a storage device to another computer system via a transmission medium or by a transmission wave in the transmission medium. The "transmission medium" that transmits the program refers to a medium that has the function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may also be for implementing part of the above-mentioned functions. Furthermore, the above-mentioned functions may be realized in combination with a program already recorded in the computer system, i.e., a differential file (differential program).

1... POS system, 10... store controller, 11... transaction status management device, 20... POS terminal device, 201... CPU, 202... ROM, 203... RAM, 204... hard disk, 205... operation display unit, 206... scanner unit, 208... card payment unit, 209... change machine, 215... communication unit, 216... camera

Claims (5)

A detection means for detecting a commodity present in a reading area of the identifier;
a product registration means for registering product information based on the identifier read in the reading area;
an image information acquiring means for acquiring image information obtained by capturing an image of a customer reading an identifier in the reading area;
a display control means for controlling a display means to display a product list image based on the product information registered by the product registration means and an image based on the imaging information acquired by the imaging information acquisition means;
Equipped with
The display control means
when the product information is not registered by the product registration means even if the product is detected by the detection means, the captured image is superimposed on the product list image to display the product.
A product sales data processing device comprising: the display control means cancels the superimposed display when a reading operation is performed after the superimposed display and product information is registered.
2. The product sales data processing device according to claim 1 . The image information acquiring means acquires first image information captured when the commodity information is not registered and second image information captured during a reading operation after the superimposed display is performed,
the display control means determines that the products match when a similarity based on the first imaging information and the second imaging information is equal to or greater than a predetermined threshold, and cancels the superimposed display.
3. The product sales data processing device according to claim 1 , wherein the product sales data processing device is a data processing device for processing a product sales data. the display control means displays, in the superimposed display, a first image information captured when the product information is not registered and a second image information captured in a reading operation after the superimposed display is performed, side by side.
The product sales data processing device according to any one of claims 1 to 3 . A computer used in a product sales data processing device,
A detection means for detecting a commodity present in a reading area of the identifier;
a product registration means for registering product information based on the identifier read in the reading area;
an image information acquiring means for acquiring image information obtained by capturing an image of a customer reading an identifier in the reading area;
a display control means for causing a display means to display a product list image based on the product information registered by the product registration means and an image based on the imaging information acquired by the imaging information acquisition means;
Function as a
The display control means
when the product information is not registered by the product registration means even if the product is detected by the detection means, the captured image is superimposed on the product list image to display the product.
A program characterized by:

Images