JP7745940B2 - Product sales data processing system and mobile terminal device - Google Patents

Description

The present invention relates to a product sales data processing system and a mobile terminal device.

A system is known in which customers can register products using a mobile terminal device (see, for example, Patent Document 1).

Japanese Patent Application Laid-Open No. 2019-204551

In the above-mentioned product sales data processing system, a customer may register a certain product and reserve it by adding it to their shopping cart, but may wish to cancel the registration, for example, because they have selected the wrong product or have decided not to purchase it. In such cases, the customer can cancel the registration of the product using their mobile device and return the deregistered product to its original display location, such as a display shelf. However, there are cases where customers do not properly follow the procedure for canceling such product registration. If the procedure for canceling product registration is not followed properly, the product registration results will not match the breakdown of the product reserved by the customer, hindering smooth checkout.

The present invention was made in light of these circumstances, and aims to enable customers to properly carry out procedures for canceling product registration in a product sales data processing system in which customers register products using a terminal device.

One aspect of the present invention that solves the above-mentioned problems is a product sales data processing system in which customers register products using a mobile terminal device, and which includes a detection means for detecting the removal of a product from a product insertion section into which products secured by the customer are inserted, and a notification means for notifying the customer of the cancellation of the registration of a registered product based on the detection results of the detection means.

Another aspect of the present invention is a mobile terminal device used by customers to register products, which includes a detection means for detecting the removal of a product from a product insertion section into which products secured by the customer are inserted, and a notification means for notifying the customer of information regarding a registration cancellation operation for a registered product based on the detection results of the detection means.

1 is a diagram illustrating an example of the configuration of a shopping system according to an embodiment of the present invention. FIG. 2 is a diagram illustrating an example of installation of a settlement device according to the present embodiment. 1 is a diagram showing an example of the appearance of a settlement device according to an embodiment of the present invention; FIG. 2 is a diagram illustrating an example of the configuration of a settlement device according to the present embodiment. FIG. 2 is a diagram illustrating an example of the configuration of a customer terminal device according to the present embodiment. 1 is a diagram illustrating an example of the configuration of a transaction management device according to an embodiment of the present invention. 1 is a diagram illustrating an example of the configuration of a transaction status management device according to an embodiment of the present invention. 3A to 3C are diagrams illustrating examples of customer information, store information, and cart information according to the present embodiment. FIG. 10 is a diagram illustrating an example of a definition of a reserved product in the present embodiment. A sequence diagram showing an example of processing procedures executed by the customer terminal device, settlement device, and transaction management device in this embodiment in response to the accounting of one transaction in self-service operation. FIG. 10 is a diagram showing an example of a screen of a shopping application in the present embodiment. 10A and 10B are diagrams showing an example of a registered product presentation screen and a screen displayed on the settlement device in this embodiment. 10A and 10B are diagrams illustrating an example of a product registration change screen and a cancellation confirmation screen according to the present embodiment. 10A and 10B are diagrams illustrating a specific example of a situation in which a non-allocated re-guidance screen is displayed in the present embodiment. 10A and 10B are diagrams illustrating a specific example of a situation in which a registration cancellation operation guide screen is displayed in the present embodiment. 10 is a flowchart illustrating an example of a processing procedure executed by the customer terminal device and the transaction management device in this embodiment in relation to canceling the registration of a product.

<Embodiment>
[Example of shopping system configuration]
1 shows an example of the configuration of a shopping system (an example of a product sales data processing system) according to this embodiment. The shopping system shown in the figure supports a type of transaction (customer terminal-based transaction) in which, for example, a customer registers the product they are purchasing while removing it from a display shelf using a customer terminal device 50, and then the same customer also uses a payment device 40 to pay for the registered product. The transaction here refers to the procedure performed for one transaction, including product registration and payment for the registered product.
The shopping system of this embodiment includes a store controller 10, a transaction management device 60, a settlement device 40, a customer terminal device 50, a transaction status management device 80, and a credit card payment server 70.

The store controller 10, settlement device 40, and transaction status management device 80 are installed in the store and are communicatively connected via a LAN 19 (which may be wired or wireless). The store controller 10 is a business terminal used by store staff for store management, etc., and is capable of communicating with the transaction management device 60.

The transaction status management device 80 is a business terminal that manages (including monitoring) the status of transactions conducted by customers using the customer terminal device 50 in the store, and is capable of communicating with the transaction management device 60.
Although four payment terminals 40 are shown in FIG. 1, the number of payment terminals 40 within one store is not particularly limited.

The customer terminal device 50 is operated by a customer (such as a shopper who is a member of the store; hereinafter simply referred to as a customer). In addition to typical communication and imaging functions (camera), the customer terminal device 50 has a recognition function that scans barcodes attached to products to read product codes, i.e., recognizes barcodes attached to products. Note that the recognition function of the customer terminal device 50 only needs to be able to read product codes, and does not need to be able to recognize which product the read product code belongs to. In other words, the customer terminal device 50 only needs to be able to read the product code from the barcode when it is present as an object in an image captured by the imaging function (an image acquired as a through image).

Such a customer terminal device 50 may be a smartphone, tablet terminal, or the like owned by the customer. A shopping application having a function for registering products in accordance with the shopping system of this embodiment is installed on the smartphone, tablet terminal, or the like serving as the customer terminal device 50.

Furthermore, the customer terminal device 50 may be, for example, a terminal loaned to the customer by the store. An application having a function for registering products in accordance with the shopping system of this embodiment is also installed on the customer terminal device 50 loaned to the customer by the store. The customer terminal device 50 loaned to the customer by the store may be one that the customer holds in their hand, or may be one that the customer uses while attached to, for example, a shopping cart.

The customer terminal device 50 also captures an image (e.g., by releasing the shutter) of the product (e.g., the surrounding area with the barcode) and generates the captured image (image data). The customer terminal device 50 may release the shutter in response to an operation by the customer who is the operator, but in this embodiment, it is preferable that the customer terminal device 50 releases the shutter at its own discretion.

The customer terminal device 50 may also be equipped with image recognition technology that extracts feature points from images (through-the-lens images, captured images) and recognizes the imaged subject (object, etc.). For example, the customer terminal device 50 may use image recognition technology to identify (estimate) the captured product.

The settlement device 40 is a settlement device that can accept at least cash payments as a settlement method. The settlement device 40 is configured to be able to communicate with the transaction management device 60. Note that the settlement device 40 may also be able to communicate with the transaction management device 60, for example, via a relay via the store controller 10.

The transaction management device 60 manages transactions carried out in accordance with the shopping system of this embodiment.

The credit card payment server 70 executes payment processing corresponding to the credit card in response to a credit card payment request sent from the settlement device 40, customer terminal device 50, etc.

[About the payment device]
Figure 2 shows an example of the installation of the payment device 40. Figure 2(A) is a perspective view of the payment device 40 and other components as seen from the customer side. Figure 2(B) is a perspective view of the payment device 40 and other components as seen from the store clerk side. As shown in Figure 2(A), a counter is placed on the right side of the payment device 40 as seen from the customer side.

3A and 3B are diagrams showing an example of the appearance of the settlement device 40. Fig. 3A is a perspective view of the settlement device 40 as seen from the customer side. Fig. 3B is a perspective view of the settlement device 40 as seen from the store clerk side.
Fig. 4 is a diagram showing an example of the configuration of the settlement device 40. In Fig. 3 and Fig. 4, the same parts are denoted by the same reference numerals.

An example configuration of the settlement device 40 shown in Figure 4 will be described below with reference to Figure 3. The settlement device 40 comprises a CPU 401, ROM 402, RAM 403, a hard disk 404, a customer display unit 405, a customer scanner unit 406, a card payment unit 408, a change dispenser 409, a store clerk display unit 410, a key operation unit 411, a store clerk scanner unit 412, a printing unit 413, an audio output unit 414, and a communication unit 415. These can communicate with each other via a bus.

The CPU 401 is a central processing unit that controls the operation of the settlement device 40 by reading and executing programs stored in the ROM 402 .
The ROM 402 is a read-only memory that stores various types of information used by the CPU 401, including programs.

RAM 403 is a read/write memory that stores various information. For example, RAM 403 stores information read from ROM 402 or hard disk 404, information obtained from the outside, information generated during processing, etc.

The hard disk 404 stores various information. For example, instead of the ROM 402, the hard disk 404 may store programs executed by the CPU 401. Also, instead of the RAM 403, the hard disk 404 may store information read from the ROM 402, information obtained from the outside, information generated during processing, etc.

The customer-side display unit 405 is a touch display for customers, which displays various information to customers and accepts various inputs from customers.
The customer-side scanner unit 406 is a scanner unit for customers, and for example, scans a code attached to a product (for example, a barcode, a two-dimensional code, or other code other than a barcode) to read the product code. The customer-side scanner unit 406 may also scan a code (a barcode, a two-dimensional code, or the like) printed on a receipt (registered trademark) to read information necessary for payment. The customer-side scanner unit 406 may also scan a code (a two-dimensional code, a barcode, or the like) displayed on the display unit of the customer terminal device 50 to read information necessary for payment.

The customer-side scanner unit 406 is used by customers to register products, but customers may also register products using other methods. For example, if a preset key (a button for ordering a product) corresponding to the product is displayed on the customer-side display unit 405, the customer may operate (press) that preset key to register the product.

The card payment unit 408 is a payment mechanism that accepts various cards (credit cards, prepaid cards for transportation, etc.). In this embodiment, the card payment unit 408 is equipped with a card recognition unit (reading unit), a display unit, and an operation unit, but it is sufficient if it is equipped with at least a card recognition unit. Note that the card recognition unit may also recognize various cards that are not directly used for payment (settlement) (for example, membership cards, point cards, etc.).

The change dispenser 409 (cash settlement unit) is a cash settlement mechanism that has an input slot for banknotes and coins and an output slot for banknotes and coins. It calculates the amount inserted into the input slot, calculates the change amount, which is the difference between the input amount and the purchase amount, and dispenses the change from the output slot. The change dispenser 409 faces the customer and is operated by the customer. When banknotes or coins are inserted into the input slot, they are detected by a sensor (detecting that they have been inserted, the number of coins by denomination, etc.).

In other words, the change dispenser 409 is used when settling the price of registered products in cash (currency) at the settlement device 40. The change dispenser 409 has a banknote slot for inserting banknotes, a coin slot for inserting coins, a banknote outlet for discharging banknotes, a coin outlet for discharging coins, a counting unit for counting the inserted or discharged currency, a currency transport mechanism between the input or output port and the storage unit, the sensor described above, etc. The banknote input port and coin input port are also referred to as deposit input ports. The banknote outlet and coin outlet are also referred to as change outlets. The banknote input port and banknote outlet may be a common device, and the coin input port and coin outlet may be a common device.

The change dispenser 409 also counts the coins replenished during the closing process and stores them in the storage section. The change dispenser 409 also counts the coins to be dispensed during the closing process and releases them from the change outlet. The closing process is a process of adjusting the amount stored in the change dispenser 409 (cash on hand/cash on hand) to a reference amount after closing or before opening.

The store clerk's display unit 410 is a touch display for the store clerk, and displays various information to the store clerk and receives various inputs from the store clerk.
The key operation unit 411 is made up of various keys (buttons) and receives various inputs from the store clerk.
The store clerk scanner unit 412 is a scanner unit for store clerks, and for example, scans barcodes attached to products to read product codes. The store clerk scanner unit 412 also scans barcodes attached to store clerk name tags to read store clerk codes.

The clerk-side scanner unit 412 is used by the clerk when registering a product, but the clerk may also register the product using other methods. For example, if the key operation unit 411 has a key corresponding to the product (such as a key corresponding to a sports newspaper), the clerk may operate (press) that key to register the product. Also, if a preset key corresponding to the product is displayed on the clerk-side display unit 410, the clerk may operate that preset key to register the product.

The printing unit 413 prints and issues various media (receipts, bills, etc.). The printing unit 413 can freely rotate to face from the store clerk side to the customer side, or from the customer side to the store clerk side (the direction of the media issuing port). The orientation of the printing unit 413 may be changed manually, or it may be changed automatically (mechanically controlled, etc.) in response to, for example, a transition (switching) of the operating mode described below. The correct orientation of the printing unit 413 may be detected by a sensor or the like.

The audio output unit 414 outputs audio. For example, the audio output unit 414 outputs audio guidance and the like.
The communication unit 415 communicates with other devices (other settlement devices 40 and store controller 10).

In addition, the settlement device 40 of this embodiment can be switched between a total of four operating modes: a customer terminal-compatible settlement mode, which is set to process settlement based on customer terminal-compatible accounting, and three accounting modes (first to third accounting modes) corresponding to three different patterns of accounting (product registration, settlement).
The first payment mode is an operating mode in which a store clerk performs operations corresponding to product registration and operations corresponding to payment on the payment device 40. The second payment mode is an operating mode in which a store clerk performs operations corresponding to product registration on the payment device 40, and a customer performs operations corresponding to payment on the same payment device 40. The third payment mode is an operating mode in which a customer performs operations corresponding to product registration and operations corresponding to payment on the payment device 40.
As a result, in a store that has introduced the shopping system of this embodiment, some of the multiple payment devices 40 can be set to customer terminal-compatible payment mode to enable customer terminal-compatible payment, and other payment devices 40 can be set to, for example, first or second payment mode so that a store employee can handle the payment with the customer. Note that some of the other payment devices 40 may be set to the third payment mode.

[Configuration example of customer terminal device]
5 shows an example of the configuration of the customer terminal device 50. The customer terminal device 50 in the figure includes a CPU 501, a storage unit 502, a RAM 503, a display unit 504, an operation unit 505, an imaging unit 506, and a network-compatible communication unit 507.
The CPU 501 is a central processing unit that controls the operation of the customer terminal device 50 by reading and executing programs stored in the storage unit 502 .
The storage unit 502 is an auxiliary storage device for the CPU 501 and stores various types of information used by the CPU 501, including programs.
The RAM 503 is the main storage device of the CPU 501 .
The display unit 504 displays images according to the display control of the CPU 501. The display unit 504 may be configured as a touch panel that allows operations to be performed on the screen.
The operation unit 505 collectively indicates various operators provided on the customer terminal device 50 and various input devices connected to the customer terminal device 50 .
The imaging unit 506 captures images under the control of the CPU 401. The imaging unit 506 of this embodiment can also function as a part that reads two-dimensional codes in a code reader that processes two-dimensional codes.
The network-compatible communication unit 507 is a unit that performs communication wirelessly via a network. The network-compatible communication unit 507 enables the customer terminal device 50 to communicate with the transaction management device 60.

[Example of configuration of transaction management device]
6 shows an example of the configuration of transaction management device 60. Transaction management device 60 in the figure includes a CPU 601, a storage unit 602, a RAM 603, and a network-compatible communication unit 604.
The CPU 601 is a central processing unit that controls the operation of the transaction management device 60 by reading and executing programs stored in the memory unit 602 .
The storage unit 602 is an auxiliary storage device for the CPU 601 and stores various types of information used by the CPU 601, including programs.
The RAM 603 is the main storage device of the CPU 601 .
The network-compatible communication unit 604 is a part that performs communication via a network.

The memory unit 602 of the transaction management device 60 stores customer information, store information, product master (product information), and cart information.

[Configuration example of a transaction status management device]
7 shows an example of the configuration of the transaction status management device 80. The transaction status management device 80 in the figure includes a CPU 801, a storage unit 802, a RAM 803, an in-store communication unit 804, a network-compatible communication unit 805, a display unit 806, and an operation unit 807.
The CPU 801 is a central processing unit that controls the operation of the transaction status management device 80 by reading and executing programs stored in the storage unit 802 .
The storage unit 602 is an auxiliary storage device for the CPU 801 and stores various types of information used by the CPU 801, including programs.
The RAM 803 is the main storage device of the CPU 801 .
The in-store communication unit 804 communicates with terminals such as the checkout device 40 and store controller 10 within the store via the LAN 19 .
The network-compatible communication unit 805 is a component that communicates with the outside via a network. In the case of the transaction status management device 80, the network-compatible communication unit 805 can communicate with the transaction management device 60 on the cloud.
The display unit 806 displays an image under the control of the CPU 801. The display unit 806 may be configured as a touch panel.
The operation unit 807 collectively indicates the operators provided in the transaction status management device 80 and the input devices connected to the transaction status management device 80 .

[Information stored in the transaction management device]
The customer information stored in the memory unit 602 is information for managing individual customers. The transaction management device 60 generates customer information when a customer is registered (it may be considered that a customer has been registered when their customer information is stored). The transaction management device 60 also updates the customer information as appropriate based on cart information, etc. The transaction management device 60 may, for example, refer to the cart information at a predetermined time every day and update the customer information.

8A shows an example of customer information stored in the memory unit 602 of the transaction management device 60. The customer information in this figure includes customer identification information, customer name, customer registration date, cancellation information, customer rank, number of points, etc.
The customer identification information is identification information that uniquely identifies a customer.
The customer identification information may be application identification information that is uniquely assigned to each installed shopping application when the shopping application of this embodiment is installed. The application identification information may be, for example, a number that is uniquely (e.g., sequentially) assigned to each installed shopping application. Alternatively, the application identification information may be terminal identification information that is unique to the customer terminal device 50 on which the shopping application is installed. The terminal identification information may be, for example, a telephone number assigned to the customer who is the user of the customer terminal device 50, or hardware identification information that is uniquely assigned to the hardware of the customer terminal device 50, such as a MAC (Media Access Control) address.
The customer name is the customer's name or nickname, etc. The customer registration date is the date and time the customer was registered. The cancellation information is information regarding the cancellation of registered products after registration. The customer rank is a rank based on the customer's purchase history. When generating customer information for a new customer, the customer identification information, customer name, and customer registration date are generated, but since this is before the start of the actual transaction (product registration), other information (cancellation information, etc.) is not generated.

The transaction management device 60 generates and stores customer identification information, for example, when a customer is registered (for example, when the customer terminal device 50 downloads or installs a shopping application from an external device (for example, a predetermined server that provides all applications, the transaction management device 60) to use the shopping service provided by the transaction management device 60). Furthermore, the transaction management device 60 acquires, for example, information entered in the name field of the registration form (input form) using the customer terminal device 50 during customer registration, and stores this as the customer name. Furthermore, the transaction management device 60 acquires, for example, the current date and time during customer registration, and stores this as the customer registration date.

In addition to or instead of storing customer information in its own memory, the transaction management device 60 may store some or all of the customer information in another device (such as a file server accessible to the transaction management device 60).

8(B) shows an example of store information stored in the memory unit 602 of the transaction management device 60. The store information in the figure includes store identification information, store name (branch name), store identification information 1, and store identification information 2.
The store identification information is identification information that uniquely identifies a store. The store identification information shown in Fig. 8B is composed of a store (trade name) or company code and a branch code.
The store name is the name of the store. The store name shown in Fig. 8B is composed of the store (trade name) or company and the branch name.
The store identification information 1 is information of a two-dimensional code (QR code (registered trademark) or the like) for identifying the store where the transaction takes place (the store where the product is bought and sold).
The store identification information 2 is store location information (GPS information) for identifying the store where the transaction is to be made.
In the example shown in FIG. 8B, the store identification information and the store specifying information 1 are different, but the store identification information and the store specifying information 1 may be the same.

The transaction management device 60 may acquire and store store information from an external source (such as a server (not shown) at a headquarters that oversees each store). Furthermore, instead of or in addition to storing store information in its own memory, the transaction management device 60 may store some or all of the store information in another device (such as a file server accessible by the transaction management device 60).

Cart information is information used to manage individual transactions. The transaction management device 60 generates cart information at the start of a transaction. The transaction management device 60 also updates the cart information as the transaction progresses (each time a product is registered). (The storage of a product in the cart information can be interpreted as the product being registered.)

FIG. 8C is an example of cart information stored in the memory unit 602 of the transaction management device 60.
The cart information in the figure includes cart identification information, transaction start date and time, transaction end date and time, customer identification information, product information in the cart (registered product information, reserved product information), cancellation information, and the like.

The cart identification information is identification information that uniquely identifies the cart information, and includes, for example, store identification information, a date, and a serial number (for example, a serial number for each store and date).
The transaction start date and time is the date and time when the transaction started. The transaction start date and time may be, for example, the date and time when the cart information was created. The transaction start date and time may also be the date and time when the first item was registered (the date and time when the registered product information (registered product 1) in FIG. 8(C) was stored). The date and time when the cart information was created and the date and time when the first item was registered may both be stored separately.

The transaction end date and time is the end date and time of the transaction. For example, the transaction end date and time may be the settlement date and time. The customer identification information is customer identification information that identifies the customer of the transaction. When cart information is generated, cart identification information, transaction start date and time, and customer identification information are generated, but since this is before the actual start of the transaction (product registration), other information (transaction end date and time, etc.) is not generated. The settlement date and time may be the settlement start date and time or the settlement end date and time. Both the settlement start date and time and the settlement end date and time may be stored separately as the transaction end date and time.

Registered product information (total) is information that is updated each time a product is registered. Registered product information (total) includes the number of items (number of products), estimated subtotal amount (for example, the subtotal amount before discounts, etc. are reflected), subtotal amount, etc. Registered product information (registered product 1) is the registration information for one product item. Registered product information (registered product 2) is the registration information for two products. Note that in the example shown in Figure 8 (C), registered product information (registered product 3) to registered product information (registered product 5) are not shown. Registered product information (N; N is an integer) includes the product code, product name (product name), price, etc.

Registered product information (N) may include the registration date and time of the Nth item. In other words, the transaction management device 60 may store the registration date and time of the registered product as registered product information. The registration date and time of each product may also be used as information to determine whether or not to apply a limited-time offer or other service, and the effect of applying it.

The reserved product information (total) is information that is updated every time a reserved product is registered.
A reserved product is a product that was not registered properly due to a specific reason and is registered in a temporarily reserved state.
The reserved product information (total) includes the number of reserved products (number of products), the number of NON-FILE items among the reserved products, the number of items among the reserved products that cannot be read (check for unauthorized operations), and the like.

NON-FILE refers to a product whose barcode or product code was successfully scanned in the store (the product code was read), but the product code is not stored as product information in the product master (a product whose product information is not registered).

A reading failure refers to a failure to scan the product code at the store (the product code could not be read), or a product whose product code has not been scanned at the store. In other words, a reading failure refers to a timeout process when it is determined that the barcode has not been recognized even after the product has been imaged for a certain period of time using image recognition technology, or a product that has been timed out.
For example, if the barcode cannot be correctly acquired (recognized) despite the customer performing an operation to read it due to wrinkles in the package, blurred or dirty barcode printing, or the barcode being printed on a transparent film, it will be determined that the reading is not successful.

Reserved Item Information (Reserved Item 1) is information about reserved item 1. Reserved Item Information (Reserved Item 2) is information about reserved item 2. Reserved Item Information (Reserved Item 3) is information about reserved item 3.

The reserved product information (reserved product N; N is an integer) includes the reserved product type (information indicating whether the reserved product is a NON-FILE or failed to read) and image data (image data captured when the reading failed). For example, if the Nth item is a reserved product due to a NON-FILE, the reserved product information (reserved product N) includes the reserved product type "1 (NON-FILE)," the product code, and image data. Furthermore, if the Nth item is a reserved product due to a reading failed, the reserved product information (reserved product N) includes the reserved product type "2 (failed to read)" and image data.

9 shows an example of definition of reserved items corresponding to the cart information of this embodiment. In this figure, the reserved items are classified into eight categories. A classification number from "1" to "8" is assigned to each of the eight reserved item categories.
The classification of the reserved product with classification number 1 is "NON-FILE" as explained in FIG.
The classification of the reserved product with classification number 2 is "reading NG" as explained in FIG.

The classification of reserved items with classification number 3 is "pseudo scan input." Pseudo scan input reserved items are items for which the customer attempted to have the customer terminal device 50 read the product's code, but the customer terminal device 50 was unable to successfully read the code.

As a specific example, a "pseudo scan input" situation occurs when the product is brought into the imaging range of the imaging unit 506, but the product code is not included in the imaging range of the imaging unit 506.
Furthermore, "pseudo scan insertion" corresponds to a situation in which the product is brought into the imaging range of the imaging unit 506, but an image of a part without a code is captured.
Furthermore, "pseudo scan insertion" refers to a situation in which a product is placed within the imaging range of the imaging unit 506 with the code portion hidden by a finger or the like.
In addition, a "pseudo scan input" situation occurs when the product code is captured by the imaging unit 506, but the product is passed through the imaging range of the imaging unit 506 at a speed that the customer terminal device 50 cannot read the code.

The classification of the reserved product with classification number 4 is “non-scanned input.” A reserved product with “non-scanned input” is an item that was added to the shopping cart without performing an operation to position the product within the imaging range of the imaging unit 506.
This non-scanning insertion can occur, for example, when the customer terminal device 50 is fixedly attached to a shopping cart and is capable of capturing images of the inside of the shopping basket placed in the shopping cart. For example, when the customer terminal device 50 determines, based on the captured image, that a product has been placed in the shopping cart even though the product code has not been scanned, it can determine that a non-scanning insertion has occurred.

The classification of reserved items with classification number 5 is "age verification items." "Age verification items" are items such as alcohol and tobacco that require a store clerk to verify the customer's age before selling them.

The classification of reserved items with classification number 6 is "pharmaceuticals." "Pharmaceuticals" are products that require a licensed person, such as a pharmacist, to confirm with the purchaser.

The classification of reserved items with classification number 7 is "items with security tags." "Items with security tags" are items that have security tags attached and require a store clerk to remove the security tags from the items when they are sold.

The classification of reserved items with classification number 8 is "cancelled items." "Cancelled items" are items that were once registered but were canceled by a customer's cancellation operation on the customer terminal device 50. "Cancelled items" are collected from the customer by a store clerk, for example, at the time of payment.

In this embodiment, pending products that fall into one of the four categories of classification numbers 1 to 7, namely "NON-FILE," "Read NG," "Pseudo-scanned input," and "Non-scanned input," are unidentified products. Unidentified products are products whose registration results are incomplete, for example, because it was not possible to identify which of the products registered in the product master they belong to.

The product master is stored, for example, by the store controller 10. The product master stores product information for each product sold in the store. The product information for one product includes a predetermined number of information items, such as a product code and a product name.
The transaction management device 60 stores the product master data acquired from the store controller 10. The transaction management device 60 uses the stored product master data for processes related to product registration, etc.

For a reserved item corresponding to classification number 1 "NON-FILE," the item code was acquired by code scanning in response to the item registration operation, but the item information associated with the acquired item code is not registered in the item master. Therefore, it is not possible to identify which of the items registered in the item master the reserved item corresponds to. Specifically, customer terminal device 50 may transmit the item code acquired by code scanning to transaction management device 60, and transaction management device 60 may compare the transmitted item code with the item code registered in the item master stored in itself. If transaction management device 60 determines, as a result of the comparison, that a product code identical to the transmitted product code does not exist in the item master, it may treat the item as a reserved item corresponding to classification number 1 "NON-FILE" and notify customer terminal device 50 of this fact.
For pending products that fall into one of the three categories of classification numbers 2 to 4, "read NG,""pseudo scan input," and "non-scan input," the product code failed to be acquired as a result of the code scan in response to the product registration operation. For this reason, it is not possible to identify which of the products registered in the product master the pending products that fall into one of the three categories of classification numbers 2 to 4, "read NG,""pseudo scan input," and "non-scan input" belong to.
Such unspecified products are products that can be released from their status as reserved products (reserved status) and registered normally by a store clerk operating the transaction status management device 80 as described below.

In addition to or instead of storing the cart information in its own memory, the transaction management device 60 may store some or all of the cart information in another device (such as a file server accessible by the transaction management device 60).

[Example of basic processing procedure for self-operation]
The sequence diagram in Figure 10 shows an example of the processing procedure executed by customer terminal device 50, settlement device 40, and transaction management device 60 in response to the accounting of one transaction in self-service operation. Specifically, the processing in Figure 10 shows an example of the processing of customer terminal device 50, settlement device 40, and transaction management device 60 from when a customer registers a product to be purchased from products displayed in the store until the payment for the registered product is completed.

Step S1: The customer terminal device 50 acquires information identifying the store (store identification information). For example, a two-dimensional code identifying the store may be displayed near the entrance of the store (the two-dimensional code may be output on a display screen, a medium with the two-dimensional code printed on it may be attached, etc.), and a customer visiting the store may scan (read) the two-dimensional code with the customer terminal device 50, thereby allowing the customer terminal device 50 to acquire the store identification information. When a customer visiting the store launches a shopping application, a start-up screen may be displayed instructing the customer to scan the two-dimensional code. Alternatively, when a customer visiting the store scans the two-dimensional code with the customer terminal device 50, the shopping application may be launched, and a start-up screen may be displayed informing the customer that they are currently connected to the transaction management device 60.

Also, for example, since a store is identified by its location, a customer visiting the store may obtain location information (GPS information) on the customer terminal device 50 at the store (i.e., the store's location information may be obtained as store-identifying information). When a customer visiting the store launches a shopping application, the location information may be obtained and a startup screen may be displayed informing the customer that they are currently connected to the transaction management device 60. If multiple stores are detected from the location information and one cannot be identified, a selection screen may be displayed to allow the customer to make a selection. Alternatively, the store may be switched to a mode that forcibly obtains a two-dimensional code.

11A shows an example of a shopping application startup screen displayed on the display unit 504 of the customer terminal device 50. The startup screen in the figure displays the store name. This store name is displayed based on the store identification information acquired in step S1.
When a customer wants to start registering a product for shopping, the customer operates a transaction start instruction button BT10 arranged on the start screen.
In response to the operation of the transaction start instruction button BT10, the customer terminal device 50 transmits a transaction start request including the acquired store specific information and customer identification information to the transaction management device 60. The transaction start request may be transmitted in response to the customer's operation on the customer terminal device 50 as described above, or may be transmitted in response to the acquisition of store specific information without receiving any particular operation.
Regarding customer identification information, the information entered in the name field of the registration form using the customer terminal device 50 during customer registration (when downloading or installing a shopping application to the customer terminal device 50) may be stored in the customer information of the transaction management device 60, but it may also be stored in the memory of the customer terminal device 50 in addition to the transaction management device 60. When a store is identified (when the initial product registration screen described below is acquired), the store name, currently offered services (flyer information distributed that day), and available coupon information may be displayed on a screen (the initial product registration screen or a screen separate from the initial product registration screen). Information on services and coupons may be acquired, for example, as screen information from the transaction management device 60.

In addition, the destination information (the address of the transaction management device 60) may also be obtained during customer registration (when the shopping application is downloaded or installed on the customer terminal device 50) and stored in the memory unit of the customer terminal device 50. If a two-dimensional code is to be scanned, the destination information may also be converted into a two-dimensional code in addition to the store-specific information, and by scanning the two-dimensional code with the customer terminal device 50, the customer terminal device 50 may obtain the destination information along with the store-specific information.

Step S2: The transaction management device 60, which has received customer identification information and store identification information as a transaction start request from the customer terminal device 50, generates cart information for the transaction. Note that, in the initial state in which the cart information is generated, the cart identification information, transaction start date and time, and customer identification information are stored, but since the actual transaction (product registration) has not yet begun, other information (transaction end date and time, product information (registered product information, pending product information), etc.) is not stored.

As described above, the transaction management device 60 stores store information such as that shown in Figure 8 (B). Therefore, when it receives store identification information (also receives customer identification information) as a transaction start request from the customer terminal device 50, if the received store identification information is a two-dimensional code, it references store identification information 1 to acquire the store identification information, and if the received store identification information is location information (GPS information), it references store identification information 2 to acquire the store identification information. Note that if the store identification information received from the customer terminal device 50 is a two-dimensional code of the store identification information, the transaction management device 60 simply acquires it as is.

In other words, when the transaction management device 60 receives customer identification information and store identification information from the customer terminal device 50 as a transaction start request, it acquires the store identification information from the store identification information received from the customer terminal device 50, acquires the current date, issues (assigns) a serial number, combines the store identification information with the current date and serial number, and stores the combined information as cart identification information in the cart information. Furthermore, when the transaction management device 60 receives store identification information and customer identification information from the customer terminal device 50 as a transaction start request, it acquires the current date and time and stores the current date and time as the transaction start date and time (generation date and time) in the cart information. Furthermore, when the transaction management device 60 receives store identification information and customer identification information from the customer terminal device 50 as a transaction start request, it stores the customer identification information received from the customer terminal device 50 as customer identification information in the cart information.

Step S3: The transaction management device 60, which has generated the cart information for the transaction, generates product registration initial screen information (screen information for the product registration screen, which is the initial screen) and transmits it to the customer terminal device 50. Specifically, the transaction management device 60, for example, generates product registration initial screen information and transmits the generated product registration initial screen information to the customer terminal device 50 together with cart identification information.

Step S4: The customer terminal device 50, which has received the cart identification information and product registration initial screen information from the transaction management device 60, stores the cart identification information and displays the registration screen on the display unit. Specifically, the customer terminal device 50 displays, for example, the product registration initial screen as shown in FIG. 11 (B).

The initial product registration screen is a product registration screen that is displayed first when product registration is started.
The product registration screen has a captured image area AR1 and a product registration result area AR2. The captured image area AR1 is an area where a captured image obtained by the imaging unit 506 is displayed. A product code (e.g., a barcode) captured so as to fit within the range of the captured image displayed in the captured image area AR1 is to be read.
The product registration result area AR2 shows the number of registered products (number of registered products) and the total price for the products registered so far, and is an area where list items corresponding to each registered product are arranged. In the case of the product registration initial screen shown in the same figure, since no products have been registered yet, the product registration result area AR2 is blank with no list items arranged.

Step S5: In response to a customer's operation, the customer terminal device 50 scans the barcode attached to the product and reads the product code. In this figure, it is assumed that the barcode scan was successful. Steps S5 to S9 are repeated each time a barcode attached to a product is scanned.

After acquiring the barcode, the customer terminal device 50 transmits the cart identification information and product registration information, including the product code obtained by scanning, to the transaction management device 60.

Step S6: Upon receiving product registration information (cart identification information and product code) from the customer terminal device 50, the transaction management device 60 identifies the cart information for the transaction from the cart identification information included in the received product registration information.

Step S7: The transaction management device 60 updates the product data in the identified cart information. Specifically, if the received product registration information includes a product code for item N, the transaction management device 60 stores the product code as the product code of the registered product information (registered product N) in the identified cart information, obtains the product name and price corresponding to the product code from the product information, and stores them as the product and price of the registered product information (registered product N). The transaction management device 60 also updates the registered product information (total) in the identified cart information.

Step S8: Having updated the product data in the cart information, the transaction management device 60 generates product registration update screen information (screen information for the product registration screen, which is the update screen to which the registered product has been added) and transmits it to the customer terminal device 50. Specifically, the transaction management device 60 generates product registration update screen information such that the customer terminal device 50 displays a product registration update screen to which the list item Itm of the currently registered product has been added in the product registration result area AR2, as shown in FIG. 11 (C), and transmits the generated product registration update screen information to the customer terminal device 50 together with the cart identification information.

The product registration screen (product registration update screen) shown in Figure 11 (C) is displayed on the customer terminal device 50 after "○○ bread" is registered as the third product item. In other words, when "XX yogurt" is stored in the cart information as the first item, the transaction management device 60 generates product registration update screen information in which "XX yogurt" is displayed as the list item It on the customer terminal device 50 and transmits the generated product registration update screen information to the customer terminal device 50 together with the cart identification information; when "XX chocolate" is stored in the cart as the second item, the transaction management device 60 generates product registration update screen information in which "XX yogurt" and "XX chocolate" are displayed as list items It on the customer terminal device 50 and transmits the generated product registration update screen information to the customer terminal device 50 together with the cart identification information; and when "XX bread" is stored in the cart information as the third item, the transaction management device 60 generates product registration update screen information in which "XX yogurt", "XX chocolate", and "XX bread" are displayed as list items It on the customer terminal device 50 as shown in FIG. 11(C) and transmits the generated product registration update screen information to the customer terminal device 50 together with the cart identification information.

In response to the registration of product 1, a registered product presentation screen WD1 may be displayed, presenting the newly registered product, as shown in FIG. 12(A), for example. Furthermore, the registered product presentation screen WD1 in the same figure shows an example in which, in response to the product being available for eat-in, a "Yes" button BT1 and a "No" button BT2 are displayed to confirm with the customer whether or not they would like to eat or drink in the restaurant. In this case, the price of the product will be calculated so that the consumption tax rate for the product is different when the "Yes" button BT1 is operated and when the "No" button BT2 is operated.

Step S9: The customer terminal device 50, which has received the cart identification information and product registration update screen information from the transaction management device 60, adds the product to the registration screen. Specifically, the customer terminal device 50 displays a product registration update screen such as that shown in FIG. 11(C). As mentioned above, the product registration screen (product registration update screen) shown in FIG. 11(C) is displayed on the customer terminal device 50 after "XX bread" has been registered as the third product item.

Step S10: The customer terminal device 50 accepts a payment instruction operation as a customer operation. For example, it accepts a touch of the "Proceed to Checkout" button shown in Figure 11 (C).

Step S11: The customer terminal device 50 that has accepted the payment transition instruction operation generates a two-dimensional code. In other words, the customer terminal device 50 converts information required to execute the payment process for the items purchased by the customer terminal device 50 (e.g., cart identification information) into a two-dimensional code. The customer terminal device 50 that has generated the two-dimensional code displays the generated two-dimensional code on the display unit. For example, a code display screen with a two-dimensional code such as that shown in Figure 11 (D) is displayed on the display unit.

Step S12: The settlement device 40 scans (reads) the two-dimensional code displayed on the display unit of the customer terminal device 50. For example, the settlement device 40 scans the two-dimensional code displayed on the display unit of the customer terminal device 50 that the customer has pointed within the recognition range of the customer-side scanner unit 406.

Step S13: The settlement device 40 reads the two-dimensional code displayed on the display unit of the customer terminal device 50 and requests the transaction management device 60 to calculate the subtotal. For example, the settlement device 40 sends a calculation request (subtotal calculation request information) requesting the calculation of the subtotal to the transaction management device 60 along with the cart identification information obtained from the two-dimensional code.

Step S14: The transaction management device 60 receives the cart identification information and the sub-calculation request information from the customer terminal device 50 and identifies the cart information for the transaction from the cart identification information.

Step S15: Having identified the cart, the transaction management device 60 calculates the subtotal amount using information such as prices in the registered product information included in the cart information corresponding to the identified cart information.

Step S16: After calculating the subtotal, the transaction management device 60 updates the cart information (storing the subtotal (calculated subtotal)) and sends the subtotal information indicating the calculated subtotal along with the cart identification information to the settlement device 40.

Step S17: Having received the cart identification information and subtotal information from the transaction management device 60, the settlement device 40 displays the subtotal amount, for example, as shown in Figure 12 (B), on the customer display unit 405.

Step S18: The settlement device 40, which has displayed the subtotal on the customer display unit 405, executes the payment (settlement). Specifically, the settlement device 40 accepts the selection of the payment type. In the case of cash, it accepts the deposit, calculates the amount of change, and, if change is due, dispenses the change and issues a receipt. Furthermore, when the settlement is complete, the settlement device 40 sends settlement completion information along with cart information to the transaction management device 60, and the transaction management device 60 stores the transaction completion date and time (settlement date and time) for that cart.

After scanning a product, the customer terminal device 50 transmits the product code to the transaction management device 60 (S5). However, the customer terminal device 50 may be configured not to transmit product codes for products other than those scanned within the store (the store where the customer visited and displayed the product registration initial screen) (e.g., products scanned at another store). For example, the customer terminal device 50 acquires and stores location information (GPS information) when the customer visits the store (or when the product registration initial screen is displayed). The customer terminal device 50 also acquires location information when scanning each product and compares the location information acquired when the customer scanned the product with the location information acquired when the customer visited the store (or when the product registration initial screen is displayed). If the two information match (or approximately match), the customer terminal device 50 may permit the product code of the product to be transmitted to the transaction management device 60. If the two information do not match (or approximately match), the customer terminal device 50 may prohibit the product code of the product from being transmitted to the transaction management device 60.
This makes it possible to prevent inappropriate product registration (for example, product registration to a cart created at another store, etc.).

If the payment device 40 prohibits the transmission of the product code as described above, it may display an error message (e.g., "Registration is not possible because the item is not in the XX store") on the customer display unit 405 after scanning the item. The payment device 40 may also display the above message on the store clerk display unit 410 instead of or in addition to the message on the customer display unit 405.

Furthermore, after reading the two-dimensional code displayed on the display of the customer terminal device 50, the settlement device 40 requests the transaction management device 60 to calculate the subtotal (S13). However, the settlement device 40 may not request calculation of the subtotal for products other than those scanned in the store (e.g., products scanned at other stores). For example, the settlement device 40 references the store identification information of the store (the settlement device 40 may store and reference the store identification information of the store, or may reference store identification information stored in another accessible device), and when reading the two-dimensional code displayed on the display of the customer terminal device 50, compares the cart identification information obtained from the two-dimensional code with the store identification information of the store. The settlement device 40 may then permit the request for calculation of the subtotal if the store identification information included in the cart identification information is configured to include the store identification information of the store, but may prohibit the request for calculation of the subtotal if the cart identification information is not configured to include the store identification information of the store.
This makes it possible to prevent inappropriate payments (for example, payments for products registered at other stores, etc.).

If the payment device 40 prohibits the request for the subtotal amount as described above, it may display an error message (e.g., "Payment cannot be made because it includes products not from the XX store") on the customer display unit 405 after reading the two-dimensional code. The payment device 40 may also display the above message on the clerk display unit 410 instead of or in addition to the message on the customer display unit 405.

In the example processing procedure of Figure 10, the transaction management device 60 calculates the subtotal. However, for example, the settlement device 40 may acquire cart information from the transaction management device 60 and calculate the subtotal based on the acquired cart information. Alternatively, the transaction management device 60 may send cart information to the store controller 10 and request calculation of the subtotal, and the store controller 10 may calculate the subtotal using the received cart information and send the calculated subtotal to the settlement device 40 via the transaction management device 60.

In addition, during the settlement process, the transaction management device 60 may check the customer's cancellation status for previous items based on the customer information and cart information for the customer in question, and determine whether or not to issue a warning against fraud. If it determines that a warning should be issued, the transaction management device 60 may, for example, cause the settlement device 40 to issue a warning notification.

Furthermore, during the checkout process, the transaction management device 60 may determine whether the product information included in the cart information includes any reserved items, and if so, may request the checkout device 40 to provide instructions to correct the reserved items. Upon receiving a request to provide instructions to correct the reserved items, the checkout device 40 displays a message indicating that there are reserved items, as shown in FIG. 12(C). Upon confirming the message, the store clerk can cancel (correct) the reserved items by entering the price corresponding to the reserved items or canceling the reserved items, etc.

[Example of procedure for canceling registration of registered products]
There are cases where a customer has registered a product but wishes to cancel the registration, for example, because they registered the wrong product or because they have decided not to purchase the product. Therefore, in the shopping system of this embodiment, while the customer terminal device 50 is in the process of registering the product, the customer can cancel the registration of the registered product by operating the customer terminal device 50. An example of the procedure a customer follows to cancel the registration of a product is described below.
In the following explanation, we will use as an example a case where the customer terminal device 50 is attached to a shopping cart so as to capture an image of the inside of the shopping cart. In this case, the captured image of the inside of the shopping cart is displayed in the captured image area AR1 of the product registration screen.

When the customer terminal device 50 is in the process of registering a product and it becomes necessary to cancel a certain product that has already been registered, the customer selects the list item Item of the product to be canceled from the list items Item in the product registration result area AR2 on the product registration screen. The operation of selecting the list item Item may be an operation of tapping the list item Item. In response to the operation of selecting the list item Item, the product registration change screen is displayed.

The product registration change screen is a screen on which the user can change the registered details (change the quantity or cancel the registration) of the product corresponding to the selected list item Itm.
13A shows an example of a product registration change screen P30. The product registration change screen P30 shown in the figure is an example in the form of a pop-up screen that is displayed superimposed on the product registration screen.
The product registration change screen P30 shown in the figure is displayed in response to the selection of the product list item It for "XX Chocolate" from among the three list items Itm in the product registration result area AR2 in the figure. In this case, in the product registration result area AR2, the two list items It for products other than "XX Chocolate" are grayed out, indicating that they are not available for selection, and the list item It for "XX Chocolate" is not grayed out, indicating that it is available for selection.

The product registration change screen P30 is arranged with a cancel button BT31, a quantity price area AR30, a plus button BT32-1, and a minus button BT32-2.
The cancel button BT31 is a button that is operated to cancel the registration of the target product. When the cancel button is operated, if the registered quantity of the target product is two or more, the entire registered quantity is canceled.
The quantity/price area AR30 indicates the registered quantity of the target product corresponding to the current time and the price according to the registered quantity.
The plus button BT32-1 is a button that is operated when making a change to increase the registered quantity of the target product. Each time the plus button BT32-1 is operated, the registered quantity is increased by one.
The minus button BT32-2 is a button that is operated to decrease the registered quantity of the target product. Each time the minus button BT32-2 is operated, the registered quantity is decreased by one.
When the plus button BT32-1 or the minus button BT32-2 is operated to change the registered quantity, the registered quantity and price in the quantity price area AR30 are changed accordingly.

In this embodiment, decreasing the registered quantity in response to the operation of the minus button BT32-2 may also be treated as canceling a product. In other words, when the registered quantity of a registered product is decreased, the product corresponding to the decreased quantity may be treated as being canceled.

When the product registration change screen P30 is displayed and, for example, the list item Itm corresponding to the target product is operated, the product registration change screen P30 is cleared and the product registration screen is returned to being displayed.

To cancel the entire quantity of the target product, the customer presses the Cancel button BT31. When the Cancel button BT31 is pressed, the Product Registration Change Screen P30 is cleared and the Cancellation Confirmation Screen is displayed instead.

13B shows an example of a cancellation confirmation screen WD40. The cancellation confirmation screen WD40 in the drawing shows an example in the form of a pop-up screen that is displayed superimposed on the product registration screen.
When a customer is performing an operation to cancel a target product, the target product is in a state where the customer has reserved it, for example, by adding it to the customer's shopping cart. The cancellation confirmation screen WD40 is a screen that notifies the customer that the registration cancellation has been successfully completed in response to the operation to cancel the product. The cancellation confirmation screen WD40 in the same figure also guides the customer to return the target product to the location where it was displayed, such as a display shelf (to return it to an unreserved state).
The cancellation confirmation screen WD40 displays a message informing the user that the product has been cancelled and guiding the user to return the product to a non-reserved state, and also includes a confirmation button BT40.

In response to the cancellation of the registration, the list item Itm of the canceled product in the product registration result area AR2 indicates that the cancellation has been made by, for example, erasing the quantity and striking through the product name and price, as shown in the figure. Furthermore, such canceled products are managed as "cancelled products," which are reserved products with classification number 8, in the cart information. The strikethrough through the product name and price of the list item Itm also indicates that the corresponding product is a reserved product classified as a "cancelled product."
Also, in the figure, although it is hidden by the cancellation confirmation screen WD40, the display of the number of registered items and the total amount in the product registration result area AR2 has been changed to reflect the cancellation of the product registration.

In addition, when a registration is canceled, the listing item Itm for the affected product may be deleted.

When the customer looks at the cancellation confirmation screen WD40 and has not yet returned the canceled product to its non-reserved state, they can, for example, return the product to the display shelf to return it to its non-reserved state, and then operate the cancel button BT31. When the cancel button BT31 is operated, the cancellation confirmation screen WD40 is cleared, and the display returns to the product registration screen. Even when the display returns to the product registration screen, the list item Itm for the canceled product will still indicate that it has been canceled.

[Notification of non-reserved status when canceling the registration of a registered product]
As described above, when a customer cancels the registration of a product that has already been registered, the customer must display the cancellation confirmation screen WD40, perform the registration cancellation operation (operate the cancel button BT31), and return the previously reserved product to its unreserved state, for example by returning it to the display shelf.
However, there are cases where a customer cancels a registration but forgets to return the item to its non-reserved state. In this case, the number of items actually in the shopping cart may be greater than the number of items registered in the cart information, resulting in a discrepancy between the breakdown of registered items and the breakdown of items actually reserved by the customer.
Alternatively, a malicious customer may cancel the registration but intentionally leave the item in the shopping cart without restoring it to a non-reserved state, and then proceed with the checkout, attempting to complete the purchase without paying for the item.

The customer terminal device 50 therefore determines whether the target product has been returned to a non-secured state in response to the registration cancellation operation, and if it determines that the product has not been returned to a non-secured state, it will provide guidance to the customer to return the target product to a non-secured state.

When determining whether the target product has been returned to an unsecured state, the customer terminal device 50 is configured to detect the removal of a product from the shopping cart by using a captured image obtained by capturing an image of the inside of the shopping cart. For example, this detection can be achieved by configuring the customer terminal device 50 to recognize product objects from the captured image. Furthermore, the customer terminal device 50 may be configured to detect that a product has been removed from the shopping cart when any of the product objects that were present in the captured image transition to a state where they are no longer recognized in the captured image.
In this case, the customer terminal device 50 may be configured to recognize a human hand from the captured image, and the result of the human hand recognition may also be used to detect the removal of a product from the shopping cart. For example, after a human hand is recognized in the captured image, the customer terminal device 50 may detect that a product has been removed from the shopping cart if the human hand is no longer recognized in the captured image and the product that was recognized in the captured image transitions to an unrecognized state in the captured image. In this case, the mobile terminal 300 may be configured to execute a process of identifying the product type for each product object.
Such detection of an item being removed from a shopping cart may be achieved by a learning model that uses images of a person removing an item from a cart as learning data.
For example, the customer terminal device 50 may transmit an image to the transaction management device 60, and the transaction management device 60 may use the transmitted image to detect the removal of an item from the shopping cart using the processing described above.

The customer terminal device 50 is then configured to detect whether or not a product has been removed from the shopping cart during a determination period, for example, from the time the cancellation confirmation screen WD40 is deleted until a certain amount of time has elapsed.
If removal of a product from the shopping cart is detected within a certain period of time from the time the cancellation confirmation screen WD40 is deleted, the customer terminal device 50 determines that the target product has been returned to an unsecured state in response to this registration cancellation operation.
If no removal of an item from the shopping cart is detected within a certain period of time from the time the cancellation confirmation screen WD40 is erased, the customer terminal device 50 determines that the target item has not been returned to an unsecured state in response to this registration cancellation operation.

It is possible that the customer may return the target product to an unsecured state while the product registration change screen P30 is being displayed or before the product registration change screen is displayed.
Therefore, the customer terminal device 50 may be configured to determine whether or not a product has been detected being removed from the shopping cart during a determination period that begins when the product registration change screen P30 begins to be displayed and ends when a certain amount of time has passed since the cancellation confirmation screen WD40 was deleted.
Alternatively, the customer terminal device 50 may be configured to determine whether or not a product has been removed from the shopping cart during a determination period that begins a predetermined time prior to the start of displaying the product registration change screen P30 and ends a predetermined time after the cancellation confirmation screen WD40 is deleted. In this case, for example, if the customer terminal device 50 stores captured images taken up to a predetermined time prior to the deletion of the cancellation confirmation screen WD40, the captured images taken up to a predetermined time prior to the deletion of the cancellation confirmation screen WD40 can be used for detection.

If the customer terminal device 50 determines that the product has not been returned to a non-reserved state in response to the registration cancellation operation, it displays a non-reserved re-guidance screen. The non-reserved re-guidance screen is a screen that again guides the customer to return the product corresponding to the registration cancellation operation to a non-reserved state.

A specific example of a situation in which the non-reserved re-guidance screen is displayed will be described with reference to Figure 14. Figures 14(A) to 14(D) show examples of screen transitions on the display unit 504 of the customer terminal device 50 in response to actions taken by the customer in relation to the cancellation of the registration of a registered product.
As shown in the product registration result area AR2 of the product registration screen in Figure 14 (A), after the customer has registered three products, "XX Potatoes,""XXTea," and "XX Chocolate," he operates the list item Itm for "XX Chocolate" to display the product registration change screen P30 in order to cancel the registration of "XX Chocolate." The customer operates the cancel button BT31 on the displayed product registration change screen P30 to cancel the registration of "XX Chocolate."

In response to the operation of the Cancel button BT31, the customer terminal device 50 erased the product registration change screen P30 and displayed the Cancel confirmation screen WD40, as shown in FIG. 14 (B). The customer operated the Confirm button BT40 located on the Cancel confirmation screen WD40.

When the confirmation button BT40 is operated, the cancellation confirmation screen WD40 is cleared and the product registration screen is displayed as shown in FIG. 14(C). In the product registration results area AR2 on the product registration screen in the same figure, the quantity display is cleared in the list item Itm for the "XX Chocolate" product, and a strikethrough is placed across the product name and price to indicate that the registration has been canceled.

Furthermore, the captured image area AR1 of the product registration screen in the same figure shows that the "XX Chocolate" product that was the subject of cancellation remains in the shopping cart without being removed. In other words, at this stage, the customer still has the "XX Chocolate" product that was the subject of cancellation. For example, in this case, the customer performed the operation to cancel the registration of the "XX Chocolate" product, but forgot to return the "XX Chocolate" product to its unreserved state by returning it to the display shelf.

The customer left the "XX Chocolate" product in the shopping cart, forgetting to return it to an unreserved state even after a certain amount of time had passed since the cancellation confirmation screen WD40 was dismissed. Therefore, the customer terminal device 50 determined that the product had not been returned to an unreserved state in response to the registration cancellation operation, even after a certain amount of time had passed since the cancellation confirmation screen WD40 was dismissed.

Therefore, the customer terminal device 50 displays the non-reserved re-guidance screen WD50 as shown in Fig. 14(D). The non-reserved re-guidance screen WD50 in Fig. 14(D) shows an example in the form of a pop-up screen that is displayed superimposed on the product registration screen.
The non-reserved re-guidance screen WD50 in the same figure displays a message for the customer to confirm that although there are products whose registration has been cancelled, the products in question have not yet been returned to a non-reserved state, and displays a confirmation button BT50.

The customer notices by looking at the displayed non-reserved re-guidance screen WD50 that they have performed the registration cancellation operation but have not returned the target product to a non-reserved state. The customer then returns the "XX Chocolate" product to the display shelf to make it non-reserved, and operates the confirmation button BT50. It does not matter which comes first: returning the "XX Chocolate" product to the display shelf to make it non-reserved, or operating the confirmation button BT50. In response to the operation of the confirmation button BT50, the non-reserved re-guidance screen WD50 is cleared and the product registration screen is displayed.

In this manner, in this embodiment, if the target product has not been returned to the non-reserved state even after a certain time has passed since the cancellation confirmation screen WD40 was deleted, the non-reserved re-guidance screen WD50 is displayed. This allows the customer to realize that the target product has not yet been returned to the non-reserved state, and to return the target product to, for example, a display shelf. In other words, the customer can properly carry out the procedure for canceling the product registration.
Furthermore, even if a malicious customer performs a registration cancellation operation but intentionally leaves the target product in the shopping cart without restoring it to a non-secured state, the non-secured re-guidance screen WD50 is displayed, increasing the possibility that the customer will restore the target product to a non-secured state, thereby deterring fraudulent behavior.

If a certain amount of time has passed since the non-reserved re-guidance screen WD50 was previously cleared without the relevant product being returned to a non-reserved state, the customer terminal device 50 may display the non-reserved re-guidance screen WD50 again. This more effectively prevents and deters fraudulent behavior, such as customers forgetting to return the relevant product to a non-reserved state or customers not returning a product that they have canceled registration of to a non-reserved state.

[Regarding the cancellation procedure for registering registered products]
Conversely to the above example, there may be cases where a customer who wants to cancel the registration of a registered product returns the product to the shelf or other means to return it to an unsecured state, but forgets to perform the cancellation operation. In such cases, the number of products registered in the cart information may be greater than the number of products actually in the shopping cart, resulting in a discrepancy between the breakdown of products actually reserved by the customer and the breakdown of registered products.

Therefore, when the customer terminal device 50 determines that a product that was once reserved by a customer has been returned to an unreserved state, it determines whether a registration cancellation operation has been performed to cancel the registration of the product that has been returned to an unreserved state, and if it determines that a registration cancellation operation has not been performed, it will provide guidance to the customer to perform a registration cancellation operation for the product in question.

In this case, the customer terminal device 50 is configured to constantly detect the removal of an item from the shopping cart while the item is being registered, using captured images of the inside of the shopping cart. The customer terminal device 50 then determines whether a registration cancellation operation has been performed during a predetermined determination period that includes the time when it is detected that the item has been removed. Specifically, the customer terminal device 50 may determine whether a registration cancellation operation has been performed within a certain period of time since it was detected that the item has been removed. The customer terminal device 50 may also include a certain period of time from the time when it was detected that the item has been removed in the period used to determine whether a registration cancellation operation has been performed.

If the customer terminal device 50 determines that a registration cancellation operation has not been performed during the above-mentioned determination period, it displays a registration cancellation operation guide screen. The registration cancellation operation guide screen is a screen that guides the customer to perform a registration cancellation operation corresponding to the product that has been set to an unsecured state.

Referring to Figure 15, we will explain a specific example of a situation in which the registration cancellation operation guide screen is displayed. Figures 15(A) to 15(C) show example screen transitions on the display unit 504 of the customer terminal device 50 in response to actions taken by a customer in relation to canceling the registration of a registered product.

The product registration screen in Figure 15 (A) corresponds to the stage where three products, "XX Potatoes," "XX Tea," and "XX Chocolate," have been registered. In this case, the captured image area AR1 displays a captured image of the three products, "XX Potatoes," "XX Tea," and "XX Chocolate," in the shopping cart. The product registration result area AR2 displays list items Itm corresponding to each of the products, "XX Potatoes," "XX Tea," and "XX Chocolate."

In this state, the customer decides to cancel the registration of the "XX Chocolate" product. To do this, the customer first removes the "XX Chocolate" product from the shopping cart and returns it to an unsecured state (for example, by returning it to the display shelf).
In this way, when the customer removes the "XX Chocolate" product from the shopping cart and returns it to an unsecured state, the image displayed in the captured image area AR1 changes to a state in which the "XX Chocolate" product is not present, as shown in Figure 15 (B).

Furthermore, the customer terminal device 50 detects that a product has been removed from the shopping cart based on the change in the captured image from when the "XX Chocolate" product was present to when it is no longer present, as described above. When the customer terminal device 50 detects the removal of a product in this manner, it sets a determination period and determines whether a registration cancellation operation has been performed between the start and end of the determination period. During the determination period, the customer terminal device 50 may not accept new product registrations.
In this case, the customer returned the "XX Chocolate" product to the shelf, but forgot to perform the corresponding registration cancellation operation. As a result, a certain amount of time passed after the customer terminal device 50 detected the product's removal, and the judgment period ended without the registration cancellation operation being performed. In this case, the customer terminal device 50 determines that the registration cancellation operation was not performed within the judgment period.

In response to determining that the registration cancellation operation has not been performed within the determination period, the customer terminal device 50 displays a registration cancellation operation guide screen. Fig. 15(C) shows an example of the registration cancellation operation guide screen WD60. The registration cancellation operation guide screen WD60 shown in Fig. 15(C) is an example in the form of a pop-up screen superimposed on the product registration screen.
The registration cancellation operation guidance screen WD60 displays a message guiding the registration cancellation operation for the product that has just been returned to the non-reserved state, and a confirmation button BT60.

By looking at the displayed registration cancellation operation guide screen WD60, the customer realizes that although he or she has returned the product to be cancelled to an unsecured state, he or she has not performed the corresponding registration cancellation operation. The customer then operates the confirmation button BT60 to close the registration cancellation operation guide screen WD60 and display the product registration screen. The customer operates the displayed product registration screen to perform the registration cancellation operation for the "XX Chocolate" product, thereby canceling the registration of the "XX Chocolate" product.

In this way, in this embodiment, if it is determined that the customer has not performed the deregistration operation within a certain period of time since returning the product to the non-reserved state, the deregistration operation guide screen WD60 is displayed. This allows the customer to realize that the deregistration operation has not yet been performed, and to perform the deregistration operation. In other words, even in this case, the customer can properly perform the procedure to deregistrate the product.

When the registration cancellation operation guide screen WD60 is deleted once, the customer terminal device 50 may set a judgment period until a certain time has elapsed, and if the judgment period has elapsed without a registration cancellation operation being performed, the registration cancellation operation guide screen WD60 may be displayed again.

[Example of processing procedure]
Referring to the flowchart of FIG. 16, an example of a processing procedure executed by the customer terminal device 50 and the transaction management device 60 in relation to canceling the registration of a product is shown.
Step S101: With the product registration screen displayed on the display unit 504, the customer terminal device 50 determines whether the product code has been read (the product code has been acquired) by code scanning in response to the product registration operation.

Step S102: In response to the reading of the product code, the customer terminal device 50 transmits product registration information to the transaction management device 60. The product registration information includes the product code read in step S101 and cart identification information.

Step S103: The customer terminal device 50 receives the registration completion notification sent from the transaction management device 60 in response to the transmission of the product registration information in step S102. The customer terminal device 50 uses the information on the product registration update screen included in the received registration completion notification to update the product registration screen, for example, as illustrated in Figure 11 (C) etc.

Step S104: After processing in step S103, or if it is determined in step S101 that the product code was not read, the customer terminal device 50 determines whether a registration cancellation operation has been performed. The registration cancellation operation here is a series of operations in which the customer taps the list item Itm of the product to be canceled on the product registration screen to display the product registration change screen P30 (Figure 13 (A)), and then taps the cancel button BT31 on the displayed product registration change screen P30.

Step S105: If it is determined that a registration cancellation operation has been performed, the customer terminal device 50 transmits registration cancellation information to the transaction management device 60. The registration cancellation information is information that notifies the transaction management device 60 that a registration cancellation operation has been performed on the customer terminal device 50. The registration cancellation information includes a product code indicating the product designated as the target for registration cancellation and cart identification information.

Step S106: The customer terminal device 50 receives the cancellation completion notice sent from the transaction management device 60 in response to the sending of the product registration information in step S105. The customer terminal device 50 uses the information on the product registration update screen included in the received cancellation completion notice to update the product registration screen so that it reflects that the target product has been canceled in the product registration result area AR2, as shown in Figure 12 (B), for example.
Step S107: The customer terminal device 50 also displays a cancellation confirmation screen WD40 as shown in FIG. 13(B) and other figures.
Step S108: In response to an operation on the confirmation button BT40 on the cancellation confirmation screen WD40, the customer terminal device 50 erases the cancellation confirmation screen WD40. As a result, the product registration screen in which the cancellation of the target product is reflected in the product registration result area AR2 as a result of the update in step S106 is displayed as the top screen.

Step S109: When the cancellation confirmation screen is cleared, the customer terminal device 50 sets a determination period and determines whether or not a product has been detected being removed from the shopping cart based on the captured image during the set determination period.
If the removal of a product is detected, the process skips steps S110 to S112 (described later) and proceeds to step S113, in which case the non-reserved item re-guidance screen WD50 is not displayed.

Step S110: If the customer terminal device 50 determines in step S109 that product removal has not been detected, it determines whether the determination period set in response to the detection of product removal in step S109 has elapsed (ended). If it determines that the determination period has not elapsed, processing returns to step S109. When processing returns from step S110 to step S109 in this way, the cancellation confirmation screen may be cleared and scanning of product codes for new product registration and registration cancellation operations for other registered products may not be accepted until the determination period has elapsed.

Step S111: If it is determined in step S110 that the determination period has elapsed without the removal of a product being detected in step S109, the customer terminal device 50 displays the non-reserved re-guidance screen WD50 (Figure 14 (D)).
Step S112: In response to an operation on the confirmation button BT50 on the non-reserved re-guidance screen WD50, the customer terminal device 50 erases the non-reserved re-guidance screen WD50 and displays the product registration screen as the top screen.

Step S113: If it is determined in step S104 that a registration cancellation operation has not been performed, or if the removal of a product within the determination period is detected in step S109, or after processing in step S112, the customer terminal device 50 determines whether the removal of a product from the shopping cart has been detected based on the captured image while in a state in which product registration can be accepted.
The determination of whether removal has been detected in step S109 corresponds to the determination of whether the target product has been returned to a non-secured state in response to the deregistration operation. In contrast, the determination of whether removal has been detected in step S113 is a process for determining whether the target product has been returned to a non-secured state prior to the deregistration operation when deregistering the product.

Step S114: If removal of a product from the shopping cart is detected in step S113, the customer terminal device 50 sets a determination period and determines whether or not a registration cancellation operation has been performed during the set determination period.
Step S115: If a registration cancellation operation is not detected in step S114, the customer terminal device 50 determines whether the determination period set in step S114 has elapsed. If the determination period has not elapsed, the process returns to step S114. When the process returns from step S115 to step S114 in this way, reading of a product code for new product registration or a registration cancellation operation for other registered products may be prohibited until the determination period has elapsed.

Step S116: If it is determined in step S114 that a registration cancellation operation has been performed during the determination period, registration cancellation information is sent to the transaction management device 60. The registration cancellation information includes a product code indicating the product designated as the target for registration cancellation and cart identification information.
Step S117: The customer terminal device 50 receives the cancellation completion notice sent from the transaction management device 60 in response to the sending of the product registration information in step S116. The customer terminal device 50 uses the information on the product registration update screen included in the received cancellation completion notice to update the product registration screen so that the product registration result area AR2 reflects that the target product has been canceled.
In this case, since the target product has already been returned to an unsecured state, the customer terminal device 50 does not need to execute the process related to displaying the cancellation confirmation screen WD40 as in steps S107 and S108 above.

Step S118: If the determination period has elapsed without the registration cancellation operation being performed in step S115, the customer terminal device 50 displays the registration cancellation operation guide screen WD60 (FIG. 15(C)).
Step S119: In response to an operation performed on the confirmation button BT60 on the registration cancellation operation guide screen WD60, the customer terminal device 50 erases the registration cancellation operation guide screen WD60 and displays the product registration screen as the top screen.

Step S120: If no product removal is detected in step S113, or after processing in step S117 or S119, the customer terminal device 50 determines whether a payment instruction has been performed while the product registration screen is displayed. If a payment instruction has not been performed, processing returns to step S101.

Step S121: When the payment transfer instruction operation is performed, the customer terminal device 50 converts the cart identification information into a two-dimensional code and displays a code display screen (Figure 11 (D)) on the display unit 504, displaying the generated two-dimensional code.

Next, an example of a processing procedure executed by the transaction management device 60 will be described.
Step S201: The transaction management device 60 determines whether or not the product registration information transmitted from the customer terminal device 50 in step S102 has been received.

Step S202: When the transaction management device 60 receives the product registration information, it identifies the cart information indicated by the cart identification information included in the received product registration information.
Step S203: The transaction management device 60 updates the cart information identified in step S202 by registering a new product. That is, the transaction management device 60 adds product information based on the product code included in the product registration information received in step S201 to the identified cart information. This registers the product in the cart information.
Step S204: When the transaction management device 60 completes the processing of step S203, it generates a product registration update screen that reflects the results of updating the cart information in step S203 (addition of the newly registered product). The transaction management device 60 sends a registration completion notice including the generated product registration update screen to the customer terminal device 50.

Step S205: After processing step S204, or if it is determined in step S201 that product registration information has not been received, the transaction management device 60 determines whether registration cancellation information sent from the customer terminal device 50 in step S105 or step S116 has been received.

Step S206: When the transaction management device 60 receives the registration cancellation information, it identifies the cart information indicated by the cart identification information included in the received registration cancellation information.
Step S207: The transaction management device 60 updates the cart information identified in step S206 so that the product indicated by the product code included in the received registration cancellation information is managed as a canceled product (a pending product classified as "cancelled product" under classification number 8).
Step S208: When the transaction management device 60 completes the processing of step S207, it generates a product registration update screen that reflects the results of updating the cart information in response to the cancellation of the product registration in step S207. The transaction management device 60 transmits a cancellation completion notice including the generated product registration update screen to the customer terminal device 50.

<Modification>
Modifications of this embodiment will be described below. The modifications described below may be combined with the above embodiment and other modifications as appropriate.

[First Modification]
Examples of ways to "return a product that is subject to deregistration to a non-secured state" include returning the product to its original display location, such as the display shelf where it was displayed, as well as handing over the product to the nearest store clerk or bringing the product to a predetermined storage location for cancelled products in the store.

[Second Modification]
In the above embodiment, the removal of an item from the shopping cart is detected using a captured image. However, the removal of an item from the shopping cart may be detected, for example, by providing a weighing scale capable of measuring the weight of the shopping cart placed in the shopping cart, and detecting the removal of an item from the shopping cart based on the change in weight measured by the weighing scale. Alternatively, instead of a shopping cart, a weighing scale may be provided on the bottom of the shopping cart.
Furthermore, removal of an item from the shopping cart may be detected based on the detection output of an optical sensor, a vibration sensor, or the like.

[Third Modification]
In the above embodiment, an example has been given in which the customer terminal device 50 is attached to a shopping cart so as to be able to capture images of the inside of the shopping cart and executes processing in response to product registration or the like.
However, for example, there may be cases where a customer removes the customer terminal device 50 from the shopping cart and holds it in their hand to read the product code of the product to be registered. In this case, there is a possibility that a registration cancellation operation will be performed on the product that was registered by reading the product code while the customer is holding the customer terminal device 50 in their hand.
As described above, if a customer holds the customer terminal device 50 in their hand, registers a product, and then immediately performs a registration cancellation operation, there is a high possibility that the customer will return the product to the display shelf without placing it in their shopping cart, thereby placing the product in an unsecured state. Therefore, the customer terminal device 50 does not need to display the unsecured re-guidance screen WD50 if the registration cancellation operation is performed within a predetermined time (e.g., several tens of seconds) after the customer holds the customer terminal device 50 in their hand and registers the product.
In other words, in this case, the customer side display unit 405 does not need to determine whether or not an item has been removed from the shopping cart within the determination period.
In this modified example, it is necessary to determine whether or not the customer is holding the customer terminal device 50. This determination may be made by the customer terminal device 50, for example, based on the detection output of a gyro sensor provided therein.

[Fourth Modification]
Furthermore, if the customer returns the target product to an unsecured state before canceling the registration, and if the item of the target product detected as being removed from the shopping cart can be identified based on the captured image, the registration of the identified product may be cancelled. The product cancellation process according to this modification may be performed, for example, by the customer terminal device 50 and the transaction management device 60 working together, without involving any operation by the customer on the customer terminal device 50.
Specifically, for example, the transaction management device 60 may identify the item of the product removed from the shopping cart based on the captured image sent from the customer terminal device 50, and manage the product as a pending product classified as a ``cancelled product.''
With the configuration of this modified example, if the target product is returned to the non-secured state before the registration cancellation operation is performed, the registration of the target product can be treated as having been properly cancelled without performing the registration cancellation operation after the product is returned to the non-secured state. In this case, for example, the registration cancellation operation guide screen WD60 does not need to be displayed.

Furthermore, if the customer returns the target product to a non-secured state before performing the registration cancellation procedure, and the item of the target product detected as being removed from the shopping cart cannot be identified, the registration cancellation operation can be made unnecessary and the registration cancellation operation guide screen WD60 can be prevented from being displayed as follows.
In other words, in this case, when the removal of an item from the shopping cart is detected, at least one of the currently registered items may be treated as a reserved item. Alternatively, a virtual (dummy) reserved item may be added to the cart information. By including the reserved item in the cart information in this way, a store clerk will be called in to release the reserved item before the checkout. When the store clerk attends, they will notice that the contents of the cart information and the breakdown of items in the shopping cart do not match, and they can then correct the cart information appropriately and proceed with the checkout.

Furthermore, if the customer returns the target product to a non-reserved state before canceling the registration as a procedure for canceling the registration, the list item Itm of the reserved product may be added to the product registration result area AR2 on the product registration screen in response to the detection of the product being removed from the shopping cart.
In addition, if the registration cancellation operation is not performed within a certain period of time after the removal of a product from the shopping cart is detected, the registration cancellation operation guide screen WD60 may be displayed.
Alternatively, instead of displaying the registration cancellation operation guidance screen WD60, a message, icon, etc. guiding the user to perform the registration cancellation operation may be displayed in the list item It of the added pending product, for example, immediately after the pending product list item It is added, or when a certain amount of time has passed since the product was detected as being removed from the shopping cart but no registration cancellation operation has been performed.

[Fifth Modification]
In the above embodiment, a product whose registration has been cancelled by a registration cancellation operation is managed as a "cancelled product" which is a reserved product with classification number 8. Furthermore, on the product registration screen, the list item Itm of a product whose registration has been cancelled by a registration cancellation operation has a strikethrough drawn through the product name and price, thereby indicating that the corresponding product is a reserved product classified as a "cancelled product."
If the cart information includes an item that has been treated as a reserved item, a store clerk will perform an operation to cancel the reserved item's status (reserved status), for example, before checkout. In other words, if the items managed in the cart information include a reserved item, the store clerk must intervene to cancel the reserved status. Considering that the shopping system of this embodiment is originally intended to allow customers to perform the entire process from product registration to checkout themselves, it is preferable that the cart information not include reserved items.
Therefore, for example, the transaction management device 60 may be configured not to manage products (properly canceled products) as reserved products in the cart information for which it has been confirmed that both the registration cancellation operation and the return to a non-secured state have been performed.
In this case, in response to the registration cancellation operation being performed, the list item Itm of the product whose registration has been cancelled may be deleted from the product registration screen of the customer terminal device 50.
The transaction management device 60 may delete the product information corresponding to the properly canceled product from the cart information, but may also add information (a flag) indicating that the product information corresponding to the properly canceled product has been canceled to the cart information so that a store clerk can check the cancellation status of the registration later, and store the information in the cart information, or store the information as part of the registered transaction history.

[Sixth Modification]
In the above embodiment, part of the processing performed by the customer terminal device 50 may be performed by, for example, the transaction management device 60, the transaction status management device 80, the settlement device 40, the store controller 10, other higher-level devices in the store, or a server device on the cloud. Similarly, part of the processing performed by the transaction management device 60, the transaction status management device 80, the settlement device 40, the store controller 10, etc. may also be performed by other devices.
As one of the above aspects, the shopping system of this embodiment may be provided with a thin client system configuration. As an example, the transaction management device 60 or another server may be configured as a thin client-compatible server device (e.g., a cloud server), and the customer terminal device 50, transaction status management device 80, etc. may function as thin clients.
In this case, for example, the cloud server may determine whether a product has been removed from a shopping cart or whether a registration cancellation operation has been performed within a determination period by acquiring the captured image sent from the customer terminal device 50 and the detection output of the weighing scale, optical sensor, vibration sensor, etc. described in the second variant as detection information, and using the acquired detection information.
Furthermore, the transaction management device 60, transaction status management device 80, store controller 10, etc. may be configured to be distributed across multiple computer devices or servers.

<Summary of the embodiment>
[Technical Field]
The present invention relates to a product sales data processing system and a mobile terminal device.
[Background technology]
2. Description of the Related Art A system is known in which a customer registers an item using a mobile terminal device (see, for example, Patent Document 1).
[Prior art documents]
[Patent Documents]
[Patent Document 1] JP 2019-204551 A [Summary of the Invention]
[Problem to be solved by the invention]
In the above-described product sales data processing system, a customer may register a certain product and reserve it by adding it to a shopping cart, but then wish to cancel the registration, for example, because they have selected the wrong product or have decided not to purchase it. In such cases, the customer can cancel the registration of the product by operating their mobile terminal, and return the deregistered product to its original display location, such as a display shelf. However, there are cases in which the customer fails to properly follow the procedure for canceling the product registration. If the procedure for canceling the product registration is not properly followed, the product registration results will not match the details of the product reserved by the customer, which will hinder smooth checkout.
The present invention has been made in consideration of the above circumstances, and aims to enable customers to properly carry out procedures for canceling product registration in a product sales data processing system in which customers register products using a terminal device.

[Means for solving the problems]
(1) As described above, one aspect of this embodiment is a product sales data processing system (e.g., a shopping system) in which a customer registers products using a mobile terminal device (e.g., customer terminal device 50), and is equipped with a detection means for detecting the removal of a product from a product insertion section (e.g., a shopping cart) into which products reserved by the customer are inserted, and a notification means for notifying the customer of the cancellation of the registration of a registered product (e.g., a non-reservation re-guidance screen WD50, a registration cancellation operation guidance screen WD60) based on the detection result by the detection means.

With the above configuration, a notification can be sent depending on whether or not a product has been detected as being removed from the shopping cart, and the customer can be guided to take the appropriate action to cancel the registration of a registered product. This allows customers to properly follow the steps to cancel the registration of a product in a shopping system where customers register products using the customer terminal device 50.

(2) One aspect of this embodiment is the product sales data processing system described in (1), in which the notification means issues a notification guiding the customer to return the deregistered product to an unsecured state when a certain period of time has passed since the customer canceled the registration of the registered product through a registration cancellation operation without the detection means detecting the product being removed.

With the above configuration, if a customer does not return the target product to a non-secured state within a certain period of time after performing the registration cancellation operation, the customer is prompted to return the target product to a non-secured state, allowing the customer to properly carry out the procedure for canceling the product registration.

(3) One aspect of this embodiment is the product sales data processing system described in (1) or (2), wherein the notification means issues a notification guiding the user to perform a registration cancellation operation corresponding to the product detected as having been removed from the product insertion section if a registration cancellation operation is not performed within a certain period of time after the detection means detects that the product has been removed.

With the above configuration, if a customer does not perform the registration cancellation operation within a certain period of time after returning the product to a non-reserved state, the system will guide the customer to perform the registration cancellation operation, allowing the customer to properly perform the procedure for canceling the product registration.

(4) One aspect of this embodiment is a product sales data processing system according to any one of (1) to (3), which includes a server device equipped with the detection means, and the detection means acquires predetermined detection information indicating a change in response to the removal of a product from the product insertion section, and detects the removal of a product from the product insertion section based on the acquired detection information.

With the above configuration, the server device in the shopping system detects when an item is removed from a shopping cart, making it possible to obtain highly accurate detection results through advanced processing. Furthermore, since the customer terminal device 50 does not need to perform processing to detect when an item has been removed, the processing load on the customer terminal device 50 is reduced, making it possible to make the operation of the customer terminal device 50 when registering items more agile.

(5) One aspect of this embodiment is a mobile terminal device used by a customer to register a product, the mobile terminal device comprising: a detection means for detecting the removal of a product from a product insertion section into which the product secured by the customer is inserted; and a notification means for notifying the customer of the cancellation of the registration of a registered product based on the detection result by the detection means.

In addition, a program for implementing the functions of the store controller 10, settlement device 40, customer terminal device 50, transaction management device 60, transaction status management device 80, etc. described in each of the above embodiments may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be loaded into a computer system and executed to perform processing as the store controller 10, settlement device 40, customer terminal device 50, transaction management device 60, transaction status management device 80, etc. Here, "loading a program recorded on a recording medium into a computer system and executing it" includes installing the program into a computer system. The term "computer system" here includes hardware such as an OS and peripheral devices. The term "computer system" may also include multiple computer devices connected via a network, including communication lines such as the Internet, WAN, LAN, and dedicated lines. The term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into a computer system. In this way, the recording medium storing the program may be a non-transitory recording medium such as a CD-ROM. The term "recording medium" also includes internal or external recording media accessible from a distribution server for distributing the program. The program code stored on the distribution server's recording medium may be different from the program code in a format executable on a terminal device. In other words, the format in which the program is stored on the distribution server is not important as long as it can be downloaded from the distribution server and installed in a format executable on a terminal device. The program may be divided into multiple parts, downloaded at different times, and then combined on a terminal device, or each part may be distributed by a different distribution server. Furthermore, "computer-readable recording medium" also includes a medium that stores a program for a certain period of time, such as volatile memory (RAM) within a computer system that serves as a server or client when a program is transmitted over a network. The program may also be a medium that implements some of the functions described above. Furthermore, the program may be a so-called differential file (differential program) that can implement the functions described above in combination with a program already stored on the computer system.

10 Store controller, 40 Payment device, 50 Customer terminal device, 60 Transaction management device, 70 Credit card payment server, 80 Transaction status management device

Claims (5)

a registration unit that registers products based on product information acquired in response to customer operations on a terminal device that moves around the store with the customer ;
a cancellation unit that cancels the registration of a target product among the registered products in response to a registration cancellation operation being performed on the terminal device ;
a display control unit that controls the display content of a display unit of the terminal device;
Equipped with
The display control unit
Displaying a list of the products registered by the registration unit;
In response to the cancellation of a product by the cancellation unit, the product whose registration has been cancelled is displayed in the list display in a manner that allows it to be distinguished from products whose registration has not been cancelled, and a first guide image that guides the user to return the cancelled product to a non-secured state is displayed in a position in the list display where the cancelled product can be distinguished.
A product sales data processing system. The registration unit
registering the product based on the product information captured by the imaging unit of the terminal device;
The display control unit
a first display area for displaying the list and a second display area for displaying the captured image captured by the imaging unit;
The first guide image is displayed superimposed on the captured image at a position that overlaps at least the second display area.
2. The product sales data processing system according to claim 1. The terminal device or the server device has a detection unit that detects the removal of a product from a product insertion unit into which the product secured by the customer is inserted,
The display control unit
When the detection unit detects that a product has been removed and the product has not been canceled by the cancellation unit, a second guide image that guides the user to perform the registration cancellation operation for the removed product is displayed superimposed on the captured image at a position that overlaps at least the second display area.
3. The product sales data processing system according to claim 2. A terminal device that moves around the store together with a customer,
a registration unit that registers products based on product information acquired in response to customer operations;
a cancellation unit that cancels the registration of a target product among the registered products in response to a registration cancellation operation being performed on the terminal device ;
a display control unit that controls the display content of a display unit of the terminal device;
Equipped with
The display control unit
Displaying a list of the products registered by the registration unit;
In response to the cancellation of a product by the cancellation unit, the product whose registration has been cancelled is displayed in the list display in a manner that allows it to be distinguished from products whose registration has not been cancelled, and a first guide image that guides the user to return the cancelled product to a non-secured state is displayed in a position in the list display where the cancelled product can be distinguished.
A terminal device characterized by: A computer as a terminal device that moves around the store with customers.
a registration unit that registers products based on product information acquired in response to customer operations;
a cancellation unit that cancels the registration of the relevant product among the registered products in response to a registration cancellation operation being performed on the terminal device;
a display control unit that controls the display content of a display unit of the terminal device;
A program for causing the device to function as a
The display control unit
Displaying a list of the products registered by the registration unit;
In response to the cancellation of a product by the cancellation unit, the product whose registration has been cancelled is displayed in the list display in a manner that allows it to be distinguished from products whose registration has not been cancelled, and a first guide image that guides the user to return the cancelled product to a non-secured state is displayed in a position in the list display where the cancelled product can be distinguished.
Program.