JP7725082B2 - Information processing system, information processing device and program - Google Patents

Description

The present invention relates to an information processing system, an information processing device, and a program.

A cash register is known that displays video footage of the operation taken by a camera as a function to check (inspect) whether any fraudulent operations have been performed (see, for example, Patent Document 1).

Japanese Patent Application Laid-Open No. 2018-120635

However, there is room for improvement in the method of checking operation status using conventional technology such as that shown in Patent Document 1.

The present invention was made in light of these circumstances, and aims to provide technology for conveniently checking operation status.

One aspect of the invention that solves the above-mentioned problem is an information processing device that progresses processing based on customer operations, and that includes an imaging means for capturing an image of the customer and a display means that can accept touch operations and displays information according to the progress of the processing, and that displays, on the same screen, an image captured by the imaging means and a display transition image showing the transition of the content displayed by the display means, with the image capture time and the display time coinciding.

1 is an example of the appearance of a registration and settlement device. 1 is a configuration example of a registration and settlement device. 10 is a display example of a registration settlement device. 10 is a display example of a registration settlement device. 10 is a display example of a registration settlement device. 10 is a display example of a registration settlement device. 10 is a display example of a registration settlement device.

(Sales system)
The registration and settlement device 10 of this embodiment is one of the devices constituting a sales system 1 (not shown). The sales system 1 can be introduced into various business types (supermarkets, convenience stores, drug stores, home improvement stores, electronics retailers, etc.). In addition to the registration and settlement device 10, the sales system 1 also includes a transaction status management device 40 (not shown), a store controller 50 (not shown), and a headquarters server 60 (not shown). The registration and settlement device 10, the transaction status management device 40, and the store controller 50 are installed within the store. These devices are communicatively connected via a LAN (which may be wired or wireless). The headquarters server 60 is installed outside the store (e.g., a headquarters or data center). The devices installed within the store (the registration and settlement device 10, the transaction status management device 40, and the store controller 50) can communicate (directly or via the store controller 50) with a device installed outside the store (the headquarters server 60). The number of devices installed is not particularly limited. The headquarters server 60 may be a cloud server. The headquarters server 60 may be made up of two or more servers.

(Registration settlement device 10)
The registration and settlement device 10 is a so-called fully self-service cash register that registers products based on customer operations and settles payments based on the customer operations. In other words, the registration and settlement device 10 executes a registration process for registering products and a settlement process for settling payments for registered products, but both processes are basically executed based on the customer's own operations. The registration and settlement device 10 can also execute an operation confirmation and verification process (described below) based on the operations of a store clerk.

(Transaction status management device 40)
The transaction status management device 40 is a device that manages devices installed in the store (monitoring and controlling processing status, operating status, etc.). The transaction status management device 40 may also manage operators of devices installed in the store (monitoring operations performed by customers while using the device, and checking operations performed by customers after use). The transaction status management device 40 includes at least a CPU, a memory unit, a display unit (which may be configured as a touch panel), an operation unit (keyboard, mouse, etc.), and a communication unit. The transaction status management device 40 is used by store staff.

(Store Controller 50)
The store controller 50 is a server that mainly handles communication of information between devices installed inside the store and devices installed outside the store. The store controller 50 includes at least a CPU, a memory unit, and a communication unit.

Note that devices other than those described above may also be installed within the store. For example, a registration and settlement device 210 (not shown) that performs registration and settlement processes based on the operations of a store clerk, a registration device 220 (not shown) that performs registration processes based on the operations of a store clerk, or a settlement device 230 (not shown) that acquires registration information from the registration device 220 and performs settlement processes based on the operations of the customer may be installed. Also, a settlement device 250 (not shown) that acquires registration information from a portable registration terminal 240 (not shown) that performs registration processes based on the operations of a customer in the sales area and performs settlement processes based on the operations of the customer may be installed. A charging device 260 (not shown) that charges electronic money may also be installed within the store.

Figure 1 shows an example of the exterior of the registration and settlement device 10. The registration and settlement device 10 comprises a main body 11, an upstream counter 12, and a downstream counter 13. The upstream counter 12 is a counter located upstream of the main body 11 on the customer flow path. For example, products that have not yet been registered or shopping baskets containing products that have not yet been registered are placed on the upstream counter 12. The downstream counter 13 is a counter located downstream of the main body 11 on the customer flow path. For example, products that have been registered or shopping bags (reusable bags, plastic bags that require payment, etc.) containing products that have been registered are placed on the downstream counter 13. Note that the upstream counter 12 (as well as the downstream counter 13) may be a separate unit attached to the registration and settlement device 10 (main body 11).

A cash payment unit (change dispenser) 106 is provided in the lower part of the main body 11. A printing unit 108, a scanner unit 104, a display unit 103, a non-cash payment unit (payment terminal) 105, an imaging unit (camera) 110, and a support pole unit 111 are provided in the upper part of the main body 11. The support pole unit 111 is provided with, in order from the top, a light-emitting unit 112, a monitoring display unit 113, and a detection unit (sensor unit) 120.

Figure 2 shows an example configuration of the registration and settlement device 10. Although some of the components have already been mentioned in Figure 1, the registration and settlement device 10 includes a CPU 101, a memory unit 102, a display unit 103, a scanner unit 104, a non-cash payment unit 105, a cash payment unit 106, a communication unit 107, a printing unit 108, an audio output unit 109, an imaging unit 110, a light-emitting unit 112, a monitoring and display unit 113, and a detection unit 120. These components can communicate with each other via a bus. Although not shown, the registration and settlement device 10 also includes a timing unit (timing function). For example, the registration and settlement device 10 can use the timing unit to manage the current time, the elapsed time from a certain point in time, and the remaining time until a certain time. The registration and settlement device 10 may also include an audio input unit (microphone).

The CPU 101 is a central processing unit that controls the operation of the registration and settlement device 10 by reading and executing programs stored in the memory unit 102. For example, the CPU 101 controls the entire transaction, from product registration to settlement. In other words, the CPU 101 executes (progresses) the registration process and settlement process.

(Generation of transaction information, electronic journal information, and operation log information)
The CPU 101 generates various types of information. For example, the CPU 101 generates information (hereinafter referred to as "transaction information") indicating transaction details, including product information (product registration information) for products registered in each transaction and payment information (information including payment type). The CPU 101 also generates transaction history information (hereinafter referred to as "electronic journal information") that records events (hereinafter referred to as "events"; described below) that occurred in each transaction, and operation history information (hereinafter referred to as "operation log information") that records the history of operations (described below) in each transaction. The CPU 101 generates transaction information, electronic journal information, and operation log information without explicit instructions (such as operations from the operator).

(Events recorded in electronic journal information)
Events stored in the electronic journal information include, for example, screen transitions (display/deletion), display/deletion of small screens, scan results (e.g., barcode reading OK, barcode reading NG, non-file (PLU failed)), product registration (product registration by scanning a barcode, product registration by product preset button), product cancellation, increase/decrease in purchase quantity, selection of payment method (selected payment method), display/deletion of various messages (e.g., error, alert, guidance), occurrence/resolution of an error, a predetermined period of no operation where no operation is performed on the display unit 103 or the scanner unit 104 (scanning operation), notification from the non-cash payment unit 105 (described below), notification from the cash payment unit 106 (described below), calling a store clerk, etc.

Notifications from the non-cash payment unit 105 include, for example, notifications that various waiting states (e.g., waiting for a credit card to be read, waiting for a PIN (Personal Identification Number) to be entered) have continued for more than a predetermined time, notifications that an unauthorized credit card (one that has been reported lost) has been read, notifications that a PIN has been entered, notifications that the entered PIN is incorrect, notifications that authorization has been denied, notifications that payment has been completed, notifications that a credit card has been left behind, and in the case of electronic money, notifications that there is insufficient balance.

Notifications from the cash settlement unit 106 include, for example, notifications that various waiting states (e.g., waiting for cash to be inserted, waiting for change to be removed) have continued for a predetermined period of time or longer, notifications regarding inserted coins (deposits) (that they have been inserted, the number of coins by denomination), notifications regarding dispensed coins (change) (that they have been dispensed, the number of coins by denomination), etc.

In other words, for example, when screen A is displayed, the registration and settlement device 10 (CPU 101) stores the fact that screen A has been displayed as an event in the electronic journal information; when small screen B is displayed, the fact that small screen C has been displayed as an event in the electronic journal information; when product C is registered using the product preset button, the fact that product C has been registered using the product preset button as an event in the electronic journal information; when product D is canceled, the fact that product D has been canceled as an event in the electronic journal information; and when a store clerk is called, the fact that a store clerk has been called as an event in the electronic journal information. Furthermore, when a PIN is entered, the registration and settlement device 10 stores the fact that the PIN has been entered as an event in the electronic journal information; and when 5,000 yen (one 5,000 yen bill) is inserted as a deposit, the fact that the deposit was 5,000 yen (one 5,000 yen bill) is stored as an event in the electronic journal information.

In addition, when the total amount (transaction amount) of registered products (registered products) reaches (or exceeds) a predetermined amount (for example, 10,000 yen), the registered settlement device 10 may store in the electronic journal information as an event that the total amount of registered products has reached (or exceeded) the predetermined amount. When the total amount of registered products using the product preset buttons reaches (or exceeds) a predetermined amount, the registered settlement device 10 may store in the electronic journal information as an event that the total amount of registered products has reached (or exceeded) the predetermined amount. When the number of operations of the product preset buttons reaches (or exceeds) a predetermined number, the registered settlement device 10 may store in the electronic journal information as an event that the number of operations of the product preset buttons has reached (or exceeded) the predetermined number.

When the number of canceled products (the number of products canceled using the cancel button) reaches (or exceeds) a predetermined number, the registered settlement device 10 may store in the electronic journal information as an event that the number of canceled products has reached (or exceeded) the predetermined number. When the number of decreased purchase quantities (the number of products decreased using the minus button) reaches (or exceeds) a predetermined number, the registered settlement device 10 may store in the electronic journal information as an event that the decreased purchase quantities have reached (or exceeded) the predetermined number. When the total amount of canceled or decreased products (products canceled using the cancel button or products decreased using the minus button) reaches (or exceeds) a predetermined amount, the registered settlement device 10 may store in the electronic journal information as an event that the total amount of canceled or decreased products has reached (or exceeded) the predetermined amount.

When an act of taking away unregistered goods (described below) occurs, the registered settlement device 10 may record in the electronic journal information as an event that an act of taking away unregistered goods has occurred. When the number of occurrences (detections) of acts of taking away unregistered goods (described below) reaches (or exceeds) a predetermined number, the registered settlement device 10 may record in the electronic journal information as an event that the number of occurrences of acts of taking away unregistered goods (described below) has reached (or exceeded) the predetermined number.

The registered settlement device 10 generates electronic journal information so that it is possible to determine at what point (timing) within a transaction an event occurs. For example, the registered settlement device 10 may store in the electronic journal information the current time at the start of a transaction and the current time at which each event occurs. In other words, by referencing the electronic journal information, it may be possible to determine that a transaction started at XX:XX minute, XX seconds, event a occurred at XX:XX minute, and event b occurred at XX:XX minute, △ seconds. The registered settlement device 10 may also store in the electronic journal information the current time at the start of a transaction and the elapsed time from the start of a transaction when each event occurred. In other words, by referencing the electronic journal information, it may be possible to determine that a transaction started at XX:XX minute, XX seconds after the start of a transaction, event a occurred, and event b occurred △△ seconds later.

(Operations recorded as history in operation log information)
The operations stored as history in the operation log information are operations that can change the display content of the display unit 103. In other words, the operations stored as history in the operation log information are operations that can change the display content of the display unit 103. The registration and settlement device 10 (CPU 101) stores operations that can change the display content of the display unit 103 in the operation log information. Specifically, the registration and settlement device 10 stores operations on the display unit 103 in the operation log information. In other words, because an operation on the display unit 103 may change the displayed information within the same screen or transition to another screen (because an operation on the display unit 103 may change the displayed content), the registration and settlement device 10 stores operations on the display unit 103 in the operation log information. For example, the registration and settlement device 10 stores the type of button touched, the touch position (coordinates), etc. as operations on the display unit 103 in the operation log information. In addition to cases where the display content of the display unit 103 changes due to a touch, even when the display content of the display unit 103 does not change due to a touch (for example, when a button does not respond when touched (when a button that cannot be operated is touched) or when a location other than a button is touched), the type of button touched (determined from the touch position), the touch position, etc. may be stored in the operation log information.

The display content of the display unit 103 can change depending on operations performed on the display unit 103, but the display content of the display unit 103 can also change depending on other operations. For example, the display content of the display unit 103 can also change depending on operations performed on the scanner unit 104 (scanning operation), operations performed on the non-cash payment unit 105 (such as operating the buttons on the non-cash payment unit 105), and operations performed on the cash payment unit 106 (such as inserting banknotes or coins, removing change, etc.). Therefore, in addition to operations performed on the display unit 103, the registration and settlement device 10 also stores operations performed on the scanner unit 104, the non-cash payment unit 105, and the cash payment unit 106 in the operation log information.

The registration and settlement device 10 (CPU 101) generates operation log information so that it is possible to determine at what point in a transaction an operation occurred. For example, the registration and settlement device 10 may store in the operation log information the current time at the start of a transaction and the current time at the time of each operation. That is, by referencing the operation log information, it may be possible to determine that a transaction started at XX:XX minute, XX seconds, operation c occurred at XX:XX minute, and operation d occurred at XX:XX minute, △ seconds. The registration and settlement device 10 may also store in the operation log information the current time at the start of a transaction and the elapsed time from the start of a transaction at the time of each operation. That is, by referencing the operation log information, it may be possible to determine that a transaction started at XX:XX minute, XX seconds after the start of the transaction, operation c occurred, and △△ seconds later.

The registration and settlement device 10 may store items other than operations in the operation log information. For example, if the display content of the display unit 103 changes due to something other than an operation (if there is an operation other than an operation that changes the display content of the display unit 103), the registration and settlement device 10 may store items other than operations that change the display content of the display unit 103 as operation log information. If the display unit 103 is configured to display a message (alert, etc.) when a no-operation state, in which no operation is performed on the display unit 103 or no operation (scanning operation) is performed on the scanner unit 104, continues for a predetermined period of time, the registration and settlement device 10 may store in the operation log information an event indicating that a no-operation state has continued for a predetermined period of time. In other words, the registration and settlement device 10 may store in the operation log information some of the events stored in the electronic journal information. The registration and settlement device 10 may store in the operation log information items other than operations that change the display content of the display unit 103 (items that do not correspond to events stored in the electronic journal information). Operations other than those that change the display content of the display unit 103 may be stored in at least one of the following: operation log information, electronic journal information, or third history information that is different from both operation log information and electronic journal information.

(Generation of display transition images)
The CPU 101 reproduces the display content on the display unit 103 (more specifically, the transition of the display content, i.e., the history of changes in the display content). Specifically, the CPU 101 generates an image (hereinafter referred to as a display transition image or a display history image) showing the transition of the display content on the display unit 103. The display content on the display unit 103 changes depending on the operator's operation (such as a change in display information within the same screen or a transition to another screen), so the CPU 101 generates (assembles) the display transition image based on the screen information of each screen and the operation log information (i.e., the operations actually performed). The display transition image is a moving image because it represents the transition of the display content and is generated based on the operation log information that includes a temporal component.

In addition, if the display content of the display unit 103 changes due to something other than an operation (for example, if a configuration is used in which a message is displayed on the display unit 103 when a predetermined period of no operation has continued), and if something other than an operation that changes the display content of the display unit 103 (that a predetermined period of no operation has continued) is stored as operation log information, the CPU 101 generates a display transition image by referring to the operation log information.

On the other hand, if the display content of the display unit 103 changes due to something other than an operation and operations other than those that change the display content of the display unit 103 are not stored as operation log information, the CPU 101 generates a display transition image by referencing not only the operation log information but also information that stores operations other than those that change the display content of the display unit 103 (e.g., electronic journal information, third history information).

The CPU 101 may generate a display transition image when there is an explicit instruction (a specified operation by a store clerk, for example, an operation by a store clerk on the screens of Figures 5(B), 7(A), and 7(B)), or it may generate a display transition image even without an explicit instruction, as with transaction information, electronic journal information, and operation log information.

The memory unit 102 is a collective term for one or more various memory components (for example, memory or storage, such as RAM, ROM, hard disk, SSD (Solid State Drive), etc.). The memory unit 102 stores various information. For example, the memory unit 102 stores programs executed by the CPU 101, information referenced by the CPU 101 (for example, product master), information generated by the CPU 101 (for example, transaction information, electronic journal information, operation log information, display transition images), images captured by the imaging unit 110, etc.

The product master is a file that stores product names, sales prices, etc. in association with product identification information (e.g., JAN codes or product codes pre-set for each product preset button).

The CPU 101 stores transaction information in the memory unit 102 so that it is possible to identify which transaction it corresponds to. For example, the CPU 101 may generate transaction information that includes transaction identification information (transaction number) that is assigned (numbered) for each transaction and store the generated transaction information in the memory unit 102, or it may generate transaction information that does not include transaction identification information and store the generated transaction information in the memory unit 102 in association with the transaction identification information. Similarly, the CPU 101 stores electronic journal information and operation log information in the memory unit 102 so that it is possible to identify which transaction it corresponds to. The transaction number is assigned (numbered) when the transaction begins (after the registration start button BT01, described below, is operated).

The CPU 101 stores the display transition image in the storage unit 102 so that it is clear which transaction it corresponds to. Specifically, the CPU 101 may generate a display transition image that includes the transaction identification number of the operation log information that was the basis for generating the display transition image and store the generated display transition image in the storage unit 102, or it may generate a display transition image that does not include the transaction identification information and store the generated display transition image in the storage unit 102 in association with the transaction identification information.

The CPU 101 stores the image captured by the imaging unit 110 in the storage unit 102 so that it is possible to identify which transaction it is for. For example, the CPU 101 associates the image captured by the imaging unit 110 with transaction identification information and stores it in the storage unit 102.

In other words, transaction information, electronic journal information, operation log information, display transition images, and captured images are all stored in the memory unit 102 so that it is clear which transaction each corresponds to.

The display unit 103 is a touch display that displays various information to the operator (customer) (e.g., a screen for the registration process, a screen for the settlement process), and also accepts operations by the operator. As described above, the operation history (touch positions) of the operator on the display unit 103 is stored in the memory unit 102 as operation log information.

The scanner unit 104 optically reads various information. For example, the scanner unit 104 scans the code attached to the product and reads the product code.

The non-cash payment unit 105 is a payment mechanism for non-cash payments (e.g., credit card payments, electronic money payments, etc.). The non-cash payment unit 105 may be a payment terminal separate from the registration and settlement device 10. The payment terminal 105 includes, for example, a reading unit, a communication unit (a communication unit that communicates with the registration and settlement device 10, and a communication unit that communicates with external servers such as credit servers that execute credit card payments), an operation unit, and a display unit. The cash payment unit 106 is a cash payment mechanism. The cash payment unit 106 has an insertion slot for banknotes and coins and an ejection slot for banknotes and coins. The cash payment unit 106 detects currency inserted into the insertion slot, identifies the number of coins inserted for each denomination, calculates the inserted amount, calculates the change amount, which is the difference between the inserted amount and the purchase amount, and ejects the change from the ejection slot. The insertion of banknotes and coins and the removal of change may be detected by a sensor.

The communication unit 107 transmits and receives information to and from other devices (e.g., the transaction status management device 40, the store controller 50, etc.). For example, the communication unit 107 receives a product master from the store controller 50 (or the headquarters server 60). The communication unit 107 also transmits transaction information to the store controller 50 (or the headquarters server 60).

The printing unit 108 prints and issues various media (receipts, coupons). A sensor may be used to detect the removal of media. The audio output unit 109 outputs audio (e.g., confirmation sounds, warning sounds, audio guidance, etc.).

The imaging unit 110 captures an image of the operator (customer). Specifically, the imaging unit 110 captures an image near the front of the scanner unit 104. That is, the imaging unit 110 can capture images of the customer's scanning operation, the operation of inserting cash into the cash settlement unit 106 (or the operation of removing cash from the cash settlement unit 106), etc. The image captured by the imaging unit 110 may be a still image or a moving image. For example, under the control of the CPU 101, the imaging unit 110 captures moving images from the start of the transaction (when the registration start button BT01 is operated) to the end of the transaction (when a receipt is issued or when removal of the receipt is detected). Note that the images (still images and moving images) captured by the imaging unit 110 are stored in the memory unit 102, as described above.

The registration and settlement device 10 may be equipped with an imaging unit 1110 (not shown) that captures an image of the customer's face in addition to the imaging unit 110 that captures an image of the area near the front of the scanner unit 104 (such as the customer's scanning operation or cash insertion operation). The registration and settlement device 10 may use the imaging unit 1110 to capture a still image of the customer's face at the start of the transaction (when the registration start button BT01 is operated) as a representative face image. The registration and settlement device 10 may also be equipped with an imaging unit 2110 (not shown) that captures an image of the upstream side (the top surface of the upstream counter 12) and an imaging unit 3110 (not shown) that captures an image of the downstream side (the top surface of the downstream counter 13). The registration and settlement device 10 may be equipped with multiple imaging units as described above, but the PIN input area in the non-cash payment unit 105 is not captured (or is stored after processing). Like imaging unit 110, imaging unit 1110 (and imaging units 2110 and 3110 as well) is capable of capturing both moving images and still images.

The light-emitting unit 112 emits light to indicate various pieces of information. For example, the light-emitting unit 112 may indicate that the operator has performed an operation to call a store clerk. The light-emitting unit 112 may also provide information related to operations such as canceling products, reducing the purchase quantity, operating a product preset button, etc. For example, when the number of product cancellations (similar to the number of purchase quantity reductions and the number of product preset button operations) exceeds a predetermined threshold, the light-emitting unit 112 may emit light in a manner that indicates that the number of product cancellations has exceeded the predetermined threshold.

The monitoring display unit 113 is, for example, an LCD display, and displays various information to those around it (the operator, other customers, and store staff). For example, the monitoring display unit 113 may display the number of the device in question while it is on standby (waiting for a customer), and may display images (moving or still images) captured by the imaging unit 110 during processing. The number of the device in question is displayed on both sides and the back of the monitoring display unit 113.

The detection unit 120 detects the presence of a person using a human presence sensor (e.g., a sensor using infrared rays). The detection unit 120 may be equipped with multiple human presence sensors with different detection positions. For example, the detection unit 120 may be composed of a first sensor that detects a person in a direction looking down near the front (front and nearby) of the upstream counter 12 (i.e., detects the presence of a person near the front of the upstream counter 12), a second sensor that detects a person in a direction looking down near the front of the main body unit 11 (i.e., detects the presence of a person near the front of the main body unit 11), and a third sensor that detects a person in a direction looking down near the front of the downstream counter 13 (i.e., detects the presence of a person near the front of the downstream counter 13).

(Operation confirmation verification process)
As described above, the registration and settlement device 10 can execute an operation confirmation and verification process based on the operation of a store clerk. The operation confirmation and verification process is a process for confirming whether a customer's operation (operation in the registration process or the settlement process) is correct (or incorrect), or for verifying whether the customer's operation is correct (or incorrect). Because the registration and settlement device 10 proceeds with the process (registration process, settlement process) based on the customer's operation, i.e., without the intervention of a store clerk, there is a higher possibility of incorrect operations (intentional incorrect operations, incorrect operations due to mistake) compared to when the store clerk is involved. As a countermeasure against incorrect operations, the store (store clerk) executes an operation confirmation and verification process as appropriate to verify whether the customer's operation is correct. Hereinafter, the function realized by (provided by) the operation confirmation and verification process may be referred to as an operation confirmation and verification function (or an operation confirmation function, or an operation verification function).

Specifically, as part of its operation confirmation verification process (operation confirmation verification function), the registration and settlement device 10 displays on the display unit 103 captured images (moving images) captured by the imaging unit 110 and display transition images (moving images) generated based on operation log information. For example, the registration and settlement device 10 displays the captured images and display transition images side by side and in sync on the same screen (e.g., small screen SG40 in FIG. 5(C) etc.) based on a specific operation by the clerk on a specific screen dedicated to the clerk (e.g., small screen SG30 in FIG. 5(B), small screen SG31 in FIG. 7(A), small screen SG32 in FIG. 7(B)). Therefore, the clerk (or, if the clerk is operating in the presence of a customer, the clerk and the customer) can simultaneously view the captured images and the display transition images that are displayed in sync with the captured images.

Figures 3 and 4 are examples of displays on the registration and settlement device 10 (examples of displays on the display unit 103). Specifically, Figures 3 and 4 show examples of displays from the start of product registration to the selection of a payment method. Figure 3 (A) is the transaction start screen (standby screen). Before the transaction begins (while waiting for a customer), the registration and settlement device 10 displays the transaction start screen as shown in Figure 3 (A). A registration start button BT01 for starting a transaction is provided (displayed) on the transaction start screen. In addition, a store clerk call button is provided at the top of the transaction start screen (as are other screens).

As shown in Figure 3(A), the registration start button BT01 is located relatively high on the transaction start screen (closer to the image capture unit 110), so when the customer operates the registration start button BT01, their line of sight is relatively upward. Therefore, when capturing a representative face image, a suitable representative face image (with their line of sight relatively upward) can be obtained.

When the registration start button BT01 on the transaction start screen is operated, the registration and settlement device 10 displays the initial registration screen (the registration screen before registering one product item) as shown in Figure 3 (B). The initial registration screen (and the registration screen as well) has a total amount display area HR10 that displays total information (total number of items purchased, total amount), and a product information display area HR11 that displays information about the registered products. The initial registration screen (and the registration screen as well) also has a checkout button BT12 that is operated to proceed to checkout (settlement) (i.e., to complete registration). Note that the checkout button BT12 becomes operable after registering one product item, and therefore on the initial registration screen, which is the registration screen before registering one product item, the checkout button BT12 is displayed in an inoperable mode (grayed out, indicated by a dashed line in the figure)

The initial registration screen (and the registration screen as well) also has various buttons (vegetable button, fruit button, fresh fish button, meat button) that can be operated when registering products that do not have barcodes. For example, when the vegetable button is operated, the registration and settlement device 10 displays product preset buttons for registering individual vegetables (cucumber, carrot, etc.) (see Figure 4 (A)). The same applies to the fruit button, fresh fish button, and meat button.

When the customer scans the barcode of a can of beer from the scene in Figure 3 (B) (initial registration screen), the registration and settlement device 10 displays a registration screen such as that shown in Figure 3 (C). Product information display area HR11 displays information about the product (can of beer) in question (product name, quantity, price). Additionally, total amount display area HR10 displays the total amount for that product.

The display column for the product (canned beer) in the product information display area HR11 is provided with a plus button (the button marked "+" in the figure) that can be operated to increase the purchase quantity of the product (canned beer), a minus button (the button marked "-" in the figure) that can be operated to decrease the purchase quantity of the product, and a cancel button (the button marked "Cancel" in the figure) that can be operated to cancel the purchase of the product. Note that the minus button is operable when the purchase quantity is 2 or more, and therefore in the scene in Figure 3 (C), the minus button is displayed in a manner that indicates that it cannot be operated.

When the vegetable button is operated from the scene in Figure 3 (C) (registration screen), the registration and settlement device 10 displays a small screen SG13 in front of the registration screen, as shown in Figure 4 (A). The small screen SG13 has product preset buttons (cucumber button, carrot button, etc.) for registering individual vegetables. Although not shown, the small screen SG13 displayed in front of the registration screen when the fruit button is operated from the scene in Figure 3 (C) (registration screen) has product preset buttons (apple button, mandarin orange button, etc.) for registering individual fruits. The same applies when the fresh fish button or meat button is operated.

The small screen SG13 also has a Next Product button that can be operated to display the next product (another product preset button) on the small screen SG13, and a Close button that can be operated to close (erase) the small screen SG13. The registration and settlement device 10 also closes the small screen SG13 when any of the product preset buttons are operated, in addition to the Close button.

When the product preset button for cucumber is operated from the scene in Figure 4 (A) (the registration screen with small screen SG13 displayed in the foreground), the registration and settlement device 10 closes small screen SG13 as shown in Figure 4 (B), adds information about the product (cucumber) to the product information display area HR11 of the registration screen, and updates the total information in the total amount display area HR10.

When the checkout button BT12 is pressed from the scene (registration screen) shown in Figure 4 (B), the registration settlement device 10 displays the payment method selection screen (also referred to as the payment type selection screen or settlement method selection screen) shown in Figure 4 (C). The payment method selection screen has buttons (cash button, credit button, electronic money button) corresponding to each payment method (cash payment, credit payment, electronic money payment).

Although not shown in the figures, the registered settlement device 10 displays (sequentially in response to the customer's operations, etc.) one or more settlement screens (one or more cash payment screens, one or more credit payment screens, one or more electronic money payment screens) according to the payment method selected on the payment method selection screen (cash payment when the cash button is pressed, credit payment when the credit button is pressed, or electronic money payment when the electronic money button is pressed), and proceeds with the settlement process. The registered settlement device 10 prints and issues a receipt as the final operation in the settlement process (or as the final operation of the transaction after the settlement process is completed).

The registration settlement device 10 is capable of reading point cards. There are no particular limitations on the timing of reading the point card before registration begins. For example, the registration settlement device 10 may read the point card before registration begins (from pressing the registration start button BT01 to registering one product item), or may read the point card during product registration (from registering one product item to pressing the checkout button BT12), or may read the point card when selecting a payment method (from pressing the checkout button BT12 to pressing a button on the payment method selection screen).

Figure 5 is an example of a display on the registration and settlement device 10 (an example of a display on the display unit 103). Specifically, Figure 5 shows an example of a display from the captured image to the display of the display transition image.

For example, when a store clerk scans the transaction start screen (FIG. 3(A)) to read the store clerk's store clerk code, the registration settlement device 10 displays small screen SG20 in front of the transaction start screen, as shown in FIG. 5(A). Small screen SG20 is a maintenance screen for the store clerk. Various buttons are provided on small screen SG20. For example, small screen SG20 has a record confirmation button BT21 that can be operated to check records (history).

In addition, a store clerk may monitor a customer who is using a registration and settlement device 10 to register or settle items (either with the naked eye in the checkout area or on a monitor displaying surveillance camera footage), and if they notice anything concerning with the customer's operation, they may move over to that registration and settlement device 10 (where the customer is), and after the transaction is completed (or during the transaction), have that registration and settlement device 10 read the clerk code and display the small screen SG20. Furthermore, if a store clerk finds a registration and settlement device 10 with the light-emitting unit 112 illuminating (i.e., a registration and settlement device 10 currently calling a store clerk, or a registration and settlement device 10 where the number of item cancellations, reductions in purchase quantities, or operation of product preset buttons exceeds a predetermined threshold), they may move over to that registration and settlement device 10, and after the transaction is completed (or during the transaction), have that registration and settlement device 10 read the clerk code and display the small screen SG20.

When the record confirmation button BT21 is operated from the scene shown in Figure 5 (A) (the transaction start screen with small screen SG20 displayed in the foreground), the registered settlement device 10 displays small screen SG30 in the foreground of the transaction start screen in place of small screen SG20, as shown in Figure 5 (B). Small screen SG30 is a transaction confirmation screen (electronic journal search and viewing screen) for the store clerk. Small screen SG30 has a transaction list display area HR31.

The transaction list display area HR31 shown in Figure 5 (B) displays the basic transaction information (described below) for each transaction in a selectable manner. In other words, the registered settlement device 10 accepts touching (selection) of the basic transaction information for that transaction (the display area for that basic transaction information) as a selection operation for that transaction. In other words, when a store clerk selects a transaction, they touch (select) the basic transaction information for that transaction (the display area for that basic transaction information).

(Basic transaction information)
The basic transaction information is basic information about a transaction. The basic transaction information includes, for example, the transaction date and time (e.g., the transaction completion date and time), a receipt number (transaction number), a journal number, and event information within the transaction. The event information within the transaction is information about events that occurred during the transaction, i.e., events recorded in the electronic journal information for the transaction. For example, information about an event indicating that the transaction amount (total amount) reached 10,000 yen (i.e., information indicating that the transaction amount was 10,000 yen or more), information about an event indicating that a store clerk was called (i.e., information indicating that the transaction involved a store clerk call), information about an event indicating that a product was registered using a product preset button (i.e., information indicating that the transaction involved an operation of the product preset button), information about an event indicating that a product was canceled (i.e., information indicating that the transaction involved a cancellation operation (operation of the cancel button)), and information about an event indicating that the purchase quantity was reduced (i.e., information indicating that the transaction involved a reduction operation of the purchase quantity (operation of the minus button)).

For example, "Man" in Figure 5 (B) indicates a transaction with a transaction amount of 10,000 yen or more. "Call" indicates a transaction in which a store clerk was called. "P" indicates a transaction in which a product preset button was operated. "Tori" indicates a transaction in which a cancellation operation was performed. "Decrease" indicates a transaction in which an operation to reduce the purchase quantity was performed.

Furthermore, various buttons are provided on the small screen SG30. For example, the small screen SG30 is provided with a display button BT32 that is operated to display an electronic journal file, an image playback button BT33 that is operated to play (display) a captured image or a display transition image, and an exit button BT34 that is operated to erase the small screen SG30 (displaying the small screen SG20 instead of the small screen SG30).

The end button BT34 of the small screen SG30 may be made inoperable while the small screen SG40 (described below) is displayed. Alternatively, the end button BT34 of the small screen SG30 may be operable even while the small screen SG40 is displayed. If the end button BT34 is operated while the small screen SG40 is displayed, the small screen SG40 may be closed (deleted) and the small screen SG20 may be displayed instead of the small screen SG30. Alternatively, if the end button BT34 is operated while the small screen SG40 is displayed, a confirmation small screen with a confirmation button and a cancel button may be displayed. If the confirmation button is operated, the confirmation small screen and the small screen SG40 may be closed and the small screen SG20 may be displayed instead of the small screen SG30. If the cancel button is operated, the confirmation small screen may simply be closed (i.e., the operation of the end button BT34 may be canceled).

When the display button BT32 is operated following the selection of a transaction in the transaction list display area HR31 (i.e., when a transaction is selected), the registered settlement device 10 displays the electronic journal information for that transaction in the transaction list display area HR31 instead of the basic transaction information.

When the registration settlement device 10 reads the clerk code during a transaction (for example, while the registration screen is displayed), it displays a small screen SG20 (Figure 5(A)) in front of the registration screen. However, if the record confirmation button BT21 is operated on the small screen SG20, the registration settlement device 10 may display electronic journal information for the transaction (electronic journal information up to the middle of the transaction) in the transaction list display area HR31 of the small screen SG30 displayed in place of the small screen SG20. In other words, when the record confirmation button BT21 is operated on the small screen SG20 displayed during a transaction, the registration settlement device 10 may display the electronic journal information for the transaction in progress in the transaction list display area HR31 from the beginning (i.e., omitting the transaction selection operation or the operation of the display button BT32).

Furthermore, when the image playback button BT33 is operated following the selection of a transaction in the transaction list display area HR31, the registration settlement device 10 displays a captured image and a display transition image of that transaction. Note that the registration settlement device 10 may also display a captured image and a display transition image of that transaction when the image playback button BT33 is operated while the electronic journal information of a transaction (including electronic journal information of a transaction currently being traded) is displayed in the transaction list display area HR31.

When a transaction (e.g., a transaction shown in Figures 3 and 4) is selected in the transaction list display area HR31 from the scene shown in Figure 5 (B) (the transaction start screen with small screen SG30 displayed in the foreground), followed by operation of the image playback button BT33, the registered settlement device 10 displays small screen SG40 in the foreground of the transaction start screen, as shown in Figure 5 (C).

When the image playback button BT33 on small screen SG30 is operated, the registration and settlement device 10 plays back video from the start (beginning) of the transaction on small screen SG40. Similarly, when small screen SG40 is displayed from small screen SG31 (FIG. 7(A)) or small screen SG32 (FIG. 7(B)), video is also played back from the start of the transaction. In other words, the video on small screen SG40 in FIG. 5(C) does not show the content played back immediately after the image playback button BT33 on small screen SG30 is operated (i.e., when the display of small screen SG40 begins), but rather shows the point at which playback has progressed since the start of display (or after the playback point is specified after display begins).

When the image playback button BT33 on small screen SG30 is operated, the registered settlement device 10 may refer to the basic transaction information (such as "P") of the transaction and play (start playback) from a specific point (time point) on small screen SG40, as shown in FIG. 5(C). Similarly, playback from a specific point may also be performed when displaying small screen SG40 from small screen SG31 (FIG. 7(A)) or small screen SG32 (FIG. 7(B)).

Figure 6 shows an example of a display on the registration and settlement device 10 (an example of a display on the small screen SG40). Specifically, Figure 6 (A) is an enlarged view of the small screen SG40 shown in Figure 5 (A). The small screen SG40 is an image confirmation screen on which an image is played back and confirmed.

As shown in FIG. 6(A), the registration and settlement device 10 displays (plays) a captured image (symbol A in the same figure) and a display transition image (symbol B in the same figure) side by side on the small screen SG40. The captured image in FIG. 6(A) is an image captured by the imaging unit 110 when two products are registered, and the display transition image in FIG. 6(A) is a display transition image when two products are registered. In other words, the registration and settlement device 10 displays a captured image and a display transition image side by side at the same time (the time when two products are registered in FIG. 6(A)).

Image GA1 in the display transition image is an image that shows the customer's operation position (touch position). When the touch position (coordinates) is included in the operation log information, the registration and settlement device 10 generates a display transition image that visualizes the touch position using image GA1 (displaying image GA1 at the touch position). Note that the registration and settlement device 10 may display image GA1 so that the center of image GA1 is the touch position (in other words, image GA1 may be displayed with the touch position at its center). Note that while image GA1 in the figure is simply a circular image, image GA1 may also be an image that evokes a touch (for example, a fingerprint-like image or a fingertip-like image).

From Figure 6(A), it can be seen that when carrots (¥70) should have been registered, cucumbers (¥50) were actually registered. It can also be seen that the customer operated (touched) the upper right portion of the cucumber product preset button. Note that because the display transition image in Figure 6(A) shows (is being played back) the moment when the customer picked up a carrot and operated the cucumber product preset button (upper right portion) on the small screen SG13, the registered information for cucumbers is not reflected (displayed) in the display transition image (product information display area HR11 of the registration screen displayed as the display transition image), but in the display transition image immediately afterwards (not shown), the small screen SG13 is erased and the registered information for cucumbers is reflected.

The small screen SG40 also has a playback bar (symbol C in the same figure) that indicates the playback location (playback time) of the video (captured image, display transition image). When a location on the playback bar is operated (touched), the registration and settlement device 10 plays the video from that location (i.e., changes the playback location). In other words, the store clerk can check the playback location of the video on the playback bar, and can also operate the desired location on the playback bar to change (adjust) the playback location.

The bottom of the playback bar displays the location (time) of the event that occurred in the transaction (start, register 1st item, register 2nd item (P), cash selection, etc.). The "S" in "Register 1st Item S" indicates that the registration was made by scanning a barcode. In other words, "Register 1st Item S" indicates that the first item was registered by scanning a barcode. The "P" in "Register 2nd Item P" indicates that the registration was made by pressing a product preset button. In other words, "Register 2nd Item P" indicates that the second item was registered by pressing a product preset button. The information below the playback bar is useful when checking or changing the playback location. The events shown in Figure 6 (A) are just an example, and other events (e.g., cancellation, purchase quantity reduction, etc.) may also be displayed. The location of the event may be displayed above the playback bar instead of or in addition to the bottom of the playback bar.

In addition to or instead of displaying the location of an event in association with the playback bar (displaying the location of the event at the bottom or top of the playback bar), buttons corresponding to each event (event-related buttons) may be provided. In the example shown in FIG. 6(A), the event-related buttons may be arranged on the small screen SG40 as a first-item registration S button, a second-item registration P button, a cash selection button, and a cash (300 yen) insertion button. When a button is operated, the registration and settlement device 10 starts playing the video from the corresponding location. For example, when the second-item registration P button is operated, the registration and settlement device 10 starts playing the video from the location where the second item was registered using the product preset button.

The small screen SG40 is also provided with a play (forward) button, stop button, fast-forward button, rewind button, and fast-rewind button. The play button is operated to play and display the moving image at a normal speed (the speed at which real time passes). The stop button is operated to freeze the moving image (display it as a still image). The fast-forward button is operated to play and display the moving image at a faster-than-normal speed. The rewind button is operated to play and display the moving image in reverse at a normal speed. The fast-rewind button is operated to play and display the moving image in reverse at a faster-than-normal speed. Note that when the image play button BT33 on the small screen SG30 is operated (i.e., when the display of the small screen SG40 starts), the registration and settlement device 10 may play and display the moving image from the beginning (or a specific point) at the normal speed, or may freeze and display the beginning (or a specific point) of the moving image.

The small screen SG40 also has a simultaneous screen (simultaneous image) button BT41, a monitoring screen (captured image) button BT42, an operation screen (display transition image) button BT43, and a close button BT44. The close button BT44 is a button that is operated to close (delete) the small screen SG40.

The simultaneous screen button BT41, monitoring screen button BT42, and operation screen button BT43 are buttons that accept image display switching operations and notify the current display state. The simultaneous screen button BT41 is a button that is operated when simultaneously displaying the captured image and the display transition image. The monitoring screen button BT42 is a button that is operated when displaying the captured image without displaying the display transition image. The operation screen button BT43 is a button that is operated when displaying the display transition image without displaying the captured image. For example, if the monitoring screen button BT42 is operated from the scene in Figure 6 (A), the registration and settlement device 10 displays the captured image without displaying the display transition image (not shown). If the operation screen button BT43 is operated from the scene in Figure 6 (A), the registration and settlement device 10 displays the display transition image without displaying the captured image, as shown in Figure 6 (B). Switching is possible even during playback.

In the scene in FIG. 6(A), the simultaneous screen button BT41 is selected, and therefore in FIG. 6(A), the simultaneous screen button BT41 is displayed in a manner indicating that it is selected (with a bold frame in the figure). In the scene in FIG. 6(B), the operation screen button BT43 is selected, and therefore in FIG. 6(B), the operation screen button BT43 is displayed in a manner indicating that it is selected.

A mode in which both captured images and display transition images are played back (displayed) may be referred to as a simultaneous playback mode, a mode in which captured images are played back without playing display transition images may be referred to as a captured image playback mode, and a mode in which display transition images are played back without playing captured images may be referred to as a display transition image playback mode. In other words, the simultaneous screen button BT41, monitoring screen button BT42, and operation screen button BT43 can also be considered mode switching buttons that switch between simultaneous playback mode, captured image playback mode, and display transition image playback mode.

(Other screens)
The registration and settlement device 10 may display a small screen SG31 as shown in FIG. 7(A) instead of the small screen SG30 (FIG. 5(B)). That is, when the record confirmation button BT21 on the small screen SG20 (FIG. 5(A)) is operated from the scene of FIG. 5(A), the registration and settlement device 10 may display the small screen SG31 in front of the transaction start screen instead of the small screen SG20, as shown in FIG. 7(A). The small screen SG31 is a screen for specifying a receipt number (transaction number) to identify a transaction. When an operation to specify a receipt number is performed on the small screen SG31 (that is, when a transaction is identified), the registration and settlement device 10 displays a small screen SG40 above the small screen SG31, and displays an image of the transaction (captured image, display transition image) on the small screen SG40.

The registration and settlement device 10 may display small screen SG32 instead of small screen SG30 (Figure 5(B)) or small screen SG31 (Figure 7(A)). In other words, when the record confirmation button BT21 on small screen SG20 (Figure 5(A)) is operated from the scene of Figure 5(A), the registration and settlement device 10 may display small screen SG32 in front of the transaction start screen instead of small screen SG20. Small screen SG32 is a screen for extracting one or more transactions (basic transaction information) by specifying a time. When an operation to specify a time is performed on small screen SG32, the registration and settlement device 10 displays one or more basic transaction information within the specified time range in a selectable manner. The small screen SG32 shown in Figure 7(B) is not the small screen SG32 immediately after it is displayed (immediately after the record confirmation button BT21 is operated), but is the small screen SG32 displaying one or more basic transaction information in a selectable manner. When a transaction (basic transaction information) is selected on small screen SG32 and the image playback button BT33 is operated, the registration settlement device 10 displays small screen SG40 at the top of small screen SG32 and displays an image of the transaction (captured image, display transition image) on small screen SG40. Note that when a transaction (basic transaction information) is selected on small screen SG32 and the display button BT32 is operated, the registration settlement device 10 erases the displayed basic transaction information and displays the electronic journal information for that transaction.

The small screen SG30 (FIG. 5(B)), the small screen SG31 (FIG. 7(A)), and the small screen SG32 (FIG. 7(B)) may be switchable. For example, a button for switching to another screen may be provided on each of the small screens SG30, SG31, and SG32.

The above describes an embodiment. The registration and settlement device 10 of the above embodiment allows for convenient confirmation of operation status. In other words, the operation status confirmation process can be carried out efficiently.

In the past, operational errors and incorrect deposits (wrong currency) were confirmed using images captured by surveillance cameras, but it could take time to understand the situation or not be possible to accurately grasp the situation using only images captured by surveillance cameras. However, with the registration and settlement device 10 of the above embodiment, the captured image and the display transition image (display history image) are synchronized and displayed (played back) on the same screen, allowing for a simple (quick) and accurate understanding of the situation. Furthermore, because the registration and settlement device 10 notifies the touch position in the display transition image, when an operational error is discovered, it is possible to infer to some extent whether it was a simple operational error or an intentional operation (an operation that may be suspected of being fraudulent).

Therefore, for example, if a store clerk and a customer check together and the check finds that there is no problem on the customer's side, the store (clerk) can accept the situation and take appropriate action (for example, by apologizing). On the other hand, if the check finds that there is a problem on the customer's side, corrections and other measures can be taken smoothly. In other words, it is less likely that a situation will arise in which action is taken without alleviating the customer's distrust (or leaving room for the customer to object), and work can proceed smoothly. In other words, regardless of the check result, the high degree of reproducibility of the situation gives a sense of acceptance to both the store clerk and the customer, allowing work to proceed smoothly.

The registered settlement device 10 can also effectively deal with pressing the wrong product preset button (intentionally pressing the button for a product cheaper than the product being purchased), deposit problems (for example, mistakenly thinking (or intentionally claiming) that a 10,000 yen bill was inserted when in fact a 5,000 yen bill was inserted, and then complaining about insufficient change), and the taking away of unregistered products (see below).

(Taking away unregistered products)
An act of taking away unregistered merchandise (also referred to as an act of taking away unpaid merchandise) is an act of taking away merchandise without settling (i.e., without paying for it). Examples of acts of taking away unregistered merchandise include an act of moving an item from the upstream counter 12 to the downstream counter 13 without scanning (putting an item from a shopping basket into a shopping bag without even pretending to scan), an act of moving an item from the upstream counter 12 to the downstream counter 13 by intentionally performing an insufficient (inappropriate) scan to prevent the barcode of the item from being read (putting an item from a shopping basket into a shopping bag while disguising the scan), an act of moving a set of items from the upstream counter 12 to the downstream counter 13 by unintentionally or intentionally reading the barcode of a single item (e.g., a single can of beer) instead of the barcode of the set of items (e.g., a six-pack of canned beer), and an act of moving an item from the upstream counter 12 to the downstream counter 13 after an unintentional failed scan without realizing it (a simple scan error).

By understanding the movement of goods (for example, by understanding the movement of goods from image recognition of images captured by the imaging unit 110 or the detection results of the detection unit 112) and the registration status before and after the movement of goods (a situation in which goods have not been added), it is generally possible to recognize the occurrence of an unregistered product being taken home, but by playing back and checking the captured images and display transition images, it is possible to recognize the occurrence more accurately and easily.

(Modifications, etc.)
Although the embodiments have been described above, the device configuration, data configuration, processing flow, display and output modes, etc. can be appropriately changed as shown in the following (1) to (18). In addition, the following (1) to (18) can be appropriately combined.

(1) The size of image GA1 (circular image, fingerprint-like image, fingertip-like image, etc.) may be changeable whether the display transition image is being played back or is paused. For example, an enlargement button and a reduction button may be arranged on small screen SG40, and the registration settlement device 10 may enlarge image GA1 around the touch position when the enlargement button is operated, and reduce image GA1 around the touch position when the reduction button is operated. If the display size of image GA1 is large, image GA1 stands out even during playback display, making it possible to immediately recognize that a touch has occurred even during playback display. On the other hand, if the display size of image GA1 is small, the touch position can be accurately confirmed even when various buttons are densely arranged. In addition, the initial display size of image GA1 (the display size before the enlargement button or reduction button is operated) may be set.

(2) Coordinate information (X coordinate value, Y coordinate value) of the touch position may be displayed in the display transition image. For example, the registration settlement device 10 may display the coordinate information of the touch position near (above or below) the image GA1 showing the touch position. This gives the customer the impression that the touch operation has been properly memorized (recorded).

(3) The display/non-display of image GA1 may be switchable. For example, a display (ON) button and a non-display (OFF) button may be placed on the small screen SG40, and the registration and settlement device 10 may display image GA1 when the display button is operated, and may not display image GA1 when the non-display button is operated. Similarly, the display/non-display of the coordinate information in (2) above may be switchable.

(4) The touch position may be immediately notified when a touch operation is performed. For example, a button that can be switched to a display mode that immediately notifies the touch position (touch position immediate display mode) may be provided on the small screen SG40, and the registration and settlement device 10 may switch to the touch position immediate display mode when the button is operated. In the touch position immediate display mode, the registration and settlement device 10 displays a mock screen for operation confirmation (e.g., a mock registration screen, a mock settlement screen, etc.), and when a touch operation is performed on the mock screen, image GA1 may be displayed at the touch position at the time of the touch operation (and while the touch is being performed). For example, if necessary (e.g., if the customer is not satisfied with the touch position), the store clerk may switch to the touch position immediate display mode, ask the customer to touch any position on the mock screen, and indicate that image GA1 will be displayed at the actual touch position. This allows for smooth handling.

(5) The transaction selection screen (small screen SG30, small screen SG31, small screen SG32) may be erased, and a screen (small screen SG40) displaying a moving image (captured image, display transition image) may be displayed. In other words, in the above embodiment, when a transaction is selected on the transaction selection screen, the registration settlement device 10 maintains the display of the transaction selection screen and displays a new screen displaying a moving image of the transaction. However, when a transaction is selected on the transaction selection screen, the registration settlement device 10 may erase the transaction selection screen and display a screen displaying a moving image of the transaction.

(6) The screen for selecting a transaction (small screen SG30, small screen SG31, small screen SG32) and the screen for displaying a moving image (small screen SG40) may be a common screen. In other words, in the above embodiment, when a transaction is selected on the screen for selecting a transaction, the registration settlement device 10 displays a moving image of that transaction on a screen (small screen SG40) different from the screen for selecting the transaction (small screen SG30, small screen SG31, small screen SG32). However, when a transaction is selected on the screen for selecting a transaction, the registration settlement device 10 may display a moving image of that transaction on the screen for selecting the transaction.

(7) Product preset buttons may be displayed on the normal screen. In other words, in the above embodiment, an example was described in which the registration and settlement device 10 displays a small screen SG13 on which product preset buttons for individual vegetables are arranged in front of the registration screen when the vegetable button on the registration screen is operated. However, when the vegetable button on the registration screen is operated, the registration and settlement device 10 may display a screen (normal screen) on which product preset buttons for individual vegetables are arranged instead of the registration screen. This makes it possible to arrange the product preset buttons at a certain distance from each other, reducing the chance of touching the product preset buttons incorrectly or making it more difficult to claim that a touch was incorrect. Note that when a product preset button or the back button is operated, the user simply returns to the registration screen.

(8) A screen where a store clerk intervenes may be displayed as a normal screen. In other words, in the above embodiment, an example was described in which the registration and settlement device 10 displayed a screen where a store clerk intervenes on a small screen (e.g., small screen SG20, small screen SG30, small screen SG31, small screen SG32, small screen SG40), but it may also be displayed on a normal screen. For example, the registration and settlement device 10 may display a moving image (captured image, display transition image) on a normal screen.

(9) The captured image may be a moving image that includes audio. The same applies to the display transition image. Audio-related buttons (e.g., an audio (ON) button, a no audio (OFF) button, and a volume button) may be placed on the small screen SG40. Note that when a captured image that includes audio and a display transition image that includes audio are played simultaneously, the audio for one of the images may be muted (or the audio output may be significantly reduced compared to the other).

(10) The display transition image may be a moving image obtained by recording (storing) the display content on the display unit 103. In other words, in the above embodiment, it was explained that the registration and settlement device 10 generates the display transition image based on operation log information, but the display transition image may also be generated by recording the display content on the display unit 103 (in other words, the display content may be recorded and used as the display transition image).

When the display transition image is a moving image generated by recording the display content, image GA1 is displayed in the display transition image, just as when the display transition image is a moving image generated based on operation log information. For example, the registration and settlement device 10 may superimpose image GA1 based on operation log information onto a recorded moving image of the display content on the display unit 103, store it as a display transition image, and display it, or may store a recorded moving image of the display content on the display unit 103 as a display transition image and display the stored display transition image with image GA1 based on the operation log information superimposed onto it. In other words, the display transition image may be a moving image of recorded display content with image GA1 superimposed on it, or a moving image of recorded display content (a moving image without image GA1 superimposed on it) may be referred to as a display transition image.

(11) The registered settlement device 10 does not need to have either the non-cash payment unit (payment terminal) 105 or the cash payment unit 106 (it is sufficient to have either one).

(12) Moving images (captured images, display transition images) may be stored in another device. In other words, in the above embodiment, an example was described in which a registered settlement device 10 stores its own moving images (moving images related to a transaction (customer) in its own registered settlement device 10) in the memory unit 102 of its own registered settlement device 10. However, a registered settlement device 10 may store its own moving images in another device (e.g., a transaction status management device 40, a store controller 50, another registered settlement device 10 (a representative registered settlement device 10), or a headquarters server 60). For example, a registered settlement device 10 that generates a moving image may transmit the generated moving image to another device, and the other device that receives the moving image may store the moving image, or the registered settlement device 10 that generates the moving image may access the other device and store the generated moving image in the memory unit of the other device.

(12-1) If there is only one registered settlement device 10 in the sales system 1 (i.e., if there is only one device to be monitored (device that generates video images)), the video images are stored on another device so that it is clear which transaction they represent.

(Images are stored in other devices/One device is monitored)
The registered settlement device 10 transmits a captured image of a certain transaction together with the transaction identification information of the transaction to another device, and the receiving device stores the captured image in association with the transaction identification information. Alternatively, the registered settlement device 10 accesses the other device and stores the captured image of a certain transaction in a memory unit of the other device in association with the transaction identification information of the transaction.

In addition, the information name of the captured image (data name, file name) or the storage location of the captured image (another device, folder, path) may be different (separate) for each transaction, making it possible to identify which transaction the captured image is for.

(Display transition images are stored on other devices/monitoring target device is 1)
As explained above, the registration and settlement device 10 may generate a display transition image that includes a transaction identification number, or may generate a display transition image that does not include transaction identification information. In the former case (when the display transition image includes transaction identification information), the registration and settlement device 10 simply transmits the display transition image of a certain transaction to another device, and the other device that is the receiving side simply stores the display transition image. Alternatively, the registration and settlement device 10 accesses the other device and simply stores the display transition image of a certain transaction in the memory unit of the other device.

On the other hand, in the latter case (when the display transition image does not include transaction identification information), the registered settlement device 10 transmits the display transition image of a certain transaction along with the transaction identification information of that transaction to another device, and the receiving device stores the display transition image in association with the transaction identification information. Alternatively, the registered settlement device 10 accesses the other device and stores the display transition image of a certain transaction in the memory of the other device in association with the transaction identification information of that transaction. Note that the information name of the display transition image or the storage location of the display transition image may be different for each transaction, so that it is possible to identify which transaction the display transition image is for.

(12-2) If there are two or more registered settlement devices 10 in the sales system 1 (i.e., if there are two or more devices to be monitored), the video image is stored in another device so that it is clear which registered settlement device 10 and which transaction it is for.

(Images are stored in another device/there are two or more devices to be monitored)
The registered settlement device 10 transmits a captured image of a certain transaction to another device along with device identification information that identifies the registered settlement device 10 and the transaction identification information of the transaction, and the other device that receives the image stores the captured image in association with the device identification information and the transaction identification information. Alternatively, the registered settlement device 10 accesses the other device and stores the captured image in the memory of the other device in association with the device identification information of the registered settlement device 10 and the transaction identification information of the transaction. Note that the information name of the captured image or the storage location of the captured image may be different for each registered settlement device 10 and each transaction, so that it is possible to identify which registered settlement device 10 and which transaction the captured image is for.

(Save the display transition image to another device/monitor two or more devices)
If the display transition image includes transaction identification information, the registered settlement device 10 transmits the display transition image of a certain transaction to another device together with device identification information that identifies the registered settlement device 10, and the other receiving device stores the display transition image in association with the device identification information. Alternatively, the registered settlement device 10 accesses the other device and stores the display transition image in the memory of the other device in association with the device identification information of its own device. Note that the information name of the device identification information or the storage location of the device identification information may be different for each registered settlement device 10, so that it is possible to identify which registered settlement device 10 the device identification information belongs to.

On the other hand, if the display transition image does not include transaction identification information, the registered settlement device 10 transmits the display transition image for a certain transaction to another device along with device identification information that identifies the registered settlement device 10 and the transaction identification information for that transaction, and the other receiving device stores the display transition image in association with the device identification information and transaction identification information. Alternatively, the registered settlement device 10 accesses the other device and stores the display transition image for a certain transaction in the memory of the other device in association with its own device's device identification information and the transaction identification information for that transaction. Note that the information name of the display transition image or the storage location of the display transition image may be different for each registered settlement device 10 and each transaction, so that it is possible to identify which registered settlement device 10 and which transaction the display transition image belongs to.

Note that the captured image may include both device identification information and transaction identification information, may include transaction identification information but not device identification information, or may include device identification information but not transaction identification information. For captured images that include both device identification information and transaction identification information, both the device and the transaction may be identified from the captured image itself (neither device identification information nor transaction identification information need to be stored in association). For captured images that do not include device identification information but include transaction identification information, the transaction may be identified from the captured image itself (neither device identification information nor transaction identification information need to be stored in association with transaction identification information). For captured images that do not include transaction identification information but include device identification information, the device may be identified from the captured image itself (neither device identification information need to be stored in association with device identification information). The same applies to display transition images.

(13) A moving image from a certain registered settlement device 10 may be displayed (played back) on a device other than the registered settlement device 10 (for example, a transaction status management device 40, another registered settlement device 10). Note that while a moving image may be transmitted when it is stored (see (12) above), even if the device that displays the moving image and the device that stores the moving image are different, the moving image may still be transmitted when it is displayed.

(13-1) If there is only one registered settlement device 10 in the sales system 1 (i.e., if there is only one device to be monitored), a transaction is specified and a moving image is displayed. For example, a screen (e.g., small screen SG30, small screen SG31, small screen SG32) on which a transaction can be specified (i.e., a transaction identification number can be identified) may be displayed, and the specified moving image may be displayed on that screen. The screen on which the above-mentioned transaction can be specified may be equipped with a search function or a list display function, like small screen SG30 or small screen SG32.

(13-2) If there are two or more registered settlement devices 10 in the sales system 1 (i.e., if there are two or more devices to be monitored), a device and transaction can be specified and a video image can be displayed. For example, a screen can be displayed on which a device and transaction can be specified (i.e., device identification information and transaction identification number can be specified), and the specified video image can be displayed on that screen. The screen on which the transaction can be specified may also have a search function and a list display function.

(14) A device other than the registration and settlement device 10 may be monitored. In other words, in the above embodiment, an example was described in which moving images (captured images, display transition images) of the registration and settlement device 10, which is capable of executing the registration process and the settlement process, were acquired. However, moving images of other devices may also be acquired. For example, moving images of the settlement device 230 (not shown), the settlement device 250 (not shown), or the charging device 260 may be acquired. A label issuing device 270 (not shown) that selects products and prints and issues labels may be installed in the store, and moving images of the label issuing device 270 may be acquired. Furthermore, a weighing device 280 (not shown) that weighs products may be installed in the store, and moving images of the weighing device 280 may be acquired. Note that the label issuing device 270 may have an object scanning function and may display (or identify) candidate products by object scanning in a selectable manner. The registration and settlement device 10, the settlement device 230, the settlement device 250, and the weighing device 280 may also have an object scanning function. The weighing device 280 may also have a label printing function. A ticket vending machine 290 (not shown) may also be installed inside the store, and video images of the ticket vending machine 290 may be acquired. The ticket vending machine 290 may be installed in a restaurant or the like. Self-service payment devices installed in restaurants or the like may also be monitored (video images of self-service payment devices installed in restaurants or the like may be acquired).

Related to (12) to (14) above, the device that generates the video (the device to be monitored), the device that stores the video, and the device that displays (plays back) the video may all be the same device or may all be different devices. For example, video from device A (e.g., registration and settlement device 10, settlement device 230, settlement device 250, charging device 260, label issuing device 270, weighing device 280, ticket vending machine 290, etc.) may be stored in device A and displayed on device A or device B (e.g., transaction status management device 40). Alternatively, video from device A may be stored in device B and displayed on device B or device C (e.g., registration and settlement device 10, settlement device 230, settlement device 250, charging device 260, label issuing device 270, weighing device 280, ticket vending machine 290, etc., other than device A). When there are two or more devices to be monitored, the device that displays the video may accept designation of the device and transaction and display the designated video from the designated device.

(15) Video images from other imaging units (cameras) may be displayed (played back). In other words, while the above embodiment describes an example in which an image captured by imaging unit 110 is displayed, images captured by other imaging units (e.g., imaging unit 1110, imaging unit 2110, imaging unit 3110) may also be displayed. A single image selected from images captured by multiple imaging units (e.g., imaging unit 110, imaging unit 1110, imaging unit 2110, imaging unit 3110) may be displayed, or multiple display areas for captured images may be provided to simultaneously display multiple captured images. In other words, one or more captured images captured in each area (an area in which a customer can be captured, an area in which customer operations (scanning operation, screen operation, deposit operation, product movement) can be captured, an area in which products on the upstream side (before movement) can be captured, and an area in which products on the downstream side (after movement) can be captured) and a display transition image may be displayed. Note that a selection button (or a switching button) may be provided to accept a selection (switching) operation between displaying one captured image and displaying multiple captured images. Furthermore, when displaying one captured image, a button may be provided that allows the user to select (switch) the captured image to be displayed. The same applies when displaying multiple captured images (for example, when displaying two out of three). Furthermore, the captured images to be displayed (number, type) may be automatically switched not by operation but by settings (by setting in advance for each event a camera image with a suitable shooting accuracy according to the event), or by using AI to automatically switch.

Images acquired from an external source may also be displayed. For example, at least one of the imaging units 110, 1110, 2110, and 3110 may not be an imaging unit provided in the registration settlement device 10, but may be an imaging unit separate from the registration settlement device 10 (a camera installed on the ceiling, wall, or pillar).

(16) Still images may be used for confirmation instead of moving images. In other words, in the above embodiment, an example was described in which captured images and display transition images were displayed as moving images on the image display screen (small screen SG40), but captured images and display transition images may also be displayed as still images. As with moving images, both captured still images and display transition images of still images may be displayed (simultaneous playback mode), or either one may be displayed (captured image playback mode, display transition image playback mode).

Devices that display still images (such as the registration and settlement device 10 and the transaction status management device 40) may display still images (captured images, display transition images) at the time an event occurs. For example, a device that has a next button and a previous button on the image display screen and displays still images may display a still image of the next event occurring in response to the operation of the next button, and a still image of the previous event occurring in response to the operation of the previous button. Furthermore, a device that has event-related buttons on the image display screen and displays still images may display a still image of the event occurring corresponding to the event-related button that was operated in response to the operation of the event-related button.

Note that the display of video may also be controlled by the next button or previous button. In other words, a device that displays video may display a video of the next event occurring (skip to the next event) in response to operation of the next button, and display a video of the previous event occurring (skip to the previous event (backskip)) in response to operation of the previous button.

(Generation of still images)
The monitored devices (registered settlement devices 10, settlement devices 230, 250, charging devices 260, label issuing devices 270, weighing devices 280, ticket vending machines 290, etc.) may generate still images from the moving images after generating moving images, or may generate still images from the beginning without generating moving images. The same applies to display transition images.

In the former case (when a captured image of an image is generated and then a captured image of a still image is generated from the captured image of a moving image), a still image at the time when the event occurred may be extracted from the moving image. The same applies to the display transition image. The time when the event occurred may also be identified by referring to electronic journal information.

In the latter case (when capturing images as still images from the beginning without generating moving images), still images may be generated continuously and a still image at the time of event occurrence may be obtained (still images at other times may be discarded). For display transition images, a still image at the time of event occurrence may be generated by referencing operation log information, etc. Note that the time of event occurrence may be identified by recognizing the occurrence of the event itself, or by referencing electronic journal information.

(Moving image mode, still image mode)
The display device may have a moving image mode in which the captured image and the display transition image are confirmed (displayed) as moving images, and a moving image mode in which the captured image and the display transition image are confirmed as still images, and these modes may be switchable by a button, etc. Also, one of the captured image and the display transition image may be confirmed as a moving image, and the other may be confirmed as a moving image.

(17) Although not explained above, events to be stored in the electronic journal information, events to be included in the basic transaction information (events displayed as "ten thousand" or "P" in the transaction list display area HR31), and events to be used as headings when displaying captured images or display transition images (events displayed below (or above) the playback bar, events arranged as event-related buttons) are set in advance. That is, N events (event 1, event 2, ..., event N) are set as events to be stored in the electronic journal information, M _A events of the N events are set as events to be included in the basic transaction information, and M _B events of the N events are set as headings. M _A and M _B may be the same number or different numbers. Furthermore, even if M _A and M _B are the same number, the set events may be different.

The above settings may be changeable as needed (for example, reconfigurable from a maintenance screen). In other words, the events stored as electronic journal information, the events included in basic transaction information, and the events used as headings may be changeable as needed.

(18) Some or all of the functions (input/output, storage, processing (including judgment)) of each device (registered settlement device 10, transaction status management device 40, settlement device 230, settlement device 250, charging device 260, label issuing device 270, weighing device 280, ticket vending machine 290, etc.) may be realized by a device other than the device described as the entity that executes the function. Each device may function as a so-called thin client specialized for input/output interfaces, etc. For example, a cloud server may execute a program to control the operation of each device, and each device may display a screen using a browsing function, etc.

<Summary of the embodiment>
[Technical Field]
The present invention relates to an information processing device and a program.
[Background technology]
A cash register is known that displays a video of the operation status captured by a camera as a function for checking (inspecting) whether or not an unauthorized operation has been performed (see, for example, Patent Document 1).
[Prior art documents]
[Patent Documents]
[Patent Document 1] JP 2018-120635 A [Summary of the Invention]
[Problem to be solved by the invention]
However, there is room for improvement in the method of checking the operation status using the conventional technology as shown in Patent Document 1. The present invention has been made in consideration of such circumstances, and has an object to provide a technology for conveniently checking the operation status.
[Means for solving the problems]

(1) One aspect of solving the above-described problem is an information processing device (e.g., the registration settlement device 10, the settlement device 230, the settlement device 250, the charging device 260, the label issuing device 270, the weighing device 280, the ticket vending machine 290) that progresses processing based on the operation of a customer, and includes imaging means (e.g., the imaging unit 110, the imaging unit 210, the imaging unit 220) that images predetermined areas (e.g., an area in which an image of the customer can be captured, an area in which an image of the customer's operation can be captured, an area in which an image of the product on the upstream side (before movement) can be captured, an area in which an image of the product on the downstream side (after movement) can be captured). The information processing device is characterized in that it comprises an image capturing unit (image capturing unit 1110, image capturing unit 2110, image capturing unit 3110), and a display means (e.g., display unit 103) that can accept touch operations and displays information according to the progress of processing, and the display means displays an image (moving image or still image) captured by the image capturing unit and a display transition image (moving image or still image) showing the transition of the display content by the display means on the same screen, with the time of capture and the time of display coinciding (e.g., see Figure 5 (C)).
According to the information processing device of (1), it is possible to conveniently check the operation status.

(2) An information processing device as described in (1) is provided with a log information acquisition means for acquiring operation log information (for example, generation of operation log information by CPU 101), and the display means displays, as the display transition image, an image generated based on the log information acquired by the log information acquisition means.
According to the information processing device of (2), it is possible to appropriately generate a display transition image based on an operation while suppressing the capacity.

(3) The information processing device described in (2) is characterized in that the log information acquisition means acquires log information of operations including touch operations, and the display means displays the display transition image that notifies the position of the touch operation (for example, by image GA1).
According to the information processing device of (3), it is possible to easily check which position has been operated.

(4) The information processing device described in (1) is characterized in that it is provided with a log information acquisition means for acquiring log information of operations including touch operations, and the display means displays touch position information indicating the position of the touch operation acquired by the log information acquisition means by superimposing it on the display transition image.
According to the information processing device of (4), it is possible to easily check which position has been operated.

(5) An information processing device according to any one of (2) to (4), characterized in that the captured image and the display transition image are displayed at the time of occurrence of an event that occurs between the start and end of processing and is specified by operation (for example, an event specified by an event-corresponding button, the next event specified by a next button, or the previous event specified by a previous button).
According to the information processing device of (5), it is possible to conveniently check the necessary parts.

(6) An information processing device described in any of (1) to (4), characterized in that it is provided with a switching means for switching between a first mode (simultaneous playback mode) in which the display means displays the captured image and the display transition image, a second mode (captured image playback mode) in which the display means displays the captured image without displaying the display transition image, and a third mode (display transition image playback mode) in which the display means displays the display transition image without displaying the captured image.
According to the information processing device of (6), the display can be switched appropriately depending on the confirmation content.

(7) An information processing device according to any one of (1) to (4), characterized in that it is provided with a transmission means for transmitting the captured image and the display transition image (transmitting when storing in another device, transmitting when displaying on another device).
The information processing device of (7) can be flexibly operated.

(8) One aspect for solving the above-mentioned problem is a program for causing an information processing device, which has a display unit capable of accepting touch operations and progresses processing based on customer operations, to function as a computer, wherein the computer functions as an image capture means for capturing an image of a predetermined area and a display control means for controlling the display of the display unit, and the display control means displays the image capture and a display transition image showing the transition of the display content of the display unit on the same screen by matching the time of capture and the time of display.
According to the program (8), the operating status can be conveniently confirmed.

The programs for implementing the functions of each device described above (such as the registration and settlement device 10 and the transaction status management device 40) may be recorded on a computer-readable recording medium, and the programs may be loaded into a computer system and executed to perform the processing of each device. Here, "loading a program recorded on a recording medium into a computer system and executing it" includes installing the program on a computer system. The term "computer system" here includes hardware such as an operating system and peripheral devices. The term "computer system" may also include multiple computer devices connected via a network, including the Internet, a WAN, a LAN, a dedicated line, or other communication lines. The term "computer-readable recording medium" refers to portable media such as floppy disks, optical magnetic disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into a computer system. Thus, recording media storing programs may be non-transitory recording media such as CD-ROMs. Recording media also include internal or external recording media accessible from a distribution server for distributing the programs. The program code stored on the distribution server's recording medium may be different from the program code executable on a terminal device. In other words, the format in which the program is stored on the distribution server does not matter as long as it can be downloaded from the distribution server and installed in a form that is executable on a terminal device. The program may be divided into multiple parts that are downloaded at different times and then combined on the terminal device, or each of the divided programs may be distributed by a different distribution server. Furthermore, the term "computer-readable recording medium" also includes a storage medium that stores a program for a certain period of time, such as volatile memory (RAM) within a computer system that acts as a server or client when a program is transmitted over a network. The program may also be a program that realizes some of the functions described above. Furthermore, the program may be a so-called differential file (differential program) that can realize the functions described above in combination with a program already stored on the computer system.

1...Sales system, 10...Registration and settlement device, 40...Transaction status management device, 50...Store controller, 60...Headquarters server, 230...Settlement device, 250...Settlement device, 260...Charging device, 270...Label issuing device, 280...Weighing device, 290...Ticket vending machine

Claims (9)

A system including an information processing device that progresses transaction processing based on a customer's operation,
an imaging unit that images a predetermined area;
a storage unit that stores event information of an event that occurs in a transaction;
a display unit that can accept touch operations and displays information according to the progress of processing;
The display unit
an image captured by the imaging unit can be displayed, and when an operation for designating one of the events is performed, the image captured at the time of occurrence of the event can be displayed ;
The event is
This includes taking home unregistered products without registering them.
An information processing system comprising: The display unit
A display transition image showing the transition of the display content of the display unit can be displayed, and the captured image and the display transition image are displayed on the same screen with the time of capture and the time of display being the same.
2. The information processing system according to claim 1, wherein : The display unit
3. The information processing system according to claim 2 , wherein an image generated based on operation log information is displayed as the display transition image. The display unit
The information processing system according to claim 3 , wherein the display transition image notifies the position of the touch operation included in the log information. The display unit
The information processing system according to claim 2 , wherein touch position information indicating a position of a touch operation included in operation log information is displayed superimposed on the display transition image. The information processing system according to any one of claims 2 to 5, further comprising a switching unit that switches between a first mode in which the display unit displays the captured image and the display transition image, a second mode in which the display unit displays the captured image without displaying the display transition image, and a third mode in which the display unit displays the display transition image without displaying the captured image. The information processing system according to claim 1 , further comprising a transmission unit that transmits the captured image . An information processing device that progresses processing based on a customer's operation,
imaging means for imaging a predetermined area;
a display means capable of receiving a touch operation and displaying information according to the progress of processing;
The display means
An information processing device characterized in that an image captured by said imaging means and a display transition image showing the transition of the display content by said display means are displayed on the same screen, with the time of imaging and the time of display being aligned. A program for causing an information processing device, which has a display unit capable of receiving touch operations and progresses processing based on customer operations, to function as a computer,
The computer
a captured image acquisition means for acquiring a captured image of a predetermined area;
a display control unit that controls the display of the display unit;
The display control means
A program characterized by displaying the captured image and a display transition image showing the transition of the display content of the display unit on the same screen, with the capture time and the display time coinciding.