JP7149761B2 - Information processing system, customer flow line analysis method - Google Patents

Description

The present invention relates to an information processing system and a customer flow analysis method.

Conventionally, a person tracking system using a laser sensor or a distance sensor has been developed to analyze customer trends in a store. In these systems, a dedicated device (for example, a laser sensor) for tracking a person is installed, resulting in high installation costs. Therefore, as a method of tracking a person at a reduced cost, a technique for analyzing the image of an existing surveillance camera has been developed. This method uses cameras already installed in the store for security purposes, so no additional equipment is required. However, it cannot be said that the accuracy of tracking is sufficient only by image analysis of surveillance images.

In view of such problems, for example, in Patent Document 1, one or more detection areas are set according to customer behavior in the measurement range, and the customer behavior recognized for each detection area is output as an behavior log, A technique is disclosed for efficiently acquiring customer flow lines without omission by organizing output action logs using time-series information and action process information.

JP 2017-083980 A

In Patent Literature 1, a flow line is acquired by sorting out a customer's action log for each detection area. However, since flow lines are not acquired based on product purchase histories, customer flow lines cannot be accurately analyzed for each product and each customer.

SUMMARY OF THE INVENTION It is an object of the present invention to provide an information processing system and a customer flow line analysis method capable of accurately analyzing customer flow lines for each product and for each customer based on product purchase histories.

In order to solve the above problems, an information processing system according to the present invention is an information processing system for analyzing customer flow lines, and is an image data acquisition system for acquiring image data captured by an imaging unit provided in a store. a sales information acquisition unit that acquires sales data indicating product sales at the store; the imaging data acquired by the imaging data acquisition unit; layout data that indicates the layout of the store; a tracking processing unit that generates tracking data including the flow line of the customer based on the display data of the products, and the tracking processing unit determines whether or not there is a missing part in the tracking data. and interpolating the missing part based on the sales data and the display data acquired by the sales information acquisition unit when it is determined that the tracking data has a missing part. Configured.

The present invention can also be grasped as a customer flow line analysis method performed by the above information processing system.

According to the present invention, it is possible to accurately analyze customer flow lines for each product and each customer based on the product purchase history.

It is a figure which shows the functional structure of the information processing system in a present Example. FIG. 3 is a diagram showing an example of data stored in a storage unit; FIG. FIG. 4 is a diagram showing an example of layout data; It is a figure which shows the example of merchandise display data. It is a figure which shows the example of imaging data. It is a figure which shows the example of POS data. It is a flowchart which shows the processing procedure of the process (flow line analysis process) for analyzing a customer's flow line using this system. 4A and 4B are diagrams showing examples of tracking data and interpolated tracking data; FIG.

Exemplary embodiments of an information processing system and a customer flow analysis method according to the present invention will be described below in detail with reference to the accompanying drawings.

FIG. 1 is a diagram showing the functional configuration of an information processing system 1000 in this embodiment. The information processing system 1000 is a system for analyzing customer flow lines. In the following, we analyze the flow lines of customers when they purchase products at a store, but we are not limited to this analysis. It can be applied when analyzing.

As shown in FIG. 1, an information processing system 1000 includes a server 100 for analyzing customer flow lines, an administrator terminal 200 operated by an administrator of the system, and an object for analyzing the customer flow lines. and a sales system 300 (300-1 to 300-N) used in different stores. The server 100 and the administrator terminal 200 are connected via a general network N1 such as a LAN (Local Area Network) or a WAN (Wide Area Network). Also, the server 100 and the sales system 300 of each store are connected via a general network N2 such as the Internet. FIG. 1 shows that a plurality of stores A to N use this system. Stores include, for example, merchandise stores such as supermarkets and convenience stores.

The server 100 is configured by a general computer as hardware. As shown in FIG. 1, the server 100 includes a storage unit 101, a control unit 102, a communication unit 103, an image acquisition unit 104, a POS (Point Of Sales) information acquisition unit 105, and a tracking processing unit 106. It is configured with

The functions of the above-described units that constitute the server 100 are realized by the control unit 102 reading programs from the memory and executing them. The program may be downloaded from the network via the communication unit 103, loaded onto the memory, and executed by the control unit 102. FIG. In addition, information can be directly loaded onto a memory from a portable computer-readable storage medium such as a CD (Compact Disk) or DVD (Digital Versatile Disk) via a reading device that reads and writes information from the storage medium. , may be executed by the control unit 102 . Furthermore, the various programs described above may be provided or distributed by being recorded on the storage medium in the form of files in a computer-installable or executable format. Furthermore, the predetermined program may be stored on a computer connected to a communication network and provided or distributed by being downloaded via the network.

The storage unit 101 is composed of a storage device such as a HDD (Hard Disk Drive) or an SSD (Solid State Drive), and stores various data necessary for this system.

FIG. 2 is a diagram showing an example of data stored in the storage unit 101. As shown in FIG. As shown in FIG. 2, the storage unit 101 stores layout data 1011, merchandise display data 1012, imaging data 1013, and POS data 1014. FIG. These data are transmitted and stored from all stores using this system.

FIG. 3 is a diagram showing an example of the layout data 1011. As shown in FIG. The layout data 1011 is a diagram showing the layout of things in the store, such as POS terminals, product display shelves, aisles, and entrances. As shown in FIG. 3, the layout data 1011 includes a POS terminal which is a sales processing unit arranged in the store, a camera which is an imaging unit 302, product display shelves A (A-1) to E (E3-4). ) and other layout drawings are stored. In FIG. 3, for example, the doorway E of the store A is provided on the upper left of the drawing, and cameras are provided on all four sides. Also, in store A, product shelves A to D are arranged on the wall surface, and product shelf E is arranged in the central portion. In this way, the layout data 1011 stores drawing data indicating the arrangement of items in the store for each store.

FIG. 4 is a diagram showing an example of product display data 1012. As shown in FIG. The product display data 1012 is data indicating the relationship between the product display shelf for each store and the products displayed on the display shelf. As shown in FIG. 4, the product display data 1012 includes the product number, which is product identification information, the product name of the product identified by the product number, and the products for which the product identified by the product name is displayed. A rack number for identifying the rack is stored in association with it. FIG. 4 shows that, for example, in a certain store (eg, store A), the product identified by product No. 0001 is ○○ chocolate, which is displayed in the upper row of E1-1. In this way, in the product display data 1012, products are stored in association with display shelves on which the products are displayed.

FIG. 5 is a diagram showing an example of the imaging data 1013. As shown in FIG. The imaging data 1013 is imaging data such as image data or moving image data captured by a camera provided in the store. As shown in FIG. 5, the imaging data 1013 stores the data captured by the camera in association with the date and time when the data was captured. In FIG. 5, for example, the data captured by the upper left camera in the layout data 1011 shown in FIG. , E1-2 are imaged, and the image of the customer P trying to pick up the product displayed in the middle of the display shelf E1-3 is imaged. In this way, in the image data 1013, data captured by cameras in the store are accumulated in time series.

FIG. 6 is a diagram showing an example of the POS data 1014. As shown in FIG. The POS data 1014 is data indicating sales of products purchased by customers at stores. As shown in FIG. 6, in the POS data 1014, at 11:11:11 am on October 10, 2017, which is the time of purchase of a product at store A, a male customer in his thirties (customer number C0001) , two ◯◯ chocolates of product number "0001" and one XX ballpoint pen of product number "0005" were purchased. Also, the amounts are XXX yen and △△△ yen, respectively. In this way, the POS data 1014 stores product purchase data for each store and each customer. Next, referring back to FIG. 1, each unit after the control unit 102 will be described.

The control unit 102 is composed of an arithmetic device such as a CPU (Central Processing Unit), for example, and controls each unit in the server 100 .

The communication unit 103 is composed of a communication device such as a NIC (Network Interface Card), for example, and transmits and receives various data necessary for this system between the server 100 and the administrator terminal 200 or sales system 300 .

The image acquisition unit 104 is a program executed by the control unit 102 , acquires the imaged data captured by the imaging unit 302 of the sales system 300 , and stores the imaged data 1013 in the storage unit 101 .

The POS information acquisition unit 105 is a program executed by the control unit 102 , acquires sales data processed by the sales processing unit 303 of the sales system 300 , and stores the sales data as POS data 1014 in the storage unit 101 .

The tracking processing unit 106 is a program executed by the control unit 102 and analyzes the flow line of the customer using the imaging data 1013 and the POS data 1014 . In addition, the tracking processing unit 106 determines whether or not there is a missing portion in the analyzed flow line, and when determining that there is a missing portion, interpolates the missing portion. Specific processing performed by the above units of the server 100 will be described later. Next, the administrator terminal 200 will be explained.

The administrator terminal 200 is a terminal used by the administrator of this system when analyzing customer flow lines, and is composed of a general computer as hardware. As shown in FIG. 1, the administrator terminal 200 includes an input section 201, an output section 202, a control section 203, and a communication section 204. FIG.

The input unit 201 is composed of, for example, input devices such as a keyboard and a mouse, and receives input of various information necessary for analysis, such as customers and stores to be analyzed for flow lines, dates and times, etc., from the administrator.

The output unit 202 is composed of, for example, a display device such as an LCD (Liquid Crystal Display), and displays various types of information output by the system, such as the results of the analysis performed by the server 100 .

The control unit 203 is composed of, for example, a computing device such as a CPU, and controls each unit in the administrator terminal 200 .

The communication unit 204 is composed of a communication device such as a NIC, for example, and transmits/receives various data necessary for this system to/from the server 100 . Specific processing performed by each of the above sections of the administrator terminal 200 will be described later. Next, the sales system 300 will be explained.

The sales system 300 is a system for recording the sales of products purchased at stores that use this system. As shown in FIG. 303 , a storage unit 304 and a control unit 305 .

The communication unit 301 is composed of a communication device such as a modem, for example, and transmits/receives various data necessary for this system to/from the server 100 .

The imaging unit 302 is composed of, for example, an imaging device such as a CCD (Charge Coupled Device) and a computer that communicates with the imaging device and the communication unit 301, and outputs an image of the inside of the store captured by the imaging device, 301 to the server 100 .

The sales processing unit 303 is composed of a POS terminal and a computer that communicates with the POS terminal, executes sales processing for products purchased at a store, and processes the POS data 1014 including the product name, quantity, amount, etc. of the product. It outputs and transmits to the server 100 via the communication unit 301 each time.

The storage unit 304 is configured by a storage device such as an HDD, and stores store layout data, merchandise display data, imaging data, and POS data. Since these data have the same configuration as in FIGS. 3 to 6, description thereof is omitted here.

The control unit 305 is composed of, for example, a computer having an arithmetic unit such as a CPU, and controls each unit and each device in the tube sales system 300 . Specific processing performed by each of the above sections of the sales system 300 will be described later. Next, processing performed in this system will be described.

FIG. 7 is a flow chart showing the procedure of processing (flow line analysis processing) for analyzing customer flow lines using this system. As shown in FIG. 7, in the sales system 300 of each store, the control unit 305 generates store layout data and product display data (step S701), and sends these data to the server 100 via the communication unit 301. Send (step S702).

The control unit 102 of the server 100 registers the store layout data and product display data received from the sales system 300 of each store as layout data 1011 and product display data 1012, and stores them in the storage unit 101 (step S703). .

In addition, the imaging unit 302 of the sales system 300 of each store generates imaging data such as images and videos captured in the store (step S704), and transmits the data to the server 100 via the communication unit 301 (step S704). step S705). Since the imaging unit 302 is always activated, the imaging data is transmitted in real time thereafter.

The control unit 102 of the server 100 registers the imaged data received from the sales system 300 of each store as the imaged data 1013 and stores it in the storage unit 101 (step S706). Thereafter, the imaging data is accumulated in the storage unit 101 in real time for each store.

In addition, the sales processing unit 303 of the sales system 200 executes sales processing for products purchased by customers visiting the store each time (step S707), and transmits POS data including the sales data to the server 100 each time (step S707). S708).

The control unit 102 of the server 100 registers the POS data received from the sales system 300 of each store as the POS data 1014 and stores it in the storage unit 101 (step S709). Thereafter, the POS data is accumulated in the storage unit 101 for each store each time the sales process is performed.

In such a case, when the input unit 201 of the administrator terminal 200 receives an input of tracking conditions, which are conditions for analyzing customer flow lines, from the administrator (step S710), the control unit 203 performs communication. A tracking request including the tracking conditions is transmitted to the server 100 via the unit 204 (step S711). The tracking conditions may include, for example, store name, imaging period of imaging data, product name included in POS data, product purchase time, customer number, and age of the customer.

When the server 100 receives the tracking request from the sales system 300, the server 100 executes tracking data generation processing (step S712).

Specifically, the image acquisition unit 104 of the server 100 reads the store name and the imaging period of the imaging data included in the tracking request, accesses the imaging data 1013 using the store name and the imaging period as keys, and obtains information about the store. imaged data for the relevant period is extracted. Then, the POS information acquisition unit 105 of the server 100 accesses the POS data 1014 using the store name, product name, product purchase time, customer number, and customer age as keys, and extracts the key from the POS data 1014. is extracted as the POS data of the corresponding customer of the corresponding store to be tracked. Further, the tracking processing unit 106 extracts the layout data of the store from the layout data 1011, and uses the extracted layout data, the extracted imaging data, and the extracted POS data to track the movement of the customer. Generate tracking data for line analysis. Generation of the tracking data will be described later.

The tracking processing unit 106 determines whether or not there is a missing portion in the customer's flow line included in the generated tracking data (step S713). If the tracking processing unit 106 determines that there is no missing portion in the customer flow line included in the tracking data (step S713; No), the tracking processing unit 106 transmits the generated tracking data to the administrator terminal 200 (step S714). After that, the control unit 203 of the administrator terminal 200 displays the received tracking data on the output unit 202 (step S715).

On the other hand, when the tracking processing unit 106 determines that there is a missing portion in the customer's flow line included in the tracking data (step S713; Yes), the tracking processing unit 106 executes interpolation processing for interpolating the missing portion (step S716). Interpolated tracking data after the processing is performed is generated (step S717). The interpolation processing and interpolation tracking data will be described later.

The tracking processing unit 106 transmits the generated interpolated tracking data to the administrator terminal 200 (step S718), and the control unit 203 of the administrator terminal 200 displays the received interpolated tracking data on the output unit 202 (step S718). S719).

Further, the tracking processing unit 106 transmits the tracking data or the interpolated tracking data to the sales system 300 of the store (step S720), and the control unit 305 of the sales system 300 controls the display unit connected to the control unit 305. , the tracking data or the interpolated tracking data received from the server 100 is displayed (step S721).

FIG. 8 is a diagram showing examples of tracking data and interpolated tracking data. The tracking data is data obtained by superimposing the flow line L of the customer on the layout data 1011 . As a method of superimposing, for example, the tracking processing unit 106 extracts feature information such as a customer's face image, clothes, appearance, etc. included in the imaging data. Further, the tracking processing unit 106 identifies image data containing the same characteristic information as the extracted characteristic information, calculates customer position information in the identified image data, and plots it on the layout data 1011 . By performing such processing, points plotted on the layout data 1011 appear as customer flow lines. In FIG. 8, a customer enters from the entrance E of the store A, passes through the aisles beside the product display shelves A-1, A-2, B-3, B-2, B-1, and C-2, and reaches D-2. , D-3 to the POS terminal. In the tracking data, it can be seen that the range BR on the line of flow is a blind spot of the camera, and the plotted points are missing points, making the line of flow discontinuous.

The tracking processing unit 106 determines whether or not there is a missing portion in step S713, and if it is determined that there is a missing portion (step S713; Yes), the layout data 1011, the product display data 1012, and the POS data 1014 are referred to. Perform an interpolation process to estimate the route. For example, the tracking processing unit 106 refers to the layout data 1011 to identify the position near the missing portion. Further, the tracking processing unit 106 refers to the product display data 1012 and the POS data 1014, compares the product name on the product display shelf adjacent to the specified position with the product name in the POS data, and finds the product with the product name. Identify the product shelves on display. After identifying the product display shelf, the tracking processing unit 106 determines that the customer has moved near the product display shelf, and plots the estimated customer flow line on the layout data. In FIG. 8, according to the above determination, the flow line L′ estimated at the missing portion is plotted, and the customer flow line L+L′ including the missing portion is generated as the interpolated tracking data and shown as the interpolated tracking data. It can be estimated that the customer's flow line is the flow line of the customer. In other words, it is possible to grasp the line of flow including missing parts when a customer purchases a certain product at a store.

As described above, in this embodiment, by performing the above processing, it is possible to accurately analyze customer flow lines for each product and each customer. In particular, in the marketing of products in large-scale stores such as supermarkets, it is important to accurately determine the flow line from customer visit to checkout. If it is possible to accurately determine the flow of customers, it is possible to refer to the POS data and collect the number of customers who did not purchase the product even though they passed by the product displayed on the product display shelf. It becomes possible. In this way, by estimating the customer's flow line together with the POS data, it is possible to improve the accuracy of the flow line analysis. In addition, by accurately grasping the flow of customers, it will be possible to estimate not only the products that were purchased, but also the products that were not purchased.

It should be noted that the present system is not limited to the above-described embodiment as it is. The components can be appropriately combined for implementation.

For example, the tracking processing unit 106 of the server 100 may acquire the flow line of customers whose layout is similar to that of other stores, and interpolate to estimate the flow line. Specifically, the presence or absence of layout data similar between stores is determined from the layout data 1011, and if it is determined that there is layout data similar between stores, the layout data is further determined to be similar. It is determined whether or not the product display data 1012 of the determined stores are similar. The similarity of layout data between stores means, for example, that the difference in store area, aisle width, and product display shelf position is within a predetermined range (for example, the difference in product display shelf position is within 1 meter). , it is determined that the two are similar. Further, when the product display data are similar, for example, when the arrangement of the products displayed on the product display shelf is similar, it may be determined that the two are similar. For example, in a store to be compared, as shown in FIGS. 3 and 4, when both chocolate products are arranged on the product display shelf in the center of the store instead of the store wall surface, the product display data of these stores are It suffices to determine that they are similar. By such determination, when the tracking processing unit 106 determines that the layout data and product display data of the store are similar, the missing part of the tracking data including the missing part is replaced by another store. Interpolation is performed using the tracking data of the portion corresponding to the missing portion of the missing tracking data. By performing such processing, even if the interpolated tracking data cannot be generated using the POS data, the flow line of the customer can be estimated.

Furthermore, when it is determined that the layout data of the store and the product display data are similar, the tracking data to be compared is selected based on the imaging date and time of the imaging unit 302 and the purchase time of the POS data. You may narrow down and estimate a customer's flow line. For example, the tracking processing unit 106 determines that the gender, age, and purchase time of the POS data (or the imaging date and time of the imaging unit 302 of the store) of the stores determined to be similar are within a predetermined range. Identify the tracking data of a certain store (for example, the date of purchase if it is the time of purchase), and if the missing part of the tracking data of the specified store is determined to be within the predetermined range based on the above criteria, there will be a missing part. Interpolation may be performed using the tracking data of the portion corresponding to the missing portion of the tracking data that is not included. In other words, customer flow lines differ depending on the time of day and season (for example, in the summer, there are many lines of flow heading to the ice cream counter), so these are used as a basis for estimation. Since it is conceivable that they are similar, the customer's flow line may be estimated based on information such as the age and sex of the customer obtained from the POS data. By performing such processing, the number of objects to be determined to be similar can be narrowed down, and even if the interpolated tracking data cannot be generated using the POS data, the flow line of the customer can be efficiently calculated. can be estimated.

1000 Information processing system 100 Server 101 Storage unit 1011 Layout data 1012 Product display data 1013 Imaging data 1014 POS data 102 Control unit 103 Communication unit 104 Image acquisition unit 105 POS information acquisition unit 106 Tracking processing unit 200 Administrator terminal 201 Input unit 202 Output Unit 203 Control unit 204 Communication unit 300 Sales system 300 (300-1 to 300-N)
301 communication unit 302 imaging unit 303 sales processing unit 304 storage unit 305 control unit N1, N2 network.

Claims (4)

An information processing system for analyzing customer flow lines,
an imaging data acquisition unit that acquires imaging data captured by an imaging unit provided in a store;
a sales information acquisition unit that acquires sales data indicating product sales at the store;
Tracking data including the flow line of the customer is generated based on the imaging data acquired by the imaging data acquisition unit, layout data indicating the layout of the store, and display data of products displayed in the store. a tracking processing unit,
The tracking processing unit determines whether or not there is a missing portion in the tracking data, and if it is determined that there is a missing portion in the tracking data, the sales data acquired by the sales information acquisition unit and the display data. When interpolating the missing part based on
The tracking processing unit determines the presence or absence of similar layout data between stores, and if it is determined that there is similar layout data between stores, the display data of the store determined to be similar is It is determined whether or not the display data are similar, and if it is determined that the display data are similar, the missing part of the tracking data is replaced with the missing part of the tracking data that does not occur in other stores. Interpolate with the part corresponding to the point,
An information processing system characterized by: When determining that the layout data and the display data of the store are similar, the tracking processing unit narrows down the tracking data to be compared based on the time of the imaging data or the sales data.
The information processing system according to claim 1 , characterized by: A customer flow line analysis method in which a computer analyzes a customer flow line,
the computer
Acquiring imaging data captured by an imaging unit provided in the store,
Acquiring sales data indicating product sales at the store,
generating tracking data including the flow line of the customer based on the acquired imaging data, layout data indicating the layout of the store, and display data of products displayed in the store;
Determining whether or not there is a missing part in the tracking data,
When it is determined that there is a missing part in the tracking data, and when interpolating the missing part based on the acquired sales data and the display data ,
Determine the presence or absence of similar layout data between stores,
when determining that there is similar layout data between stores, determining whether the display data of the stores determined to be similar are similar;
When it is determined that the display data are similar, interpolate the missing part of the tracking data with a part corresponding to the missing part of the tracking data that is not missing in other stores.
A customer flow line analysis method characterized by: the computer
Narrowing down the tracking data to be compared based on the time of the imaging data or the sales data when determining that the layout data and the display data of the store are similar;
The customer flow line analysis method according to claim 3 , characterized in that:

Images