JP6825613B2 - Information processing method, program, information processing device and trained model generation method - Google Patents

Description

The present invention relates to an information processing method, a program, an information processing device, and a method of generating a trained model.

A system is provided to support companies in formulating store opening and closing plans. For example, in Patent Document 1, a trade area analysis system that calculates the frequency at which a user visits a predetermined commercial area and the time during which the user stays based on the location information acquired from a mobile terminal owned by the user and performs trade area analysis. It is disclosed.

Japanese Unexamined Patent Publication No. 2015-197696

However, the invention according to Patent Document 1 merely provides data as a reference for the store opening / closing plan, and does not evaluate the store opening / closing plan itself.

In one aspect, the purpose is to provide an information processing method or the like that can appropriately support marketing.

In one aspect, the information processing method passes through the relevant points at a plurality of time zones based on an image obtained by imaging the relevant points in the public space from an imaging device attached to a power distribution facility installed in the public space. get the passer-by information of the analysis of the passerby, the passer information when you enter, before Ki該visit the number and該来store number of people of another time zone to the store store opening schedule to the person point of each time zone The acquired passerby information is input to the trained model that has been trained to output the evaluation information indicating the predicted value of the breakdown of each attribute of, the evaluation information is acquired from the trained model, and the acquired evaluation. It is characterized in that a computer executes a process of outputting information.

On one side, marketing can be adequately supported.

It is a schematic diagram which shows the configuration example of the store evaluation system. It is a block diagram which shows the configuration example of a server. It is explanatory drawing which shows an example of the record layout of a utility pole DB, a pedestrian DB, and a peripheral information DB. It is explanatory drawing about learning process of pedestrian information. It is explanatory drawing about a pedestrian analysis process. It is explanatory drawing about learning process of store evaluation. It is explanatory drawing about a store evaluation process. It is a flowchart which shows the processing procedure of the learning process of pedestrian information. It is a flowchart which shows the processing procedure of a pedestrian analysis process. It is a flowchart which shows the processing procedure of the learning process of a store evaluation. It is a flowchart which shows the processing procedure of a store evaluation process. It is explanatory drawing which shows the outline of Embodiment 2. It is a flowchart which shows the processing procedure of the store evaluation processing which concerns on Embodiment 2.

Hereinafter, the present invention will be described in detail with reference to the drawings showing the embodiments thereof.
(Embodiment 1)
FIG. 1 is a schematic diagram showing a configuration example of a store evaluation system. In the present embodiment, a store evaluation system that evaluates newly opened stores based on pedestrian information obtained by analyzing pedestrians based on images of pedestrians captured by a camera 2 installed in a public space such as a road. explain. The store evaluation system includes an information processing device 1, a camera 2, a sensor 3, and a terminal 4. Each device is communicated and connected via a network N such as the Internet.

The information processing device 1 is an information processing device capable of transmitting and receiving various types of information processing and information, and is, for example, a server device, a personal computer, or the like. In the present embodiment, it is assumed that the information processing device 1 is a server device, and in the following description, it is read as a server 1 for the sake of brevity. The server 1 acquires an image of a pedestrian passing through the road from the camera 2, analyzes the acquired image, and analyzes attributes such as age and gender, behavior such as walking speed, direction, and movement, or personal belongings. Pedestrians are classified (analyzed) according to various criteria of the above, and big data accumulating pedestrian information showing the pedestrian classification results is constructed. In the present embodiment, the server 1 analyzes passers-by using a classification model (first learned model) that has been learned by machine learning so as to classify passers-by from captured images.

Further, the server 1 predicts how much the store will achieve when the store is opened at a certain point (corresponding point) in the public space, and evaluates the store opening. Specifically, the server 1 is an evaluation model (second learned model) different from the above classification model, and inputs the passerby information output from the classification model to open a store at the corresponding point. The store is evaluated using an evaluation model that outputs the evaluated evaluation information.

In the present embodiment, the evaluation is performed on the newly opened store, but the evaluation may be performed on the existing store that already exists. That is, this system may be used for a store opening plan, a review of an existing store, or a store closing plan.

The camera 2 is an image pickup device installed in a public space, and images a passerby passing through a road. In the present embodiment, the camera 2 is attached to a power distribution facility installed along the road, specifically, a utility pole. The camera 2 is attached to a utility pole installed at each of a plurality of points along the road, and continuously images a passerby passing through the road from each point. The server 1 acquires an image captured by the camera 2 and analyzes the attributes of passersby reflected in the acquired image. In addition, depending on the measurement environment, a motion sensor that detects a person by laser or infrared rays may be installed instead of the camera 2 to acquire data on the flow line information of the person such as the number of subjects, speed, and residence time. Good.

In the present embodiment, utility poles are given as an example of distribution equipment, but the distribution equipment is not limited to utility poles, and may be, for example, ground equipment (transformers) of underground electric wires. In addition, the place (point) where the power distribution equipment is installed is not limited to the road, and may be a public space that pedestrians can pass through. Moreover, the "road" in this specification may include both a sidewalk and a roadway.

The sensor 3 is a sensing device attached to a utility pole together with a camera 2, and is, for example, a thermometer, a hygrometer, a rainfall meter, an illuminance meter, an anemometer, or the like. The sensor 3 measures weather information such as temperature, humidity, and rainfall at the point where the camera 2 is installed. The server 1 acquires the weather information of the image capturing point from the sensor 3 and stores it in association with the classification result of the passersby.

The terminal 4 is a terminal device used by a user who receives big data from the server 1, and is an information processing terminal such as a personal computer, a smartphone, or a tablet terminal. The user who uses this system is not particularly limited, but for example, the user is a business operator who markets the relevant point, and the terminal 4 receives the evaluation information of the store opening at the relevant point from the server 1 and displays the evaluation information. To do.

FIG. 2 is a block diagram showing a configuration example of the server 1. The server 1 includes a control unit 11, a main storage unit 12, a communication unit 13, and an auxiliary storage unit 14.
The control unit 11 has one or a plurality of arithmetic processing units such as a CPU (Central Processing Unit), an MPU (Micro-Processing Unit), and a GPU (Graphics Processing Unit), and stores a program P stored in the auxiliary storage unit 14. By reading and executing, various information processing, control processing, etc. related to the server 1 are performed. The main storage unit 12 is a temporary storage area for SRAM (Static Random Access Memory), DRAM (Dynamic Random Access Memory), flash memory, etc., and temporarily stores data necessary for the control unit 11 to execute arithmetic processing. Remember. The communication unit 13 is a communication module for performing processing related to communication, and transmits / receives information to / from the outside.

The auxiliary storage unit 14 is a large-capacity memory, a hard disk, or the like, and stores a program P and other data necessary for the control unit 11 to execute processing. Further, the auxiliary storage unit 14 stores the classification model 141, the evaluation model 142, the utility pole DB 143, the pedestrian DB 144, and the peripheral information DB 145. The classification model 141 is a classifier (classifier) that classifies passers-by in the captured image, and is a trained model generated by machine learning. The evaluation model 142 is a generator (identifier) that generates evaluation information of a store based on pedestrian information, and is a trained model generated by machine learning. The utility pole DB 143 is a database that stores information on the utility pole on which the camera 2 is installed. The pedestrian DB 144 is a database that stores the classification results of pedestrians classified using the classification model 141. The peripheral information DB 145 is a database that stores peripheral information of the utility pole on which the camera 2 is installed.

The auxiliary storage unit 14 may be an external storage device connected to the server 1. Further, the server 1 may be a multi-computer composed of a plurality of computers, or may be a virtual machine virtually constructed by software.

Further, in the present embodiment, the server 1 is not limited to the above configuration, and may include, for example, an input unit that accepts operation input, a display unit that displays an image, and the like. Further, the server 1 may download the program P from another computer via the network N and store it in the auxiliary storage unit 14. Further, the server 1 is provided with a reading unit that reads a portable storage medium 1a such as a CD (Compact Disk) -ROM, a DVD (Digital Versatile Disc) -ROM, and reads and executes the program P from the portable storage medium 1a. It may be. Alternatively, the server 1 may read the program P from the semiconductor memory 1b.

FIG. 3 is an explanatory diagram showing an example of the record layout of the utility pole DB 143, the pedestrian DB 144, and the peripheral information DB 145. The utility pole DB 143 includes a utility pole ID string, a position information string, a camera information string, and a sensor information string. The utility pole ID column stores a utility pole ID for identifying each utility pole installed on the road. The position information string, the camera information column, and the sensor information string are associated with the utility pole ID, respectively, and store the position information of the point where the utility pole is installed, the information of the camera 2 installed on the utility pole, and the information of the sensor 3. There is. The information of the camera 2 may include, for example, information such as the height at which the camera 2 is attached, the imaging angle, the magnification, and the imaging range. The information of the sensor 3 may include information about the type of the sensor 3 such as a thermometer, a hygrometer, and a rainfall meter.

The pedestrian DB 144 includes a utility pole ID row, a date and time row, a weather row, and a pedestrian row. The utility pole ID stores a utility pole ID for identifying each utility pole installed on the road. The date and time column, the weather column, and the pedestrian column are associated with the utility pole ID to classify the date and time when the image was taken by the camera 2 attached to the utility pole, the weather information around the utility pole at that date and time, and the pedestrians reflected in the image. It remembers the pedestrian information obtained by doing so.

The peripheral information DB 145 includes an area ID column, an area name column, and a peripheral information column. The area ID column stores the area ID for identifying each area in which the utility pole to which the camera 2 is attached is installed. The area name column stores the name of each area in association with the area ID. The peripheral information column stores peripheral information of each region in association with the regional ID. Peripheral information includes, for example, geographical data such as the population of the trade area, the size of the trade area, the location of the nearest station, statistical data such as the number of passengers getting on and off the nearest station, and store information of nearby stores existing in the area.

FIG. 4 is an explanatory diagram relating to the learning process of pedestrian information. FIG. 4 conceptually illustrates the process of performing machine learning to generate the classification model 141. The generation process of the classification model 141 will be described with reference to FIG.

In the present embodiment, the server 1 uses the classification model 141 to perform deep learning to learn the image features related to the appearance, behavior, belongings, etc. of the passerby reflected in the image captured by the camera 2. Is used as an input, and a neural network is generated that outputs the passerby information that classifies (analyzes) the passersby. The neural network is, for example, CNN (Convolution Neural Network), specifically R-CNN (Regions with CNN), and has an input layer that accepts input of captured images, an output layer that outputs classification results, and image features of captured images. It has an intermediate layer from which the amount is extracted.

The input layer has a plurality of neurons that receive input of pixel values of each pixel included in the captured image, and passes the input pixel values to the intermediate layer. The middle layer has a plurality of neurons, recognizes an object (passerby, etc.) from the captured image, and extracts the feature amount of the image area (shown by the dotted rectangular frame in FIG. 4) in which the recognized object is reflected. And pass it to the output layer. For example, when the classification model 141 is CNN, the intermediate layer alternates between a convolutional layer that convolves the pixel values of each pixel input from the input layer and a pooling layer that maps the pixel values convoluted by the convolutional layer. It has a concatenated configuration, and finally extracts the image feature amount while compressing the pixel information of the image area of the object. The output layer is a classifier related to SVM (Support Vector Machine), has one or more neurons that output the classification result of pedestrians, and classifies pedestrians based on the image features output from the intermediate layer. ..

Although the classification model 141 is described as being a CNN in the present embodiment, the classification model 141 is not limited to the CNN, and is constructed by other learning algorithms such as a neural network other than the CNN, a Bayesian network, and a decision tree. It may be a trained model.

The server 1 performs learning using teacher data in which a plurality of images of passersby are associated with correct answer values of passerby information when the passersby in each image are classified according to attributes and the like. For example, as shown in FIG. 4, the teacher data is data in which the coordinate range of the image area in which the pedestrian is reflected and the pedestrian information are labeled with respect to the captured image of the pedestrian.

In the drawings of the present specification, for the sake of brevity, one passerby is shown in one image, but it goes without saying that a plurality of passersby may be shown in one image. ..

The server 1 inputs the captured image, which is the teacher data, to the input layer, performs arithmetic processing in the intermediate layer, and acquires pedestrian information indicating the pedestrian classification result from the output layer. The pedestrian information output from the output layer may be a discrete value (for example, a value of "0" or "1") and a continuous probability value (for example, a value from "0" to "1"). It may be a value in the range). For example, the server 1 has pedestrian attributes such as gender and age, walking speed, walking direction, and pedestrian behavior such as entering and exiting a facility (for example, a store on the side of a road) as a classification result output from the output layer. And acquire pedestrian information classified according to personal belongings. The details of the above classification criteria will be described later.

The server 1 compares the passerby information output from the output layer with the information labeled for the captured image in the teacher data, that is, the correct answer value, and intermediately so that the output value from the output layer approaches the correct answer value. Optimize the parameters used for arithmetic processing in the layer. The parameters include, for example, the weight between neurons (coupling coefficient), the coefficient of the activation function used in each neuron, and the like. The method of optimizing the parameters is not particularly limited, but for example, the server 1 optimizes various parameters by using the backpropagation method.

The server 1 performs the above processing on each captured image included in the teacher data to generate the classification model 141. When the captured image is acquired from the camera 2, the server 1 classifies the passersby using the classification model 141.

FIG. 5 is an explanatory diagram regarding a pedestrian analysis process. FIG. 5 conceptually illustrates how pedestrians are classified (analyzed) from captured images based on various criteria using the classification model 141. The pedestrian analysis process will be described with reference to FIG.
The server 1 acquires images of passersby from cameras 2, 2, 2, ... Attached to each utility pole installed at a plurality of points along the road. For example, the camera 2 continuously captures images, and the server 1 periodically acquires a moving image captured by the camera 2 on a road on which a pedestrian passes, or recorded data of the moving image on a regular basis.

In addition, the server 1 acquires weather information such as temperature, humidity, and rainfall at the time when an image is taken from each sensor 3 attached to the same utility pole as each camera 2. The server 1 may acquire the weather information of the corresponding point from, for example, a predetermined external API (Application Programmable Interface) regardless of the sensor 3. That is, the installation of the sensor 3 on the utility pole is not essential.

The server 1 inputs the image acquired from the camera 2 into the classification model 141, and acquires the pedestrian information that classifies the pedestrians reflected in the image as an output. For example, the server 1 sequentially inputs the frame images at the time of each imaging that constitute the moving image captured by the camera 2 into the classification model 141. The server 1 performs arithmetic processing for extracting the feature amount of the frame image in the intermediate layer of the classification model 141, inputs the extracted feature amount to the output layer of the classification model 141, and is imaged at each point and at each imaging time. The passerby information that classifies the passersby shown in the frame image is acquired as an output.

In the present embodiment, the server 1 classifies pedestrians based on various criteria. As the first classification criterion, the server 1 classifies according to the attributes of passersby. The attributes of passersby are, for example, gender and age. For example, the server 1 classifies pedestrians by gender and by classifying minors in their 20s to 30s, 40s to 50s, and 60s or older, for a total of 8 classifications.

The attributes of the passerby are not limited to gender and age, and for example, emotions that can be recognized from the face (facial expression) of the passerby may be detected (classified) as attributes.

As the second classification standard, the server 1 classifies according to the behavior of a passerby. The behavior of pedestrians is, for example, the walking speed of pedestrians, the walking direction (destination), the distance between pedestrians (whether or not they are a group, etc.), and the walking position on the road (right side or left side). Is it a passage, etc.) and other actions.

In particular, in the present embodiment, the server 1 detects the entry / exit movement of a passerby to a store located on the side of the road, that is, a facility existing in a public space, and classifies the passerby according to the detected entry / exit movement. .. The entrance / exit operation is an operation in which a passerby enters (enters) the store entrance (enters the store) or exits (exits) from the store entrance / exit. Note that the server 1 may detect only one of entry and exit. Further, the entrance / exit operation is not limited to the operation when actually entering / exiting the store, and may be an operation indirectly related to the entrance / exit, such as a stop operation in front of the store or an operation of directing the line of sight to the store. Further, the server 1 may detect whether a single pedestrian has entered or exited or a plurality of pedestrians have entered or exited as a group as a more detailed entry / exit operation. The server 1 classifies pedestrians who have entered and exited the store and pedestrians who have passed without entering the store among the pedestrians who have passed in front of the store according to the detected entry / exit operation.

As the third classification standard, the server 1 classifies according to the belongings of passersby. The belongings of a passerby are, for example, an umbrella, a paper bag, a bag, a suitcase, a container for drinks, clothes of the passerby, and a pet (dog, etc.) carried by the passerby. The server 1 generates a classification model 141 that learns the image features of these belongings at the time of learning, detects the belongings of the passerby using the classification model 141, and determines the type of the detected belongings.

The server 1 not only detects the belongings of the passerby from the captured image, but also estimates the gender, age, etc. of the passerby according to the belongings, thereby accurately classifying the attributes of the passerby. For example, when the weather is fine and the server 1 has an umbrella, the server 1 determines that it is an umbrella and classifies passersby as women. Further, for example, when the passerby wears high heels, a skirt, or the like, the server 1 classifies the passerby as a woman. In this way, the server 1 detects the belongings using the classification model 141 and classifies the attributes of the passersby according to the detected belongings.

The server 1 inputs each frame image in the moving image acquired from each camera 2 into the classification model 141, and classifies the passersby who passed through each point at each imaging time according to the above-mentioned classification criteria. The server 1 may perform classification according to each of the above classification criteria with one classification model 141. A plurality of classification models 141 are prepared according to each classification criterion, and a plurality of classification models 141 are used. Classification may be performed.

The server 1 associates the passerby information obtained by classifying the passersby reflected in the frame image with the time of imaging (date and time) when the frame image was captured and the installation point of the camera 2 which captured the frame image, and the passersby. Store in DB 144. The server 1 continues the above processing and accumulates pedestrian information in the pedestrian DB 144.

In the present embodiment, the server 1 uses the pedestrian information accumulated in the pedestrian DB 144 for marketing as described above to evaluate the store where the user plans to open a new store. Specifically, the server 1 constructs an evaluation model 142 different from the classification model 141 by machine learning, inputs pedestrian information to the evaluation model 142, and outputs the evaluation information that evaluates the newly opened store as output. get.

FIG. 6 is an explanatory diagram relating to the learning process of store evaluation. FIG. 6 conceptually illustrates how machine learning is performed by inputting passerby information output from the classification model 141 to generate the evaluation model 142. A machine learning process for generating the evaluation model 142 will be described with reference to FIG.

For example, the server 1 generates a neural network related to LSTM (Long Short-Term Memory) as the evaluation model 142. LSTM is a kind of RNN (Recurrent Neural Network), and is a neural network that inputs time series data before a certain point in time and outputs a predicted value at that time point.

RSTMs, like CNNs, have an input layer, an intermediate layer, and an output layer. The input layer has a plurality of neurons that receive input of passerby information acquired from the captured images at each imaging time according to the time series at the time of imaging. The output layer has a plurality of neurons corresponding to each neuron in the input layer and outputs a predicted value predicted for a store visitor at each imaging time. The middle layer has a plurality of neurons for calculating the output value (predicted value) in each neuron of the output layer with respect to the input value (passer information) to each neuron of the input layer. Each neuron in the middle layer is called LSTM Block, and time-series data up to the latest time point is obtained by performing an operation on the input value at the next time point using the calculation result in the middle layer on the input value at the past time point. Calculates the value at the next time from.

The configuration of the LSTM shown in FIG. 6 is an example, and the present embodiment is not limited to this. For example, the intermediate layer is not limited to one layer, and may be two or more layers. Further, the number of neurons in the input layer and the output layer is not limited to the same number, and for example, the number of outputs may be at least good with respect to the input.

Further, the evaluation model 142 is not limited to the LSTM, and may be a trained model constructed by another learning algorithm such as a neural network other than the LSTM, an SVM, a Bayesian network, and a decision tree.

When performing machine learning related to store evaluation, for example, the server 1 uses information on an existing store that already exists (opens a store) as a correct answer value of the evaluation information output from the output layer. The existing store information is, for example, an actually measured value regarding the number of visitors to the existing store, such as the number of visitors by time zone and the breakdown of the number of visitors by attribute in the number of visitors in each time zone.

The server 1 may acquire an actually measured value such as the number of visitors from a management device that aggregates and manages store sales, for example, as in a POS (Point of Sale) system. Alternatively, the server 1 may calculate the number of visitors to the existing store, the breakdown of the number of visitors by attribute, and the like based on the entrance operation of the passersby detected (classified) from the image captured by the camera 2. As described above, the acquisition route of the measured value for the visitor is not particularly limited.

In addition, the server 1 acquires the industry of the existing store, the location information of the point where the existing store exists, and the like as the existing store information.

The server 1 uses the pedestrian information corresponding to the relevant point where the existing store exists, that is, the pedestrian acquired from the image of the pedestrian passing in front of the existing store by the camera 2 attached to the electric pole near the existing store. Person information is read from the passerby DB 144 and input to the evaluation model 142. The server 1 acquires not only the passerby information analyzed from the image acquired from the camera 2 that images the front of the existing store, but also the image captured by the other cameras 2 located within a predetermined range from the existing store. You may also use the passerby information. As a result, it is possible to carry out a more bird's-eye view and comprehensive learning in consideration of the flow of passersby around the existing store. As described above, the pedestrian information may include attributes such as gender and age, walking speed, walking direction, actions such as entrance / exit movement to / from the store, and belongings possessed by the pedestrian.

The server 1 inputs the pedestrian information read from the pedestrian DB 144 to the input layer of the evaluation model 142 according to the time series at the time of imaging in which each pedestrian is imaged. For example, the server 1 has the number of pedestrians passing in front of the existing store in each time zone (for example, every hour) when each image is captured, and the number of pedestrians by attribute, behavior, and belongings. Aggregate the breakdown of. The server 1 sequentially inputs the aggregated passerby information for each time zone to each neuron in the input layer in chronological order.

Further, the server 1 inputs the passerby information of each time zone, reads the weather information of each time zone from the passerby DB 144, and inputs the information to each neuron of the input layer. The weather information is information such as the weather, temperature, and humidity in each time zone. By inputting the weather information together, the server 1 can construct the evaluation model 142 in consideration of the influence of the weather on the number of store visits and the like.

In addition to this, the server 1 reads peripheral information indicating the peripheral situation of the existing store from the peripheral information DB 145 and inputs it to the evaluation model 142. Peripheral information includes, for example, statistical data such as the population of the trade area in the area where the existing store exists, the size of the trade area, the number of passengers getting on and off at the nearest station, geographical data such as the distance to the nearest station, and the surrounding area existing in the area. It may include data of stores (competitive stores). The data of the peripheral stores includes, for example, the type of business and location information of the peripheral stores.

For example, the server 1 reads out the location information of the peripheral stores whose industry matches the store to be opened, and calculates the distance between the peripheral stores and the stores to be opened. The server 1 inputs the calculated distance to the surrounding stores, the above statistical data, geographical data, and the like into the evaluation model 142.

The server 1 acquires the predicted value regarding the visitor in each time zone as evaluation information from each neuron in the output layer. The predicted value output from the output layer may include, for example, the number of visitors by time zone and the breakdown of the number of visitors by attribute in the number of visitors in each time zone.

The server 1 compares the predicted value output from the output layer with the measured value of the visitor of the existing store, which is the correct answer value, and performs an operation in the intermediate layer so that the predicted value output from the output layer approaches the correct answer value. Optimize the parameters used for processing. The server 1 performs the above-mentioned machine learning for a plurality of existing stores (for example, each affiliated store in which a chain is developed). As a result, the server 1 generates an evaluation model 142 that inputs passerby information such as the number of passersby and outputs a predicted value such as the number of visitors as evaluation information.

Further, at the time of the machine learning described above, the server 1 not only learns the number of visitors to the existing store, but also learns the number of visitors to the peripheral stores located in the vicinity of the existing store, thereby opening a new store. An evaluation model 142 that can predict an increase or decrease in the number of visitors to neighboring stores may be generated.

For example, the server 1 learns using the pedestrian information before the opening of the existing store and the pedestrian information after the opening of the existing store. Specifically, the server 1 is pedestrian information obtained from an image of the front of a nearby store taken by the camera 2, and the date of the imaging is the same as the pedestrian information before the existing store opens. Learning will be conducted using information on passers-by after the store opens in different years. The server 1 refers to the passerby information in front of the surrounding stores, calculates the actual measurement value of the number of visitors to the surrounding stores from the entrance operation of the passersby, and the actual measurement value of the number of visitors before and after the opening of the existing store. Calculate the difference between. That is, the server 1 calculates the actual value of the increase / decrease in the number of visitors to the neighboring stores due to the opening of the existing store.

The server 1 inputs the above-mentioned various data including the distance to the peripheral stores into the evaluation model 142, and calculates a predicted value of increase / decrease in the number of visitors to the peripheral stores due to the opening of the existing store. Then, the server 1 compares the calculated predicted value with the actual value and optimizes various parameters of the intermediate layer to generate an evaluation model 142 that predicts an increase or decrease in the number of visitors to neighboring stores.

The server 1 generates an evaluation model 142 that outputs evaluation information when pedestrian information, weather information, surrounding information, and the like are input as described above. The server 1 acquires the evaluation information of the newly opened store by using the generated evaluation model 142.

FIG. 7 is an explanatory diagram regarding the store evaluation process. FIG. 7 conceptually illustrates a state in which a user evaluates a store opening using this system when opening a new store. Based on FIG. 7, a process of generating and outputting evaluation information of a store opening using the above-mentioned evaluation model 142 will be described.

For example, the server 1 first accepts, via the terminal 4, the designated input of the minimum necessary information for evaluating the store opening, such as the location information indicating the corresponding point where the store is scheduled to open and the type of business of the store opening.

The server 1 reads the pedestrian information, the weather information, the surrounding information, etc. corresponding to the corresponding points from each database and inputs them into the evaluation model 142. For example, the server 1 aggregates the number of passersby in front of the store and the breakdown of the number of passersby by attribute, behavior, and belongings by time zone (at the time of imaging) as in the above-mentioned learning, and weather information and surrounding information. Etc. are sequentially input to each neuron in the input layer of the evaluation model 142.

From the evaluation model 142, the server 1 acquires predicted values related to store visitors, such as the number of store visitors to the store opening and the breakdown of the number of store visitors by attribute, as evaluation information. The server 1 outputs the evaluation information acquired from the evaluation model 142 to the terminal 4 and displays it as a simulation result.

For example, the terminal 4 receives the designated input of the month and the day of the week from the user, and displays the simulation result showing the evaluation information such as the predicted number of visitors on the designated month and the day of the week. For example, the terminal 4 displays the predicted value of the number of visitors per day and the attributes (age and gender) of the visitors who are predicted to have the largest number of visitors, and also displays the transition of the number of visitors by time zone. Display the graph shown.

Further, the terminal 4 is not only the information (predicted value) of the visitor who is expected to visit the store, but also a passerby other than the visitor, who passes in front of the store or around the store. Information about potential customers who are passersby is also displayed. A potential customer is a person who has not reached the store but has reached the state immediately before the store visit, and is a person who can potentially become a store customer. For example, the terminal 4 displays the attributes of the potential customer, the unit of the expected number of visitors to the store, and the like as the information of the potential customer.

For example, the server 1 refers to the pedestrian information stored in the pedestrian DB 144 and totals the number of pedestrians by attribute. Then, the server 1 calculates the number of passersby different from the visitor, that is, the number of potential customers who are not expected to come to the store, by attribute by subtracting the predicted value of the number of visitors from the number of passersby by attribute.

For example, the terminal 4 displays the attributes of the potential customer having the largest number of people, the unit of the number of potential customers visiting the store (the number of people visiting the store as a group), and the like. Further, the terminal 4 displays a graph showing the number of visitors and passersby by attribute. By presenting the potential customer information about the potential customer, it is possible to provide marketing reference information as to what kind of person the advertisement or the like is to be advertised.

Further, the server 1 not only outputs information such as visitors and potential customers of the store opening store, but also predicts an increase / decrease in the number of visitors of the peripheral stores existing in the vicinity of the store opening store and outputs the information to the terminal 4. For example, the server 1 aggregates and outputs the number of visitors to the surrounding stores with reference to the passerby DB 144, and inputs various data including the distance to the surrounding stores into the evaluation model 142 from the evaluation model 142. Acquires the predicted value of increase / decrease in the number of visitors to neighboring stores and outputs it to the terminal 4. The terminal 4 displays the number of visitors to the peripheral stores acquired from the server 1, and also displays the predicted value of increase / decrease in the number of visitors to the peripheral stores.

From the above, the server 1 inputs the passerby information collected by using the camera 2 attached to each utility pole into the evaluation model 142, acquires the evaluation information as an output, and distributes it to the terminal 4. As a result, marketing support such as store opening plans can be provided.

FIG. 8 is a flowchart showing a processing procedure of the learning process of pedestrian information. The contents of the machine learning process for generating the classification model 141 from the teacher data will be described with reference to FIG.
The control unit 11 of the server 1 acquires teacher data in which an image of a pedestrian passing through a road is associated with a correct answer value of a classification result when the pedestrians shown in the image are classified (step). S11). The teacher data is data in which information such as attributes such as gender and age, behavior, and belongings is labeled for each of one or more passersby in the image, for example.

The control unit 11 uses the teacher data to generate a classification model 141 (learned model) that outputs a classification result of classifying the passerby when the captured image of the passerby is input (step S12). Specifically, the control unit 11 inputs the captured image, which is the teacher data, to the input layer of the neural network, and acquires the classification result of classifying the passersby in the captured image from the output layer. The control unit 11 compares the acquired classification result with the correct answer value of the teacher data (information labeled for the captured image), and makes the classification result output from the output layer closer to the correct answer value in the intermediate layer. Optimize the parameters (weights, etc.) used for arithmetic processing. The control unit 11 stores the generated classification model 141 in the auxiliary storage unit 14, and ends a series of processes.

FIG. 9 is a flowchart showing a processing procedure of the pedestrian analysis process. Based on FIG. 9, the content of the process of analyzing (classifying) passersby using the classification model 141 will be described.
The control unit 11 of the server 1 acquires an image of a pedestrian passing through the road from a camera 2 provided on a utility pole installed along the road (step S31). Specifically, the control unit 11 acquires moving images of the road continuously captured from the cameras 2, 2, 2, ... Corresponding to the utility poles installed at each of the plurality of points along the road. In addition, the control unit 11 acquires the date and time when the image was captured, the weather information at each point at each imaging time, and the like (step S32).

The control unit 11 inputs the image acquired from the camera 2 into the classification model 141, and acquires the passerby information indicating the classification result of classifying the passersby reflected in the image from the classification model 141 (step S33). Specifically, the control unit 11 inputs each frame image constituting the moving image acquired from each camera 2 into the classification model 141, and classifies the passersby imaged at each imaging time point and at each point. To get. For example, the control unit 11 acquires a classification result obtained by classifying passers-by by attributes such as gender and age, behavior including entrance / exit movement to / from a facility, and belongings. The control unit 11 stores the pedestrian information indicating the classification result of the pedestrian in the pedestrian DB 144 (step S34) in association with the imaging time and the imaging point, and ends a series of processes.

FIG. 10 is a flowchart showing a processing procedure of the learning process of store evaluation. The contents of the machine learning process for generating the evaluation model 142 will be described with reference to FIG.
The control unit 11 of the server 1 acquires the existing store information used for the teacher data of machine learning (step S51). The existing store information is information about the existing store existing along the road (public space), and includes the actual measurement value about the visitor who visited the existing store in addition to the location information of the existing store. The measured value regarding the number of visitors may include, for example, the number of visitors by time zone and the breakdown of the number of visitors by attribute.

Further, the control unit 11 stores passerby information obtained from images captured by each camera 2 installed in the surrounding area of the existing store, weather information at the time of capturing the captured image, peripheral information related to the surrounding area, and the like. Read from (step S52). Peripheral information is data showing the surrounding conditions of the store, for example, statistical data such as the population of the trade area, the size of the trade area, the number of passengers getting on and off at the nearest station, geographical data such as the distance to the nearest station, and the surroundings of the existing store. It may include data of nearby stores located in. The data of neighboring stores is, for example, the distance from existing stores.

The control unit 11 uses the existing store information acquired in step S51, the passerby information acquired in step S52, and the like as teacher data, and outputs an evaluation model 142 that outputs the evaluation information of the store when the passerby information is input. Generate (step S53). Specifically, the control unit 11 is passerby information obtained from images captured by one or a plurality of cameras 2 corresponding to the corresponding points where the existing store is located, and is captured at each time zone (at the time of imaging). Passerby information obtained from the obtained image is input to the input layer of the neural network. In addition, the control unit 11 inputs weather information for each time zone, peripheral information of existing stores, and the like to the input layer. The control unit 11 acquires, for example, a predicted value regarding a visitor who has visited an existing store from the output layer as evaluation information of the existing store. Predicted values for store visitors are, for example, the number of store visitors by time zone, the breakdown of store visitors by attribute, and the like. The control unit 11 compares the predicted value related to the store visitor with the measured value related to the visitor of the existing store, which is the correct answer value, and performs arithmetic processing in the intermediate layer so that the predicted value output from the output layer approaches the correct answer value. Optimize the parameters (weights, etc.) used for. The control unit 11 stores the generated classification model 141 in the auxiliary storage unit 14, and ends a series of processes.

FIG. 11 is a flowchart showing a processing procedure of the store evaluation process. Based on FIG. 11, the content of the process of evaluating the stores to be opened at the relevant points using the evaluation model 142 will be described.
The control unit 11 of the server 1 receives input of location information indicating a corresponding point where a new store is to be opened, the type of business of the store to be opened, and the like via the terminal 4 (step S71). The control unit 11 reads out the pedestrian information, the weather information, the surrounding information, and the like corresponding to the corresponding points from each database (step S72). For example, the control unit 11 has pedestrian information acquired from images captured at each time zone (at the time of imaging) by one or a plurality of cameras 2 installed around the corresponding point, and weather information at each time zone. And read from the passerby DB 144. Further, the control unit 11 reads out the peripheral information of the area corresponding to the corresponding point from the peripheral information DB 145.

The control unit 11 inputs the pedestrian information or the like read out in step S72 into the evaluation model 142, and acquires the evaluation information that evaluates the stores that open at the corresponding points as output (step S73). Specifically, the control unit 11 inputs peripheral information into the evaluation model 142, and sequentially inputs passerby information corresponding to the captured image in each time zone (time point of imaging) into the evaluation model 142 in chronological order. Acquire the predicted value for the visitor in each time zone as evaluation information. Further, for example, the control unit 11 acquires not only the predicted value regarding the visitor to the store opening store but also the predicted value predicting the increase / decrease of the visitor to the peripheral store.

The control unit 11 is a potential customer regarding a potential customer who is a passerby who passes around the store and is a passerby other than the visitor, based on the evaluation information acquired in step S73 and the passerby information acquired in step S72. Information is generated (step S74). For example, the control unit 11 extracts the attributes of the most pedestrians among the pedestrians other than the visitors as potential customer information from the pedestrian information. The control unit 11 outputs the evaluation information acquired in step S73, the potential customer information generated in step S74, and the like to the terminal 4 (step S75), and ends a series of processes.

In the above, the number of visitors to the store is mentioned as the evaluation information of the store, but the present embodiment is not limited to this, and may be, for example, a predicted value such as a customer unit price or sales. That is, the evaluation information is not limited to the predicted value regarding the visitor.

Further, in the above, the server 1 performs both machine learning and processing using the trained model which is a product of machine learning, but the present embodiment is not limited to this. For example, the evaluation model 142 generated by the server 1 may be installed in the terminal 4, and the terminal 4 may perform the store evaluation. That is, the processing subject of machine learning and the processing subject of processing using the trained model may be separate hardware.

Further, in the above, the server 1 performs both the pedestrian analysis process and the store evaluation process, but the present embodiment is not limited to this, and the processing subject of each process may be separate.

From the above, according to the first embodiment, pedestrian information is collected based on the images captured by the camera 2 attached to each utility pole along the road, and the collected pedestrian information is input to the evaluation model 142 to store the store. The evaluation information of is acquired and presented to the user. This makes it possible to appropriately support marketing such as store opening plans.

Further, according to the first embodiment, it is possible to present predicted values related to the store visitor such as the number of store visitors to the store and the attributes of the store visitor, and to support marketing more appropriately.

Further, according to the first embodiment, not only the visitor who visits the store but also the potential customer who can be a potential visitor can be analyzed and presented to the user.

Further, according to the first embodiment, it is possible to evaluate the store in consideration of not only the information of the passersby but also the statistical or geographical data around the store and the information of the surrounding stores.

Further, according to the first embodiment, the evaluation can be performed in consideration of the influence of the opening of the store on the surrounding stores.

Further, according to the first embodiment, by using the classification model 141, passerby information can be analyzed accurately and automatically from the captured image, and a more appropriate store evaluation system can be constructed.

(Embodiment 2)
In this embodiment, not only the evaluation information for considering the pros and cons of opening a store is presented, but also it is checked whether or not the store planning to open a store is suitable for the expected visitor. The form will be described. The contents overlapping with the first embodiment are designated by the same reference numerals and the description thereof will be omitted.
FIG. 12 is an explanatory diagram showing an outline of the second embodiment. An outline of the present embodiment will be described with reference to FIG.

In the present embodiment, when the server 1 evaluates the store opening using the evaluation model 142, the server 1 includes the location information of the corresponding point where the store is scheduled to open, the type of business of the store opening, and the store information related to the store opening. To get. The store information is, for example, information regarding the layout of a store to be opened. The information regarding the layout is data indicating the quantity or arrangement of various layouts, for example, regarding seats, floor area, doorways, and the like.

Similar to the first embodiment, the server 1 inputs pedestrian information and the like into the evaluation model 142, and acquires evaluation information that evaluates the store opening store, such as a predicted value of the number of visitors to the store opening store. Then, the server 1 compares the evaluation information acquired from the evaluation model 142, that is, the predicted value regarding the visitor, with the above-mentioned store information, that is, the layout of the store, and the store planning to open the store is the predicted visitor. Estimate the degree of compatibility between the two to see if they are suitable for.

For example, the server 1 compares the predicted value of the number of visitors in each time zone with the number of seats, and counts the number of times the predicted value of the number of visitors exceeds the number of seats. As a result, the server 1 estimates the frequency of loss of store visit opportunities, that is, the degree of conformity between the store visitor and the store layout. For example, when the number of times counted is equal to or greater than a predetermined threshold value, the server 1 notifies the terminal 4 of the shortage of seats and proposes the expansion of the store scale to the user.

Further, when the above notification is given, the server 1 generates and outputs a modification plan of the store layout according to the estimated degree of conformity. In the case of the explanation according to the above example, the server 1 calculates the recommended increase number of seats by calculating the average value of the excess (difference) of the number of visitors to the number of seats, and informs the user. Notice. As a result, the user is presented with a plan as to how much the store scale should be expanded.

In addition to the number of seats, the server 1 may estimate a shortage of floor area, number of entrances and exits, etc. according to the number of visitors, and present correction plans for various objects. Further, the data to be targeted in estimating the degree of conformity is not limited to the quantity of various objects, and may be, for example, the arrangement position (coordinate value) of a seat, an entrance / exit, or the like.

Further, in the above description, the store layout has been described as an example for checking the degree of conformity, but for example, the server 1 targets the products or services to be provided at the store, and the attributes of the predicted visitor and the planned provision. The degree of conformity may be estimated by matching with products and the like. Further, for example, the server 1 may estimate the degree of conformity by comparing the planned number of employees, business hours, etc. with the predicted value of the visitor. As described above, the store information to be compared with the evaluation information is not limited to the information related to the layout of the store opening.

FIG. 13 is a flowchart showing a processing procedure of the store evaluation process according to the second embodiment.
The control unit 11 of the server 1 receives input of store information from the terminal 4 including information on the layout of the store to be opened, in addition to the location information of the corresponding point where the store is newly opened, the type of business of the store to be opened, and the like (step S201). ). The information regarding the layout is, for example, data indicating the quantity and arrangement of seats, floor area, doorways, etc. in the store. The control unit 11 shifts the process to step S72.

The control unit 11 acquires evaluation information indicating the predicted value of the number of visitors from the evaluation model 142 (step S73), generates potential customer information (step S74), and then executes the following processing. Based on the evaluation information acquired in step S73 and the store information acquired in step S201, the control unit 11 estimates the degree of conformity between the store visitor who is expected to visit the store and the store scheduled to open ( Step S202). For example, the control unit 11 has an appropriate number of seats, whether or not the floor area is insufficient, and the position of the entrance / exit based on the layout information included in the store information and the predicted value of the visitor included in the evaluation information. Estimate whether or not. Further, for example, the control unit 11 generates a correction plan for the store information according to the estimated degree of conformity. The control unit 11 outputs the estimated degree of conformity and the correction plan of the generated store information to the terminal 4 together with the evaluation information, the potential customer information, and the like (step S203), and ends a series of processes.

Based on the above, according to the second embodiment, by estimating the degree of conformity between the store information regarding the store to be opened and the visitor who is expected to visit the store, what kind of store should be opened is comprehensively included. Support can be provided.

Further, according to the second embodiment, more appropriate support can be provided by presenting a revision plan of the store opening plan.

Further, according to the second embodiment, it is possible to urge the user to review the store layout.

The embodiments disclosed this time should be considered to be exemplary in all respects and not restrictive. The scope of the present invention is indicated by the scope of claims, not the above-mentioned meaning, and is intended to include all modifications within the meaning and scope equivalent to the scope of claims.

1 Server (information processing device)
11 Control unit 12 Main storage unit 13 Communication unit 14 Auxiliary storage unit P program 141 Classification model 142 Evaluation model 143 Utility pole DB
144 Passerby DB
145 Peripheral information DB

Claims (12)

Based on the image of the relevant point in the public space from the image pickup device installed in the power distribution equipment installed in the public space, the passerby information obtained by analyzing the passersby passing through the relevant point at multiple time zones is acquired. And
If you enter the passerby information of each time zone, before Ki該evaluation information indicating the predicted value of the attribute a breakdown of the number of visit's to the store of another time zone store openings planned and該来shop's number to the person point Enter the acquired passerby information into the trained model that has been trained to output
The evaluation information is acquired from the trained model, and the evaluation information is obtained.
An information processing method characterized in that a computer executes a process of outputting the acquired evaluation information. Claim 1 is characterized in that, based on the evaluation information acquired from the trained model and the passerby information, potential customer information regarding a potential customer who is the passerby other than the visitor is generated and output. Information processing method described in. Acquire store information about the store and
The information processing method according to claim 1 or 2, wherein the degree of conformity between the visitor and the store is estimated based on the evaluation information acquired from the trained model and the store information. The information processing method according to claim 3, wherein a correction plan for the store information is generated and output based on the estimated degree of conformity. Acquire the store information regarding the layout of the store,
The information processing method according to claim 3 or 4, wherein the degree of conformity is estimated based on the evaluation information and the layout indicated by the store information. Obtain peripheral information about the surrounding situation of the relevant point,
The information processing method according to any one of claims 1 to 5, wherein the passerby information and peripheral information are input to the trained model to acquire the evaluation information. The peripheral information includes information on peripheral stores located in the vicinity of the relevant point.
The sixth aspect of claim 6, wherein the passerby information and the peripheral information are input to the trained model to obtain a predicted value for predicting an increase or decrease in the number of visitors to the peripheral stores due to the opening of the store. Information processing method. The image acquired from the imaging device is input to the first trained model that has been trained to classify the passersby when the image is input.
From the first trained model, the pedestrian information indicating the classification result of classifying the pedestrian is acquired.
The information processing method according to any one of claims 1 to 7, wherein the acquired pedestrian information is input to the second trained model to acquire the evaluation information. Based on the image of the relevant point in the public space from the image pickup device installed in the power distribution equipment installed in the public space, the passerby information obtained by analyzing the passersby passing through the relevant point at multiple time zones is acquired. And
If you enter the passerby information of each time zone, before Ki該evaluation information indicating the predicted value of the attribute a breakdown of the number of visit's to the store of another time zone store openings planned and該来shop's number to the person point Enter the acquired passerby information into the trained model that has been trained to output
The evaluation information is acquired from the trained model, and the evaluation information is obtained.
A program characterized in that a computer executes a process of outputting the acquired evaluation information. Based on the image of the relevant point in the public space from the image pickup device installed in the power distribution equipment installed in the public space, the passerby information obtained by analyzing the passersby passing through the relevant point at multiple time zones is acquired. Acquisition department and
If you enter the passerby information of each time zone, before Ki該evaluation information indicating the predicted value of the attribute a breakdown of the number of visit's to the store of another time zone store openings planned and該来shop's number to the person point An input unit for inputting the acquired passerby information into a trained model that has been trained to output
An evaluation unit that acquires the evaluation information from the trained model, and
An information processing apparatus including an output unit that outputs the acquired evaluation information. Based on the image of the relevant point in the public space from the image pickup device installed in the power distribution equipment installed in the public space, the passerby information obtained by analyzing the passersby passing through the relevant point at multiple time zones is acquired. And
To get the correct value of the evaluation information indicating the previous Ki該attribute the breakdown of the time zone of coming to the store's number and該来store's number of existing restaurants located to the skilled point,
When the passerby information is input based on the passerby information and the correct answer value of the evaluation information , the number of visitors to the store scheduled to open at the corresponding point corresponding to the passerby information and the number of visitors by time zone method for generating a learned model, characterized in that to execute a process of generating a learned model for outputting evaluation information indicating the predicted values of the demographic breakdown of該来shop's number to the computer. Accepted the position information indicating the corresponding point of the opening scheduled for the store, and the input of the store information relating to the shop to have you in the public space,
When the passerby information that analyzes the passersby passing through the relevant point is input based on the image obtained by capturing the relevant point from the image pickup device attached to the power distribution facility installed in the public space at each of a plurality of time zones. Obtained from a trained model that has been trained to output evaluation information indicating the number of visitors to the store scheduled to open at the relevant location by time zone and the predicted value of the breakdown of the number of visitors by attribute. The evaluation information is displayed on the display unit, and the evaluation information is displayed.
A program characterized in that a computer executes a process of displaying an estimation result of estimating the degree of conformity between the store and a visitor who visits the store based on the evaluation information and the store information on the display unit.

Images