JP5418005B2 - Tracking system and tracking method - Google Patents

Description

  The present invention relates to a tracking system and a tracking method for tracking a person who moves in a tracking area such as in a store.

  A tracking system for tracking a person who moves in a tracking area such as a store as follows has been proposed.

  For example, a technique for tracking a person by photographing the tracking area with one or a plurality of cameras and extracting a person image from the captured image has been proposed (see, for example, Patent Documents 1 to 4). . In the technique of tracking a person by shooting with a plurality of cameras, the movement path of the same person can be obtained individually from images shot by each camera. In this technique, these individually obtained movement paths are integrated into one movement path by, for example, an image processing method such as collating the characteristics of persons captured in each captured image.

  For example, in order to track a person, a technique has been proposed in which a person has a radio wave source such as a so-called RFID (Radio_Frequency_IDentification) tag (see, for example, Patent Documents 5 and 6). In the tracking area, the person is tracked by tracking the radio wave from the radio wave transmission source given to the person.

  By tracking the person in this way in the tracking area, it is possible to grasp the background of the person purchasing the product or the background of leaving without purchasing the product. Then, the background ascertained will be used for reference for product display, product development, etc. at a later date.

Japanese Patent Laid-Open No. 11-175597 Japanese Patent Laid-Open No. 2003-263641 JP 2005-309951 A International Publication No. 2005/111880 Pamphlet JP 2004-348681 A JP 2007-133461 A

  Here, in the technique of tracking a person by shooting with a camera, if there is a blind spot in shooting, the tracking may be interrupted in the middle. For this reason, in this technique, in many cases, photography is performed by as many cameras as possible so that blind spots cannot be taken. As a result, this technique has a problem that the installation cost for the camera, the processing cost for the captured image, and the like increase.

  Further, in the technique of tracking a person by tracking radio waves from a radio wave transmission source given to the person, there is a risk that accuracy may be lowered due to the influence of so-called multipath and fading. In addition, there is a problem that a person who does not have a radio wave transmission source cannot be tracked in the first place.

  In view of the above circumstances, an object of the present invention is to provide a tracking system and a tracking method capable of reliably tracking a person at a reduced cost.

  The basic form of the tracking system that achieves the above object includes a plurality of sensing units and mediation units.

  Each of the plurality of sensing units includes a detection unit, an identification information search unit, and a notification unit.

  A detection part detects the mobile body in a sensing area | region.

  The identification information search unit is configured to issue a search command for the identification information of the unknown mobile body when the unknown mobile body is detected by the detection unit.

  The notification unit notifies the identification information when the identification information is received in response to a search instruction for the identification information.

  The mediation unit is connected to the plurality of sensing units, receives an identification information search command from any of the sensing units, and issues a search command for the identification information to a sensing unit other than the sensing unit. Then, the mediation unit receives the notification of the identification information from a sensing unit other than any of the sensing units, and performs mediation to notify the identification information to any of the sensing units.

  Further, the basic form of the tracking method for achieving the above object has the following four processes.

  The basic form of this tracking method is that when any sensing unit of the plurality of sensing units detects an unknown moving body, any one of the sensing units issues a search command for identification information of the unknown moving body. A process of issuing to an intermediary unit connected to the plurality of sensing units.

  Further, the basic form of this tracking method is an identification information search process in which the intermediary unit receives the search command and issues a search command for the identification information to a sensing unit other than any of the sensing units that has issued the search command. Have

  Further, the basic form of this tracking method is that when a sensing unit other than any of the sensing units receives the search command from the mediation unit and holds the identification information of the unknown mobile body, the mediation A process of notifying the identification information of the unknown moving body to the unit.

  Furthermore, in the basic form of this tracking method, the intermediary unit receives the notification of the identification information of the unknown mobile body, and performs the search instruction for the identification information of the unknown mobile body. A notifying process for notifying the sensing unit.

  According to this case, it is possible to reliably track a person at a reduced cost.

It is a schematic diagram which shows specific embodiment of the tracking system demonstrated about the basic form. It is a block diagram which shows the functional block of the 1st-3rd sensing apparatus in the tracking system of FIG. It is a block diagram which shows the 4th sensing apparatus of each sensing apparatus in the tracking system of FIG. It is a figure which shows typically a mode that the flow line showing the movement path | route in a person's shop is calculated | required by the flow line calculation part of FIG. It is a schematic diagram which shows the search process of the identification information performed in the identification information search and provision part of FIG. It is a figure which shows the action dictionary of FIG. It is a figure which shows an example of the action record produced in the action history recording part of FIG. It is a figure which shows from step S101 to step S108 in the flowchart which shows the process until the action record about the person of identification information "A" is created. It is a figure which shows from step S109 to step S115 in the flowchart which shows the process until the action record about the person of identification information "A" is created. It is a figure which shows from step S201 to step S209 in the flowchart which shows the history until the action record about the person of identification information "B" is created. It is a figure which shows from step S210 to step S216 in the flowchart which shows the process until the action record about the person of identification information "B" is created.

  Hereinafter, specific embodiments of the tracking system and the tracking method described above for the basic mode will be described with reference to the drawings.

  FIG. 1 is a schematic diagram showing a specific embodiment of the tracking system described for the basic mode.

  A tracking system 100 shown in FIG. 1 is a system that executes tracking of a person in a store 110 that accepts and sells a mobile phone, and records a behavior of the person in the store 110.

  A person in the store 110 to be tracked in the tracking system 100 of the present embodiment corresponds to an example of the moving object in the basic mode described above.

  In addition, the moving body in this basic form is not limited to a person, but may be an object other than a person such as a vehicle that moves within a predetermined site.

  The tracking system 100 includes four types of first to fourth sensing devices 120,..., 150 installed in the store 110 and an action history recording device 160 installed outside the store 110. Each of the sensing devices 120,..., 150 corresponds to an example of a sensing unit in the basic form described above.

  The first sensing device 120 includes a camera 121 that keeps the entire area in the store 110 within the field of view, and a first control device 122 that performs position recording and the like for a person in a captured image obtained by the camera 121. In the present embodiment, the camera 121 repeatedly performs still image shooting in the store 110 at predetermined time intervals.

  The second sensing device 130 is operated by a person and issues a ticketing machine 131 that issues a number tag with a tag number indicating the order of acceptance, and second control that executes recording of the position of the person who operated the ticketing machine 131, etc. Device 132.

  The third sensing device 140 has a function of accepting payment, and performs a point-of-sale system (POS) 141 that aggregates sales results for each product, and records the position of the person who paid the price at the POS 141. And a third control device 142. The POS 141 of the third sensing device 140 is operated by a store clerk when a person pays.

  The fourth sensing device 150 is installed on the floor of the product display area 110a and includes three mat sensors 151N, 151P, and 151F that detect pressure when stepped on. These three mat sensors 151N, 151P, and 151F are installed at positions that can be stepped on by a person standing in front of the product shelf of each of the three types of products, product N, product P, and product F. Each mat sensor has a square mat shape of approximately 30 cm square which is a minimum required to be stepped on by one person. The fourth sensing device 150 includes a fourth control device 152 that is connected to each mat sensor and records the position of a person who has stepped on any of the mat sensors.

  The behavior history recording device 160 is a device that is connected to each sensing device and records the behavior of the person in the store 110 based on the position of the person detected by each sensing device.

  Hereinafter, the control device and the action history recording device 160 included in each sensing device will be described in detail.

  FIG. 2 is a block diagram showing functional blocks of the first to third sensing devices in the tracking system of FIG. FIG. 3 is a block diagram showing the fourth sensing device and the action history recording device 160 in the tracking system of FIG.

  First, the 1st sensing apparatus 120 provided with the camera 121 is demonstrated.

  The first control device 122 of the first sensing device 120 includes a moving body detection unit 122A, a position detection unit 122B, a flow line calculation unit 122C, a timing unit 122D, a sensing information storage unit 122E, and a communication unit 122F.

  The moving object detection unit 122A detects an image of the moving object in each captured image captured at predetermined time intervals by the camera 121 as an image of a person moving in the store 110. As a method for detecting an image of a moving object, known methods such as a background subtraction method, an optical flow, a particle filter, and the like can be cited, but here, there is no particular limitation.

  The position detection unit 122B detects the position corresponding to each captured image in the store 110 of a person who moves in the store 110 from the detection result obtained for each captured image. Here, in the present embodiment, the position of the person is detected by the two-dimensional coordinates of the center point of the person's image when the inside of the store 110 is viewed from the ceiling.

  Here, a combination of the camera 121, the moving body detection unit 122A, and the position detection unit 122B corresponds to an example of the detection unit in the basic mode described above.

  The flow line calculation unit 122C obtains a flow line representing the movement route of the person in the store 110 by sequentially connecting the two-dimensional coordinates detected by the position detection unit 122B for each captured image in time series.

  FIG. 4 is a diagram schematically illustrating a state in which a flow line representing a movement route of a person in a store is obtained by the flow line calculation unit in FIG.

The flow line calculation unit 122C in FIG. 2 first detects the two-dimensional coordinates (X _{i + 1} , Y _{i + 1} ) of the center point C _{i + 1} of the human image A _{i + 1} at a certain point in time when the position detection unit 122B detects the two-dimensional coordinates. The previous two-dimensional coordinates (X _i , Y _i ) to be connected to the dimensional coordinates (X _{i + 1} , Y _{i + 1} ) are recognized as follows. For example, when a plurality of persons are moving in the store 110, two-dimensional coordinates for each person are detected from the captured image. The flow line calculation unit 122C, among the two-dimensional coordinates detected from the previous photographed image, the two-dimensional coordinates existing in the vicinity of a certain person within the predetermined range of the current two-dimensional coordinates (X _{i + 1} , Y _{i + 1} ), It is adopted as the previous two-dimensional coordinates (X _i , Y _i ) of the person.

Then, the flow line calculation unit 122C connects the current two-dimensional coordinates (X _{i + 1} , Y _{i + 1} ) with the adopted previous (X _i , Y _i ). Such a process is executed each time an image is taken at a certain time interval, whereby a flow line L1 representing a movement path of one person is obtained. The flow line calculation unit 122C temporarily associates each two-dimensional coordinate belonging to the flow line L1 thus obtained with identification information for identifying the person acquired as described later in a memory (not shown). store.

  Here, in the present embodiment, as shown in FIG. 1, in the store 110, a range (a store entry / exit range) 111 a adjacent to the entrance 111 and through which a person entering or leaving the store must pass is set. Yes.

  Then, the flow line calculation unit 122C determines that the current two-dimensional coordinates are located in the entrance / exit passing range 111a and the previous two-dimensional coordinates associated with the current two-dimensional coordinates exist when determining the flow line L1. If not, the current two-dimensional coordinates are determined to be the position of the person who newly entered the store. When the flow line calculation unit 122C determines that the current two-dimensional coordinate is the position of the person who newly enters the store, the flow line calculation unit 122C notifies the sensing information storage unit 122E of the fact together with the current two-dimensional coordinate.

  Further, the flow line calculation unit 122C corresponds to the flow line L1 when there is no two-dimensional coordinate associated with the tip this time about the flow line L1 whose tip was previously in the entrance / exit passage range 111a. It is determined that the person with the identification information to be closed has left the store. Then, the flow line calculation unit 122C notifies the sensing information storage unit 122E of the determination result that the person of the identification information has left the store. However, in this case, the current two-dimensional coordinates do not exist. At this time, it is already known what identification information the flow line L1 corresponds to. For this reason, in this case, the determination result about the person leaving the store and the identification information of the person are notified to the sensing information storage unit 122E.

  In the present embodiment, as shown in FIG. 1, a range (a ticketing machine operation range) 131 a adjacent to the ticketing machine 131 and where a person operating the ticketing machine 131 is necessarily located is set in the store 110. Yes. Further, a range (POS payment range) 141a adjacent to the POS 141 in which a person who pays for the POS 141 is always located is set. Further, the flow line calculation unit 122C recognizes the range in which each of the three mat sensors 151N, 151P, and 151F is installed as a range in which a person who picks up a product corresponding to each mat sensor is located.

  In FIG. 4, the installation ranges of the ticket machine operating range 131a, the POS payment range 141a, and the three mat sensors 151N, 151P, and 151F are indicated by an abstract square region S.

  Then, the flow line calculation unit 122C determines that the flow line L1 has reached the square area S when the current two-dimensional coordinates are present in the square area S. The flow line calculation unit 122C makes such a determination on the installation ranges of the ticketing machine operation range 131a, the POS payment range 141a, and the three mat sensors 151N, 151P, and 151F. Then, the flow line calculation unit 122C notifies the sensing information storage unit 122E of the result of such determination together with the current two-dimensional coordinates.

  2 and 3 again, the description of the functional blocks of the tracking system 100 will be continued.

  The timing unit 122D of the first sensing device 120 shown in FIG. 2 indicates each shooting time when the camera 121 takes a still image at predetermined time intervals, and a detection time when the position of the person in the store 110 is detected. Measure as Then, the time measuring unit 122D stores the measured detection time until the next detection time is measured. In the present embodiment, the detection time measured and stored by the time measuring unit 122D is appropriately read out by the sensing information storage unit 122E as will be described later.

  Next, the sensing information storage unit 122E will be described.

  When the sensing information storage unit 122E is notified from the flow line calculation unit 122C that the current two-dimensional coordinates are the position of the person who newly entered the store, the sensing information storage unit 122E stores the new person in the action history recording device 160. The identification information to be identified is inquired using the communication unit 122F. Further, at the time of this inquiry, the sensing information storage unit 122E makes an inquiry with the determination result in the flow line calculation unit 122C, that is, the person related to this inquiry is a newly entered person. .

  Then, the identification information notified from the action history recording device 160 in response to this inquiry is transferred from the sensing information storage unit 122E to the flow line calculation unit 122C. The flow line calculation unit 122C temporarily stores the identification information in a memory (not shown) together with the two-dimensional coordinates sequentially connected after the current two-dimensional coordinates and forming the flow line as described above.

  Through the processing as described above, the flow line calculation unit 122C follows the flow line from when one person enters the store to when the person leaves the store while identifying the person.

  Here, in the store 110 of FIG. 1, the installation ranges of the ticket machine operating range 131a, the POS payment range 141a, and the three mat sensors 151N, 151P, and 151F are places where a plurality of persons are likely to get crowded. In such a place, it is assumed that a person is hidden behind another person's shadow and enters the blind spot of shooting by the camera 121, and the two-dimensional coordinates of the person in the blind spot cannot be detected temporarily. The In such a case, when two-dimensional coordinates of a person who has left the blind spot are detected later, the previous two-dimensional coordinates that should be linked to the two-dimensional coordinates at that time to form a flow line are calculated as a flow line. It becomes unknown to the part 122C. In other words, it is unclear to the flow line calculation unit 122C which person in the store 110 the current two-dimensional coordinates are.

  In the present embodiment, the flow line calculation unit 122C notifies the sensing information storage unit 122E together with the current two-dimensional coordinates when the previous two-dimensional coordinates are unknown within the various ranges described above. . Then, the sensing information storage unit 122E makes an inquiry to the behavior history recording device 160 for identification information for identifying a person located at the current two-dimensional coordinates using the communication unit 122F. In this inquiry, the sensing information storage unit 122E reads the current detection time from the time measuring unit 122D and notifies the action history recording device 160 of the current detection time and the current two-dimensional coordinates.

  Then, the identification information notified from the action history recording device 160 in response to this inquiry is transferred from the sensing information storage unit 122E to the flow line calculation unit 122C. The flow line calculation unit 122C temporarily stores the current two-dimensional coordinates in a memory (not shown) in association with the passed identification information. By such processing, the flow line calculation unit 122C resumes the tracking of the movement route for one person who has been interrupted.

  In addition, when the sensing information storage unit 122E is notified from the flow line calculation unit 122C that it is determined that the current two-dimensional coordinate is the position of the person who newly enters the store, as described above, this time Are associated with the identification information of the person and stored in a predetermined memory.

  In addition, the sensing information storage unit 122E, when notified from the flow line calculation unit 122C that the flow line of a certain person has reached the above various ranges such as the ticketing machine operation range 131a, the two-dimensional coordinates of this time are displayed. And stored in a predetermined memory in association with the identification information of the person.

  Furthermore, if the sensing information storage unit 122E is notified from the flow line calculation unit 122C that a certain person has left the store, the sensing information storage unit 122E stores that fact in the memory.

  Here, when the sensing information storage unit 122E stores the current two-dimensional coordinates or the fact that the person has left the store as described above in the memory, the time measurement unit 122D receives the current shooting time, that is, the current two-time coordinate. The detection time of the dimensional coordinate or the person's closing time is read out. Then, the sensing information storage unit 122E stores the read current detection time and store closing time in the memory in association with the current two-dimensional coordinates and the fact that the person has left the store.

  The sensing information storage unit 122E corresponds to an example of the identification information search unit in the basic form described above.

  In the present embodiment, when the sensing information storage unit 122E stores the two-dimensional coordinates and the detection time related to the store for a person with certain identification information, the person with the identification information enters the store at the detection time. Event information indicating that it has been generated is generated. This event information includes an event of detection of entry of a person and identification information of the person. Then, the sensing information storage unit 122E sends the generated event information to the action recording device 160 during the above storage.

  Further, even when the sensing information storage unit 122E stores the fact that the person with certain identification information has left the store and the store closing time, the event information indicating that the person with the identification information has left the store at the store closing time. Is generated. The event information includes an event of detecting the person leaving the store and identification information of the person. Then, the sensing information storage unit 122E sends the generated event information to the action recording device 160 during the above storage.

  Next, the 2nd sensing apparatus 130 provided with the ticket issuing machine 131 is demonstrated.

  The second control device 132 of the second sensing device 130 includes a tag number detector 132A, a timer unit 132B, a sensing information storage unit 132C, and a communication unit 132D.

  As described above, when the ticket issuing machine 131 is operated by a person, the ticket issuing machine 131 issues a number tag on which a bill number is written. Here, in the present embodiment, the ticket issuing machine 131 stores in advance two-dimensional coordinates for prediction of the standing position of the person when operating the ticket issuing machine 131. In the present embodiment, the center coordinates of the ticketing machine operation range 131a shown in FIG. 2 are adopted as the two-dimensional coordinates for the prediction. Then, when issuing the ticket, the ticket issuing machine 131 passes the two-dimensional coordinates stored in advance to the tag number detection unit 132A as a detection result of the position of the person operating the ticket issuing machine 131.

  The ticket issuing machine 131 also corresponds to an example of the detection unit in the basic form described above.

  The bill number detection unit 132A detects a bill number written on a number tag operated by a person and issued by the ticket issuing machine 131. Then, the bill number detection unit 132A passes the bill number and the two-dimensional coordinates delivered from the ticket issuing machine 131 as described above to the sensing information storage unit 132C.

  When the ticket issuing machine 131 is operated by a person, the timer unit 132B measures the operation time as the detection time when the position of the person is detected as described above. And the time measuring part 132B memorize | stores the measured detection time until the ticketing machine 131 is operated next and the operation time is measured.

  When the two-dimensional coordinates are passed from the tag number detection unit 132A, the sensing information storage unit 132C transmits identification information identifying the person who has operated the ticket issuing machine 131 at the two-dimensional coordinates to the action history recording device 160. Query using 132D. In this inquiry, the sensing information storage unit 132C reads the detection time from the time measuring unit 132B, and uses the detection time and the two-dimensional coordinates passed from the tag number detection unit 132A to the action history recording device 160. Notify

  When identification information for identifying the person is notified from the action history recording device 160 in response to this inquiry, the sensing information storage unit 132C performs the following storage process. That is, the sensing information storage unit 132C stores the two-dimensional coordinates passed from the tag number detection unit 132A, the detection time read from the time measuring unit 132B, and the notified identification information in the memory in association with each other.

  The sensing information storage unit 132C also corresponds to an example of the identification information search unit in the basic form described above.

  Furthermore, the sensing information storage unit 132C issues the number tag of the tag number detected by the tag number detection unit 132 at the time of the storage as described above. Event information indicating that the machine 131 has issued a ticket is generated. This event information includes a phenomenon of ticket number issuance, identification information of the person, and a tag number of the number tag. Then, the sensing information storage unit 132C sends the generated event information to the action recording device 160 at the time of the above storage.

  Next, the 3rd sensing apparatus 140 provided with POS141 is demonstrated.

  The third control device 142 of the third sensing device 140 includes a product detection unit 142A, a timer unit 142B, a sensing information storage unit 142C, and a communication unit 142D.

  As described above, the POS 141 has a function of accepting payment by a person, and totals sales results for each product. Here, in the present embodiment, the POS 141 stores in advance two-dimensional coordinates for prediction of a person's standing position when paying with the POS 141. In the present embodiment, the center coordinates of the POS payment range 141a shown in FIG. 2 are adopted as the two-dimensional coordinates of the prediction. Then, when accepting the price payment, the POS 141 passes the two-dimensional coordinates stored in advance to the product detection unit 142A as a detection result of the position of the person who pays the price.

  The POS 141 also corresponds to an example of the detection unit in the basic form described above.

  The product detection unit 142A detects the product name of the product for which the person paid the price at the POS 141. Then, the product detection unit 142A passes the product name and the two-dimensional coordinates passed from the POS 141 as described above to the sensing information storage unit 142C.

  When the payment is made at the POS 141, the time measuring unit 142B measures the time as the detection time when the position of the person is detected as described above. Then, the time measuring unit 142B stores the measured detection time until payment is made at the POS 141 and the time is measured.

  The sensing information storage unit 142C makes an inquiry to the behavior history recording device 160 using the communication unit 142D for identification information for identifying the person who is located at the two-dimensional coordinates passed from the product detection unit 142A and has paid the price. At the time of this inquiry, the sensing information storage unit 142C reads the detection time from the time measuring unit 132B and sends the detection time and the two-dimensional coordinates passed from the product detection unit 142A to the action history recording device 160. Notice.

  When identification information for identifying the person is notified from the action history recording device 160 in response to this inquiry, the sensing information storage unit 142C performs the following storage process. That is, the sensing information storage unit 142C stores the two-dimensional coordinates passed from the product detection unit 142A, the detection time read from the time measuring unit 142B, and the notified identification information in the memory in association with each other. .

  This sensing information storage unit 142C also corresponds to an example of the identification information search unit in the basic form described above.

  Furthermore, the sensing information storage unit 142C pays the identification information person with the POS 141 at the detection time for the product with the product name detected by the product detection unit 142A. Event information indicating that the process has been performed is generated. This event information includes an event of product purchase, identification information of the person, and the product name of the purchased product. Then, the sensing information storage unit 142C sends the generated event information to the action recording device 160 at the time of the above storage.

  Next, the fourth sensing device 150 including the three mat sensors 151F, 151P, and 151N illustrated in FIG. 3 will be described.

  The fourth control device 152 of the fourth sensing device 150 includes a pressed position detection unit 152A, a timing unit 152B, a sensing information storage unit 152C, and a communication unit 152D.

  The three mat sensors 151F, 151P, and 151N are turned on when a person steps on the mat sensors 151F, 151P, and 151N, and turned off when the person leaves. The mat sensor that is in the ON state among the three mat sensors is pressed to indicate that it is in the ON state at a predetermined time interval together with sensor identification information indicating which of the three mat sensors it is. Notify the detector 152A. Of the three mat sensors, the mat sensor that has changed from the ON state to the OFF state presses together with the sensor identification information that the mat sensor has been turned OFF after the above time interval has elapsed from the timing when the ON state was notified immediately before. The position detection unit 152A is notified.

  The pressed position detector 152A stores two-dimensional coordinates representing the installation positions of the three mat sensors 151F, 151P, and 151N. First, when it is notified that the sensor is in the ON state together with the sensor identification information, the pressed position detection unit 152A uses the two-dimensional coordinates representing the installation position of the mat sensor indicated by the sensor identification information as the mat sensor. It detects as the position of the person who is stepping on. In this case, the pressed position detection unit 152A includes a sensing information storage unit that indicates the two-dimensional coordinates representing the position of the person thus detected and that the mat sensor indicated by the sensor identification information is in the ON state. Notify 142C.

  When the sensor identification information notifies that the sensor is in the OFF state, the pressed position detection unit 152A detects a two-dimensional coordinate indicating the mat sensor installation position indicated by the sensor identification information. The two-dimensional coordinates represent the position of the person when the person leaves the mat sensor. In this case, the pressed position detection unit 152A stores the two-dimensional coordinates representing the position of the person thus detected and the fact that the mat sensor indicated by the sensor identification information is in the OFF state. Notify unit 142C.

  A combination of the three mat sensors 151F, 151P, 151N and the pressed position detection unit 152A also corresponds to an example of the detection unit in the basic mode described above.

  The timing unit 152B measures the time at which the mat sensor in the ON state notifies the pressed position detection unit 152A to that effect at predetermined intervals as the detection time at which position detection was performed for the person in the store 110. In addition, the time when the mat sensor that has changed from the ON state to the OFF state notifies the pressed position detection unit 152A to that effect is also measured as the detection time when the position of the person in the store 110 is detected. Then, the time measuring unit 152B stores the measured detection time until the next notification is made that any one of the mat sensors is in the ON state or the OFF state, and the detection time is measured.

  The sensing information storage unit 152C sends the communication unit 152D to the action history recording device 160 at the time when the first notification among the above notifications performed by the pressing position detection unit 152A for the ON mat sensor is performed at predetermined intervals. The following query is performed using That is, the sensing information storage unit 152C inquires of the action history recording device 160 about identification information for identifying a person located at the two-dimensional coordinates related to the first notification. At the time of this inquiry, the sensing information storage unit 152C reads the detection time related to the first notification from the time measuring unit 152B, and uses the detection time and the two-dimensional coordinates passed from the pressed position detection unit 152A as an action. The history recording device 160 is notified.

  When the identification information for identifying the person is notified from the action history recording device 160 in response to this inquiry, the sensing information storage unit 152C performs the following storage process. That is, the sensing information storage unit 152C stores the two-dimensional coordinates related to the first notification, the detection time read from the time measuring unit 152B, and the notified identification information in the memory in association with each other. In addition, regarding the two-dimensional coordinates and the detection times related to the second and subsequent notifications, the sensing information storage unit 152C performs storage associated with the identification information obtained by the above inquiry about the first notification. Such storage is repeatedly executed for each notification until the mat sensor that has been notified of being in the ON state is notified of being in the OFF state. When the mat sensor is notified that the mat sensor is in the OFF state and then is notified again that the mat sensor is in the ON state, the sensing information storage unit 152C executes the identification information inquiry once again. .

  The sensing information storage unit 152C also corresponds to an example of the identification information search unit in the basic form described above.

  Furthermore, the sensing information storage unit 152C stores the two-dimensional coordinates and the detection time related to the first notification and the two-dimensional coordinates and the detection time related to the notification that the sensor is in the OFF state. The following event information is generated.

  First, for the first notification, the sensing information storage unit 152C generates event information indicating that the mat sensor indicated by the sensor identification information is turned on by the person of the identification information at the detection time. This event information includes an event that the mat sensor indicated by the sensor identification information is in the ON state and the identification information of the person.

  Next, regarding the notification that the sensor has been turned off, the sensing information storage unit 152C indicates that the mat sensor indicated by the sensor identification information has been turned off by the person having the identification information at the detection time. Generate event information. This event information includes an event that the mat sensor indicated by the sensor identification information is in the OFF state and the identification information of the person.

  The sensing information storage unit 152C sends the event information generated at the time of the above storage about the first notification or the notification of the OFF state to the behavior recording device 160.

  Next, the action history recording device 160 will be described.

  The behavior history recording device 160 includes an identification information search / giving unit 161, a communication unit 162, a behavior detection unit 163, a behavior dictionary 164, and a behavior history recording unit 165.

  First, the inquiry about the identification information is received by the identification information search / giving unit 161 using the communication unit 162.

  Hereinafter, processing executed by the identification information search / giving unit 161 for this inquiry will be described.

  In response to an inquiry about identification information for identifying a new person, the identification information search / grant unit 161 sequentially changes the alphabet from “A” to “Z” in order from “A” each time such an inquiry is made. The first sensing device 120 is notified as identification information.

  In addition, the identification information search / grant unit 161 receives the identification information from the first sensing device 120 or any of the second to fourth sensing devices when resuming the acquisition of the flow line that has been interrupted. When there is an inquiry, the following identification information search process is executed.

  FIG. 5 is a schematic diagram showing identification information search processing executed by the identification information search / grant unit of FIG.

  In FIG. 5, as an example of the identification information search process, a search process when there is an inquiry for identification information from the second sensing device 130 is schematically shown.

  The identification information search / giving unit 161 that has received an inquiry about the identification information from the second sensing device 130 requests the sensing information storage unit of another sensing device to perform the following search process.

  As described above, when each sensing device inquires about identification information, the action history recording device 160 is notified of the two-dimensional coordinates detected for the person whose identification information is unknown and the detection time. Then, the identification information search / giving unit 161 requests each sensing information storage unit to perform a search process using a set of these two-dimensional coordinates and time as a search key.

  Whether each sensing information storage unit requested for the search processing stores in the memory a detection time that is close to a predetermined level to the detection time that belongs to the search key and a two-dimensional coordinate that is close to a predetermined level to the two-dimensional coordinates that belong to the search key Check for no. Then, the sensing information storage unit in which such detection time and two-dimensional coordinates are found notifies the identification information search / grant unit 161 of the identification information stored in association with these detection times and two-dimensional coordinates. To do.

In the example of FIG. 5, the identification information “B” corresponding to the two-dimensional coordinates (x _B , y _B ) corresponding to the set of the two-dimensional coordinates (xs, ys) belonging to the search key and the time ts and the time t _B is obtained. , Stored in the sensing information storage unit 122B of the first sensing device 120. The identification information “B” is obtained in the sensing information storage unit 122 </ b> B of the first sensing device 120 by the search process. Then, the identification information “B” obtained by the search is notified from the sensing information storage unit 122 </ b> B of the first sensing device 120 to the identification information search / grant unit 161. On the other hand, “NULL” information indicating that fact is notified to the identification information search / grant unit 161 from another sensing information storage unit that does not have a pair of two-dimensional coordinates and time corresponding to the search key.

  When the identification information search / giving unit 161 in FIG. 3 obtains the identification information in this way, the identification information is notified to the sensing device that has made the inquiry using the communication unit 162.

  When the first sensing device 120 makes an inquiry as described above when resuming the acquisition of a flow line that has been interrupted, the first sensing device 120 uses the person identified by the identification information as the flow line. The flow line acquisition is resumed after recognizing that it is. In addition, when the other second to fourth sensing devices make an inquiry about the identification coordinates of the person from whom the two-dimensional coordinates are detected, each sensing information storage unit stores the two-dimensional coordinates and the detection time of the two-dimensional coordinates. And the received identification information are stored in association with each other.

  In the present embodiment, by the above-described exchange of identification information, the behavior of a person in the store 110 is tracked in a state where each sensing device always knows who the person is.

  Said identification information search and provision part 161 is equivalent to an example of the mediation part in the above-mentioned basic form. Moreover, the sensing information storage part which each said sensing apparatus has also serves as an example of the notification part in the above-mentioned basic form.

  As described above, according to the present embodiment, even if a person is crowded at the three mat sensors 151N, 151P, 151F, etc. Lines and behavior records can be continued reliably. In the tracking system 100 according to the present embodiment, the blind spot of shooting by the camera 121 is supplemented by various sensors installed in a place where people are particularly crowded in the store 110. As a result, in the present embodiment, a configuration that causes an increase in cost such as installing a large number of cameras 121 or installing mat sensors at a large number of locations in the store 110 is avoided. From the above, according to the tracking system 100 of the present embodiment, it is possible to reliably track a person at a reduced cost.

  In the present embodiment, as a sensor for detecting the position of a person in the store 110, a plurality of types of sensors such as a sensor including the camera 121, a ticket issuing machine 131, a POS 141, a sensor including a mat sensor, and the like are employed. Has been. Thereby, it is possible to detect the positions of persons at various locations in the store 110 by an appropriate detection method according to the locations.

  This means that an application mode in which the plurality of sensing units include two or more sensing units having different detection methods of the moving body by the detection unit is preferable to the basic mode described above. doing.

  Each of the first to fourth sensing devices 120, 130, 140, and 150 corresponds to an example of a sensing unit in this application mode.

  In the present embodiment, two-dimensional coordinates representing the position of the person and a detection time when the two-dimensional coordinates are detected are used as search keys for searching for the identification information of the person. In the present embodiment, this makes a search with higher accuracy.

  This means that the application modes described below are preferable to the basic mode described above. In this application mode, the sensing unit issues a search command for the identification information of the moving object together with information on the position and detection time of the moving object detected by the detecting unit. Furthermore, in this application mode, when the sensing unit receives a search command for identification information of the mobile object, the sensing unit determines whether or not the identification information of the mobile object is held using the position and the detection time as a clue. It is supposed to be.

  Each sensing device of this embodiment is equivalent to an example of a sensing part in this application form.

  Further, in the tracking system 100 of the present embodiment, the flow line calculation unit 122C of the first sensing device 120 determines whether or not the person whose current two-dimensional coordinates have been detected has newly entered the store 110. judge. The flow line calculating unit 122C determines that the current two-dimensional coordinate is located in the entrance / exit passing range 111a and there is no previous two-dimensional coordinate associated with the current two-dimensional coordinate. Is determined to be the position of a newly entered person. The sensing information storage unit 122E makes an inquiry about the identification information of the person with the fact that the person is a newly entered person. Then, the identification information search / grant unit 161 gives new identification information to the person determined in this way, and notifies the first identification device 122 of the new identification information.

  In this embodiment, new identification information is surely given to a newly entered person who needs to be given new identification information by such simple processing.

  This means that the application modes described below are preferable to the basic mode described above. In this application mode, when the search command is a search command for identification information about a new mobile object whose identification information is not held in any sensing unit of the plurality of sensing units, the mediation unit, The following processing is executed. In such a case, the mediation unit, in addition to the mediation, further adds new identification information to the new mobile body and notifies the sensing unit that has issued the search command of the new identification information. Do.

  The identification information search / grant unit 161 of the present embodiment also corresponds to an example of an intermediary unit in this applied mode.

  In this embodiment, as an example of a detection unit that detects the position of a person in the store 110, a ticketing machine 131 operated by a person and a POS 141 operated by a store clerk when the person pays are adopted. Has been. Each of the ticket issuing machine 131 and the POS 141 stores in advance two-dimensional coordinates for prediction of the standing position of the person when operated by the person or the store clerk. When each of the ticket issuing machine 131 and the POS 141 is operated by a person or a store clerk, the two-dimensional coordinates of the prediction are output as a detection result of the position of the person.

  Thus, in this embodiment, since the apparatus installed as a part of the store function is used as a sensor for detecting the position of a person, the introduction cost of the tracking system 100 is suppressed.

  This means that the application modes described below are preferable to the basic mode described above. In this application mode, at least one sensing unit of the plurality of sensing units includes a device that receives an operation, and detects a moving body when the device receives the operation.

  Each of the second sensing device 130 having the ticketing machine 131 and the third sensing device 140 having the POS 141 corresponds to an example of “one sensing unit” in this application mode.

  In the present embodiment, a combination of the camera 121, the moving body detection unit 122A, and the position detection unit 122B is also employed as an example of a detection unit that detects the position of a person in the store 110. According to this configuration for detecting the position of a person as a moving body in the store 110 from an image taken by the camera 121, the movement path of the person can be tracked with high accuracy. In this embodiment, the accuracy of the entire tracking system 100 is improved by adopting such a high-precision sensor as one of the sensors for detecting the position of the person.

  This means that the application modes described below are preferable to the basic mode described above. In this application mode, at least one sensing unit among the plurality of sensing units has a camera, and detects a moving body from an image photographed by the camera.

  The 1st sensing part 120 which has the camera 121 grade of this embodiment is equivalent to an example of "one sensing part" in this application form.

  In the present embodiment, a combination of the three mat sensors 151F, 151P, 151N and the pressed position detection unit 152A is also employed as an example of a detection unit that detects the position of a person in the store 110. The mat sensor is a high-sensitivity sensor because the state transition surely occurs between the ON state and the OFF state when a person steps on or moves away. And the improvement of the sensitivity of the tracking system 100 whole is aimed at by the structure that 152 A of press position detection parts output the detection position about the position of the person as the detection position about the position of the mat sensor stepped on by the person.

  This means that the application modes described below are preferable to the basic mode described above. In this application mode, at least one sensing unit among the plurality of sensing units has a mat sensor installed on the floor, and the mat sensor detects when the moving body passes.

  The fourth sensing device 150 having the mat sensor or the like of the present embodiment also corresponds to an example of “one sensing unit” in this application mode.

  Next, operations of the behavior detection unit 163, the behavior dictionary 164, and the behavior history recording unit 165 included in the behavior history recording device 160 of FIG. 3 will be described.

  In the present embodiment, detection and recording of human behavior based on event information sent from each sensing device as described above is executed by these units.

  Event information from each sensing device is received by the behavior detection unit 163 using the communication unit 162. The behavior detection unit 163 extracts an event such as detection of a person entering the store from the received event information. Then, the behavior detection unit 163 searches the behavior dictionary 164 using the extracted event as a search key.

  In the behavior dictionary 164, various phenomena assumed to be included in the event information in each sensing device and behavior information representing the behavior of the person corresponding to each event are described in association with each other.

  FIG. 6 is a diagram showing the behavior dictionary of FIG.

  The behavior dictionary 164 is provided with an event description column 164A in which the various events described above are described, and an action description column 164B in which behavior information corresponding to each event is described.

  As shown in FIG. 6, for example, a sentence “entered a store” is associated as behavior information with an event of detection of a person entering the store.

  In addition, a sentence “I stood in front of the product F” is associated as action information with the event that the mat sensor F is in the ON state. As described with reference to FIG. 1, in the present embodiment, the three mat sensors 151N, 151P, and 151F are persons standing in front of the product shelves of the three types of products, product N, product P, and product F. It is installed in a position where it can be stepped on. That is, the phenomenon that the mat sensor F is in the ON state means that a person stands in front of the product shelf of the product F. For this reason, in the behavior dictionary 164 of FIG. 6, the sentence “I stood before the product F” is associated as behavior information with the event that the mat sensor F is in the ON state. For the same reason, in the behavior dictionary 164 of FIG. 6, the sentence “I have left the product F” is associated as behavior information with the event that the mat sensor F has changed from the ON state to the OFF state. .

  The behavior detection unit 163 in FIG. 3 extracts behavior information corresponding to the event from the behavior dictionary 164 by a search process using the event extracted from the event information as a search key. Then, the behavior detection unit 163 passes the extracted behavior information to the behavior history recording unit 165 together with the event information.

  The action history recording unit 165 first extracts person identification information and detection time from the event information passed from the action detection unit 163. When the event information is created by the second sensing device 130, the tag number of the number tag issued by the ticket issuing machine 131 is also extracted. If the event information is created by the third sensing device 140, the product name of the product for which payment has been made at the POS 141 is also extracted.

  Then, the action history recording unit 165 records the detection time and the action sentence indicating the action of the person at the detection time for each identification information extracted from the event information, that is, for each person in the store 110. This action sentence is a sentence in which the action history recording unit 165 appropriately edits the action information passed from the action detection unit 163 with a tag number or a product name extracted from the event information. The action history recording unit 165 sequentially performs such recording each time event information is sent from the action detection unit 163, thereby recording an action record representing a series of actions in the store 110 of each person. create.

  FIG. 7 is a diagram illustrating an example of an action record created by the action history recording unit of FIG.

  FIG. 7 shows action records created for each of the two persons schematically shown in FIG. Part (A) of FIG. 7 shows an action record 165A for a person whose identification information is “A”. In addition, part (B) in FIG. 7 shows an action record 165B for a person whose identification information is “B”.

  Each of the action records 165A and 165B has detection time description columns 165A_1 and 165B_1 in which detection times are described in descending order, and action sentence description columns 165A_2 and 165B_2 in which action sentences corresponding to the detection times are described. .

  FIG. 7 also shows information such as events and identification information extracted from the event information when creating the action sentence in order to make it easier to understand the process of creating each action sentence in the action sentence description columns 165A_2 and 165B_2. Has been.

  The action sentence corresponding to the detection time “10:25:00” in the action record 165A for the person with the identification information “A” has the action information “number ticket issued” as the action number “**”. It was created by editing. In addition, although not described in the example of FIG. 7, for example, the behavior information “product purchased” indicates that the product with the product name *** has been purchased with the product name “***”. Will be edited into an action sentence.

  As described above, in this embodiment, with the above event information generated each time the position of a person is detected by each sensing device as a clue, the action of the person at each detected position is sequentially described as the above action sentence. To be recorded. This makes it possible to obtain a series of action records of the person in the store 110, which is an important reference for product development in the future and product arrangement in the store 110.

  This means that the application modes described below are preferable to the basic mode described above. In this application mode, the sensing unit further includes an event output unit that generates event information corresponding to the position of the moving object detected by the detection unit and outputs the event information. In this application mode, the tracking system further includes an action dictionary, an action search unit, and an action history recording unit. In the behavior dictionary, the event information and behavior information representing the behavior of the moving body corresponding to the event information are described in association with each other. The behavior search unit searches for behavior information corresponding to the event information output from any one of the plurality of sensing units from among the behavior information described in the behavior dictionary. is there. The behavior history recording unit records the behavior information searched by the behavior search unit for each identification information of the moving body.

  The sensing information storage unit included in each sensing device of the present embodiment corresponds to an example that also serves as an event output unit in this application mode. Further, the behavior dictionary 164 of the present embodiment corresponds to an example of a behavior dictionary in this application mode. Further, the behavior search unit 163 of the present embodiment corresponds to an example of a behavior search unit in this applied mode. In addition, the action history recording unit 165 of the present embodiment corresponds to an example of an action history recording unit in this applied form.

  Next, the process from when each of the two persons in FIG. 1 enters the store until the two action records 165A and 165B in FIG. 7 are created will be described.

  FIG. 8 is a diagram showing steps S101 to S108 in the flowchart showing the process until the action record for the person with the identification information “A” is created. FIG. 9 is a diagram showing steps S109 to S115 in the flowchart showing the process until the action record for the person with the identification information “A” is created. The process shown in the flowcharts of FIGS. 8 and 9 corresponds to a specific embodiment of the tracking method described above for the basic form.

  First, when the first person enters the store and passes through the entrance / exit passage range 111a of FIG. 1, the first sensing device 120 detects the entry of the person (step S101).

  Next, an inquiry about the identification information of the person is made from the first sensing device 120 to the action history recording device 160, and the identification information from the identification information search / assignment unit 161 of the behavior history recording device 160 to the first sensing device 120. “A” is notified (step S102).

  In the first sensing device 120, first, the identification information “A” notified from the action history recording device 160 is given to the person whose entry has been detected as described above (step S103). At the same time, the sensing information storage unit 122E stores the two-dimensional coordinates representing the position of the person “A” at the time of entering the store and the detection time when the two-dimensional coordinates are detected. Further, event information including an event that the person “A” has detected a store entry is generated and sent from the first sensing device 120 to the action history recording device 160.

  In the behavior history recording device 160, the behavior detection unit 163 first obtains behavior information “entered a store” corresponding to the event of store detection for the person “A”. Then, the action history recording unit 165 creates an action sentence using the action information as it is, and records it as the first line of the action record 165A in FIG. 7 in association with the detection time in the event information (step). S104).

  On the other hand, in first sensing device 120, detection of a flow line representing the movement path of person “A” to which identification information is given as described above is started (step S105).

  Thereafter, when the flow line of the person “A” reaches the ticketing machine operation range 131a shown in FIG. 1, the two-dimensional coordinates indicating the position of the person “A” detected at that time and the detection time are obtained as a sensing information storage unit. It is stored by 122E (step S106).

  Then, when the person “A” operates the ticket issuing machine 131 to issue the number tag, the second sensing device 130 detects the two-dimensional coordinates and the tag number of the issued number tag (step S107). Then, an inquiry about the identification information of the person at the position indicated by the two-dimensional coordinates is made to the action history recording device 160 from the second sensing device 130. The inquiry about the person identification information to the action history recording device 160 in step S107 is issued to the intermediary unit connected to the plurality of sensing units, as described in the above-mentioned basic form. It corresponds to an example of “process”.

  Then, the identification information search / assignment unit 161 of the action history recording device 160 requests a sensing process other than the second sensing device 130 using the above two-dimensional coordinates and the detection time as a search key. (Step S108).

  The request for the search process executed in step S108 corresponds to an example of the identification information search process in the basic form described above.

  In the example of FIG. 8, the identification information “A” corresponding to the search key is obtained from the sensing information storage unit 122E of the first sensing device 120. Then, the first sensing device 120 notifies the identification information search / grant unit 161 of the identification information “A”. The notification of the identification information “A” to the identification information search / grant unit 161 executed in step S108 is “the process of notifying the unknown mobile unit of identification information to the intermediary unit” as described in the basic mode. Is equivalent to an example.

  In step S <b> 108, the identification information search / grant unit 161 further notifies the second sensing device 130 of the identification information “A” obtained in this way. The notification to the second sensing device 130 executed in step S108 corresponds to an example of the notification process in the basic mode described above.

  In the second sensing device 130, the sensing information storage unit 132C stores the two-dimensional coordinates and the detection time in association with the identification information “A” notified from the identification information search / grant unit 161 (step S109). ). Furthermore, the sensing information storage unit 132 </ b> C generates event information including an event of number tag issuance and a tag number “**”, and sends the event information to the action history recording device 160.

  The behavior detection unit 163 of the behavior history recording device 160 first obtains behavior information “number card issued” corresponding to an event of number card ticketing for the person “A”. Then, the action history recording unit 165 edits the action information using the tag number, and creates an action sentence “I have issued a ticket with the tag number **”. Then, the action history recording unit 165 records this action sentence as the second line of the action record 165A in FIG. 7 in association with the detection time in the event information (step S110).

  Thereafter, when the person “A” reaches the mat sensor “F” 151F in FIG. 1, the first sensing device 120 uses the two-dimensional coordinates representing the position of the person “A” detected at that time, the detection time, Is stored by the sensing information storage unit 122E (step S111).

  When the person “A” steps on the mat sensor “F” 151F, the fourth sensing device 150 detects two-dimensional coordinates (step S112). Then, an inquiry about the identification information of the person at the position indicated by the two-dimensional coordinates is made from the fourth sensing device 150 to the action history recording device 160. The inquiry about the identification information of the person to the action history recording device 160 in step S112 is also issued to the intermediary unit connected to the plurality of sensing units, as described in the above basic form. It corresponds to an example of “process”.

  Then, first, the identification information search / giving unit 161 of the action history recording device 160 executes a search processing request using the two-dimensional coordinates and the detection time as search keys (step S113).

  The request for search processing executed in step S113 also corresponds to an example of the identification information search process in the basic mode described above.

  In the example of FIG. 9, the identification information “A” corresponding to the search key is obtained from the sensing information storage unit 122E of the first sensing device 120. Then, the first sensing device 120 notifies the identification information search / grant unit 161 of the identification information “A”. The notification of the identification information “A” to the identification information search / grant unit 161 executed in this step S113 is also referred to as “the process of notifying the intermediate unit of the identification information of the unknown mobile body” as described in the basic mode. Is equivalent to an example.

  In step S113, the identification information search / grant unit 161 further notifies the fourth sensing device 150 of the identification information “A” thus obtained. The notification to the fourth sensing device 150 executed in step S113 also corresponds to an example of the notification process in the basic mode described above.

  In the fourth sensing device 150, the sensing information storage unit 152C stores the two-dimensional coordinates and the detection time in association with the identification information “A” notified from the identification information search / grant unit 161 (step S114). ). Furthermore, the sensing information storage unit 152C generates event information including an event that the mat sensor F is in the ON state, and sends the event information to the action history recording device 160.

  The behavior detection unit 163 of the behavior history recording device 160 first obtains behavior information “Standing in front of the product F” corresponding to an event that the mat sensor F is in the ON state for the person “A”. Then, the action history recording unit 165 creates an action sentence using the action information as it is, and records the action sentence as the third line of the action record 165A in FIG. 7 in association with the detection time in the event information. (Step S115).

  Through the series of processes described above, an action record 165A for the person “A” shown in Part (A) of FIG. 7 is created.

  Next, the process until the action record 165B for the person “B” shown in Part (B) of FIG. 7 is created will be described.

  FIG. 10 is a diagram showing steps S201 to S209 in the flowchart showing the process until the action record for the person with the identification information “B” is created. FIG. 11 is a diagram showing steps S210 to S216 in the flowchart showing the process until the action record for the person with the identification information “B” is created. The process shown in the flowcharts of FIGS. 10 and 11 also corresponds to a specific embodiment of the tracking method described above for the basic form.

  First, when the person with the identification information “B” enters the store and passes through the entry / exit passage range 111a of FIG. 1, the first sensing device 120 detects the entry of the person (step S201).

  Next, an inquiry about the identification information of the person is made from the first sensing device 120 to the action history recording device 160, and the identification information from the identification information search / assignment unit 161 of the behavior history recording device 160 to the first sensing device 120. “B” is notified (step S202).

  In the first sensing device 120, first, the identification information “B” notified from the action history recording device 160 is given to the person whose entry has been detected as described above (step S203). At the same time, the sensing information storage unit 122E stores the two-dimensional coordinates representing the position when the person “B” enters the store and the detection time when the two-dimensional coordinates are detected. Furthermore, event information including an event that the person “B” has entered the store is generated and sent from the first sensing device 120 to the action history recording device 160.

  In the behavior history recording device 160, the behavior detection unit 163 first obtains behavior information “entered a store” corresponding to the event of store entry detection for the person “B”. Then, the action history recording unit 165 creates an action sentence using the action information as it is, and records it as the first line of the action record 165B in FIG. 6 in association with the detection time in the event information (step). S204).

  On the other hand, in first sensing device 120, detection of a flow line representing the movement path of person “B” to which identification information is given as described above is started (step S205).

  Thereafter, when the person “B” reaches the mat sensor “N” 151N of FIG. 1, the first sensing device 120 detects the position of the person “B” detected at that time, the detection time, Is stored by the sensing information storage unit 122E (step S206).

  In this example, as described in step S111 of the process of FIG. 9, the person “A” is on the mat sensor “F” 151F on the camera 121 side with respect to the mat sensor “N” 151N of FIG. Standing. As a result, in this example, the person “B” on the mat sensor “N” 151N is hidden behind the person “A” on the mat sensor “F” 151F when viewed from the camera 121. As a result, in the example here, the flow line detection in the first sensing device 120 using the image captured by the camera 121 is interrupted (step S207).

  On the other hand, when the person “B” steps on the mat sensor “N” 151N, the fourth sensing device 150 detects the two-dimensional coordinates (step S208). Then, an inquiry about the identification information of the person at the position indicated by the two-dimensional coordinates is made from the fourth sensing device 150 to the action history recording device 160. The inquiry about the person identification information to the action history recording device 160 in this step S208 is also issued to the intermediary unit connected to the plurality of sensing units, as described in the above-mentioned basic form. It corresponds to an example of “process”.

  Then, the identification information search / grant unit 161 of the action history recording device 160 requests another sensing device to perform a search process using the two-dimensional coordinates and the detection time as a search key (step S209).

  The request for search processing executed in step S209 also corresponds to an example of the identification information search process in the basic mode described above.

  In the example of FIG. 10, the identification information “B” corresponding to the search key is obtained from the sensing information storage unit 122E of the first sensing device 120. Then, the first sensing device 120 notifies the identification information search / grant unit 161 of the identification information “B”. The notification of the identification information “B” to the identification information search / grant unit 161 executed in step S209 is also referred to as “the process of notifying the intermediate unit of the identification information of the unknown mobile body” as described in the basic mode. Is equivalent to an example.

  In step S209, the identification information search / grant unit 161 further notifies the fourth sensing device 150 of the identification information “B” obtained in this way. The notification to the fourth sensing device 150 executed in step S209 also corresponds to an example of the notification process in the basic mode described above.

  In the fourth sensing device 150, the sensing information storage unit 152C stores the two-dimensional coordinates and the detection time in association with the identification information “B” notified from the identification information search / grant unit 161 (step S210). ). Furthermore, the sensing information storage unit 152C generates event information including an event that the mat sensor N is in the ON state, and sends the event information to the action history recording device 160.

  The behavior detection unit 163 of the behavior history recording device 160 first obtains behavior information “Standing in front of the product N” corresponding to the event that the mat sensor N is in the ON state for the person “B”. Then, the action history recording unit 165 creates an action sentence using the action information as it is, and records the action sentence as the second line of the action record 165B in FIG. 7 in association with the detection time in the event information. (Step S211).

  Thereafter, when the person “B” leaves the mat sensor “N” 151N, the mat sensor “N” 151N is turned off (step S212). At that time, the fourth sensing device 150 generates event information including an event that the mat sensor “N” 151N is turned off and the identification information “B”. The generated event information is sent from the fourth sensing device 150 to the action history recording device 160.

  Then, the behavior detection unit 163 of the behavior history recording device 160 first obtains behavior information “I have left the product N” corresponding to an event that the mat sensor N is in an OFF state for the person “B”. Then, the action history recording unit 165 creates an action sentence using the action information as it is, and records the action sentence as the third line of the action record 165B in FIG. 7 in association with the detection time in the event information. (Step S213).

  Further, when the person “B” is separated from the mat sensor “N” 151N as described above, the person “B” enters the field of view of the camera 121, and the two-dimensional representing the position of the new person is obtained. Coordinates are detected (step S214). However, in the first sensing device 120, it is unclear who the person who newly entered the field of view of the camera 121 and has detected two-dimensional coordinates is. Therefore, the behavior history recording device 160 is inquired of the behavior history recording device 160 from the first sensing device 120 about the identification information of the person at the position indicated by the two-dimensional coordinates. The inquiry about the person identification information to the action history recording device 160 in step S214 is also issued to the intermediary unit connected to the plurality of sensing units, as described in the above-mentioned basic form. It corresponds to an example of “process”.

  Then, the identification information search / grant unit 161 of the action history recording device 160 executes a search process request using the two-dimensional coordinates and the detection time as a search key (step S215).

  The request for the search process executed in step S215 also corresponds to an example of the identification information search process in the basic form described above.

  In the example of FIG. 11, the identification information “B” corresponding to the search key is obtained from the sensing information storage unit 152 </ b> C of the fourth sensing device 150. The fourth sensing device 150 notifies the identification information search / grant unit 161 of the identification information “B”. The notification of the identification information “B” to the identification information search / grant unit 161 executed in step S215 is “the process of notifying the intermediary unit of the identification information of the unknown mobile unit” as described in the basic mode. Is equivalent to an example.

  In step S215, the identification information search / grant unit 161 further notifies the first sensing device 120 of the identification information “B” obtained in this way. The notification to the first sensing device 120 executed in step S215 also corresponds to an example of the notification process in the basic form described above.

  In the first sensing device 120, first, identification information “B” notified from the action history recording device 160 is given to a person who has newly entered the field of view of the camera 121 as described above (step S216). ). Then, by adding the identification information “B”, the detection of the flow line of the person “B” is resumed.

  Through the series of processes described above, the action record 165A for the person “B” shown in Part (B) of FIG. 7 has no problem even though the flow line detection for the person “B” is temporarily interrupted. Will be created.

  In the above, as an embodiment of the tracking system described for the basic form, detection of a person using a camera is mainly used, and a temporary interruption of the detection is detected using another ticketing machine, a POS, or a mat sensor. The form of making up with is illustrated. However, the tracking system described for the basic mode is not limited to this mode. The tracking system described for the basic form is, for example, a form in which a mat sensor is laid on the main part of a floor in a store, and detection at a place where the mat sensor is not installed is supplemented by another sensor such as a camera. Also good.

  Also, in the above, as an embodiment of the tracking system that describes the basic form, a form in which a ticketing machine, a POS, or a product shelf that is expected to be crowded with people is adopted as a part that supplements detection of a person using a camera. Was illustrated. However, the tracking system described for the basic mode is not limited to this mode. The tracking system described with respect to the basic form may be, for example, a form in which detection in the shade of a pillar that is a blind spot for photographing by a camera is supplemented by another sensor such as a mat sensor.

  Moreover, in the above, the example which has the camera 121, the ticket-issuing machine 131, POS141, and the three mat sensors 151F, 151P, and 151N was shown as an example of the sensing part in the above-mentioned basic form. However, the sensing unit in the basic form described above is not limited to these. The sensing unit in this basic form may include, for example, an acceleration sensor, an RFID tag, a pulse sensor, an infrared sensor, a proximity sensor, a microphone, and the like. The acceleration sensor is incorporated in an exhibition product or a product shelf and detects the position of the person by detecting the acceleration when the person picks up the product. The RFID tag is a radio wave transmission source that transmits radio waves constantly or periodically. In the example using the RFID tag, the person is provided with the RFID tag, and the position of the person is detected by detecting the radio wave from the RFID tag. The pulse sensor detects the position of the person by detecting the pulse of the person when the person approaches. The infrared sensor detects the position of the person by detecting the blocking of the infrared ray by the person. The proximity sensor detects the position of the person by detecting the proximity of the person.

  Moreover, in the above, as an example of the event information and the behavior information in the behavior dictionary in the type of application that records the behavior sentence representing the behavior of the person, the entrance and exit of the person, the operation of the ticket issuing machine 131 and the POS 141, the mat The example which concerns on ON / OFF of a sensor, etc. was shown. However, the event information and the behavior information in the behavior dictionary in the above application form are not limited to these examples. As the event information and the behavior information in the behavior dictionary in this application mode, for example, the following explanation can be considered as an example when the above-described acceleration sensor is adopted as the sensor. In an example in which this acceleration sensor is employed, various detection waveforms detected by the acceleration sensor affixed to the product are listed as event information. Each event information, that is, action information corresponding to each detected waveform includes various operation states corresponding to each detected waveform such as opening / closing of the lid, left / right swing, and the like.

  Hereinafter, the following additional remarks are disclosed regarding various forms including the basic form described above.

(Appendix 1)
A detection unit that detects a moving body in the sensing region, an identification information search unit that issues a search command for identification information of the unknown moving body when an unknown moving body is detected by the detection unit, and A plurality of sensing units having a notification unit that notifies the identification information when the identification information is held in response to a search command;
Connected to the plurality of sensing units, receives a search command for identification information from any one of the sensing units, issues a search command for the identification information to a sensing unit other than any of the sensing units, and A tracking system, comprising: an intermediary unit that receives a notification of the identification information from a sensing unit other than the sensing unit and notifies the identification information to any one of the sensing units.

(Appendix 2)
The tracking system according to claim 1, wherein the plurality of sensing units include two or more sensing units having different detection methods of the moving object by the detection unit.

(Appendix 3)
In the case where the search command is a search command for identification information about a new mobile object whose identification information is not held in any sensing unit of the plurality of sensing units, the mediation unit, in addition to the mediation, Further, the tracking system according to appendix 1 or 2, wherein the new identification information is given to the new mobile body, and the new identification information is notified to the sensing unit that has issued the search command.

(Appendix 4)
The sensing unit further includes an event output unit that generates event information corresponding to the position of the moving object detected by the detection unit and outputs the event information.
The tracking system further comprises:
A behavior dictionary in which the event information and behavior information representing the behavior of the moving body according to the event information are described in association with each other;
An action search unit that searches for action information corresponding to the event information output from any of the plurality of sensing units from among the action information described in the action dictionary;
The tracking system according to any one of supplementary notes 1 to 3, further comprising: an action history recording unit that records the action information searched by the action search unit for each identification information of the moving object.

(Appendix 5)
The sensing unit performs a search command for identification information of the moving body together with information on the position and detection time of the moving body detected by the detection unit, and
When the sensing unit receives a search command for identification information of a moving object, the sensing unit determines whether the identification information of the moving object is held based on the position and the detection time. 5. The tracking system according to any one of 1 to 4.

(Appendix 6)
Any one of Supplementary notes 1 to 5, wherein at least one sensing unit of the plurality of sensing units includes a device that receives an operation, and detects a moving body when the device receives the operation. The tracking system described in the section.

(Appendix 7)
The tracking system according to any one of appendices 1 to 6, wherein at least one sensing unit of the plurality of sensing units includes a camera and detects a moving body from an image captured by the camera. .

(Appendix 8)
Any one of Supplementary notes 1 to 7, wherein at least one sensing unit of the plurality of sensing units includes a mat sensor installed on a floor, and the mat sensor detects when the moving body passes. A tracking system according to claim 1.

(Appendix 9)
When any sensing unit of the plurality of sensing units detects an unknown moving body, the one sensing unit connects a search command for identification information of the unknown moving body to the plurality of sensing units. The process of issuing to the mediation department
An identification information search process in which the intermediary unit receives the search command and issues a search command for the identification information to a sensing unit other than the sensing unit that has performed the search command,
When the sensing unit other than any one of the sensing units receives the search command from the mediation unit and holds the identification information of the unknown mobile body, the identification information of the unknown mobile body in the mediation unit The process of notifying
A notification process in which the intermediary unit receives a notification of the identification information of the unknown moving body and notifies the identification information to any of the sensing units that have performed a search instruction for the identification information of the unknown moving body; A tracking method characterized by comprising:

DESCRIPTION OF SYMBOLS 100 Tracking system 110 Store 110a Goods display area 111 Entrance 111a Passing range at the time of entrance / exit 120 First sensing device 121 Camera 122 First control device 122A Moving body detection unit 122B Position detection unit 122C Flow line calculation unit 122D, 132B, 142B, 152B Timing unit 122E, 132C, 142C, 152C Sensing information storage unit 122F, 132D, 142D, 152D, 162 Communication unit 130 Second sensing device 131 Ticketing machine 131a Ticketing machine operation range 132 Second controller 132A Bill number detection unit 140 3 sensing device 141 POS
141a POS payment range 142 Third control device 142A Product detection unit 150 Fourth sensing device 151F, 151P, 151N Matt sensor 152 Fourth control device 152A Pressed position detection unit 160 Action history recording device 161 Identification information search / granting unit 163 Behavior detection Part 164 Action dictionary 164A Event description column 164B Action description column 165 Action history recording unit 165A, 165B Action record 165A_1, 165B_1 Detection time description column 165A_2, 165B_2 Action sentence description column

Claims (5)

A detection unit that detects a moving body in the sensing region, an identification information search unit that issues a search command for identification information of the unknown moving body when an unknown moving body is detected by the detection unit, and in response to the search instruction have a notification unit that notifies the identification information to the case of holding the identification information, camera, mat sensor, a plurality of sensing unit including a ticket dispenser,
Connected to the plurality of sensing units, receives a search command for identification information from any one of the sensing units, issues a search command for the identification information to a sensing unit other than any of the sensing units, and A tracking system, comprising: an intermediary unit that receives a notification of the identification information from a sensing unit other than the sensing unit and notifies the identification information to any one of the sensing units.   In the case where the search command is a search command for identification information about a new mobile object whose identification information is not held in any sensing unit of the plurality of sensing units, the mediation unit, in addition to the mediation, 2. The tracking system according to claim 1, further comprising: adding new identification information to the new moving body and notifying the new identification information to a sensing unit that has issued the search command. The sensing unit further includes an event output unit that generates event information corresponding to the position of the moving object detected by the detection unit and outputs the event information.
The tracking system further comprises:
A behavior dictionary in which the event information and behavior information representing the behavior of the moving body according to the event information are described in association with each other;
An action search unit that searches for action information corresponding to the event information output from any of the plurality of sensing units from among the action information described in the action dictionary;
The tracking system according to claim 1, further comprising: an action history recording unit that records the action information searched by the action search unit for each identification information of the moving body. The sensing unit performs a search command for identification information of the moving body together with information on the position and detection time of the moving body detected by the detection unit, and
The sensing unit, when receiving a search command for identification information of a mobile object, determines whether the identification information of the mobile object is held by using the position and the detection time as a clue. Item 4. The tracking system according to any one of Items 1 to 3. When any sensing unit of a plurality of sensing units including a camera, a mat sensor, and a ticketing machine detects an unknown moving body, the sensing unit searches for identification information of the unknown moving body. Process to the mediation unit connected to the plurality of sensing units,
An identification information search process in which the intermediary unit receives the search command and issues a search command for the identification information to a sensing unit other than the sensing unit that has performed the search command,
When the sensing unit other than any one of the sensing units receives the search command from the mediation unit and holds the identification information of the unknown mobile body, the identification information of the unknown mobile body in the mediation unit The process of notifying
A notification process in which the intermediary unit receives a notification of the identification information of the unknown moving body and notifies the identification information to any of the sensing units that have performed a search instruction for the identification information of the unknown moving body; A tracking method characterized by comprising:

Images