JP7423904B2 - Information output device, method, and program - Google Patents

Description

The disclosed technology relates to an information output device, an information output method, and an information output program.

In retail stores, losses due to shoplifting have become a major problem, and experts such as security personnel are hired to prevent shoplifting. Efforts have also begun to automatically detect suspicious persons using image recognition technology, in order to reduce the burden of visual inspection by experts and prevent them from being overlooked.

For example, a person search device has been proposed that extracts information about a person included in a video captured by an imaging device and displays it on a screen. This device searches for a corresponding person based on conditions set in advance for a storage means that stores person characteristic information for each time-series image included in the video, and stores information corresponding to the search results. Extract by means. Then, a screen is generated and displayed to display preset human characteristics or actions corresponding to the extracted person, or patrol position information of a security robot equipped with an imaging means, and information displayed on the screen is generated and displayed. output to an external device.

In addition, there is a face recognition verification device that performs face recognition verification processing to match the image of the person captured by the camera with a pre-stored image of the person to be cautioned, and a server connected to the face recognition verification device via a network. A crime prevention system comprising a device has been proposed. In this system, if the image of the person captured by the camera matches the stored image of the person to be cautioned through facial recognition matching processing, the face recognition matching device sends the captured image of the matching person to the server device. do. Further, upon receiving the image transmitted by the face authentication collation device, the server device sends a push notification to the pre-registered smartphone via the network that the person to be cautioned has been detected. Furthermore, information regarding the person to be cautioned about, including the captured image of the person transmitted from the face recognition verification device, is provided.

In addition, a shoplifting prevention system is provided that can effectively prevent shoplifting by increasing the accuracy of shoplifting detection without causing a rise in running costs, and without the risk of privacy invasion. This system takes pictures of people who enter the store, identifies them based on their gait and clothing characteristics, and selects suspicious people based on the data captured by the camera. Determine whether or not the person is the one who committed the action, and identify the suspicious person. Then, the identified person is tracked, and before the tracked person leaves the store, the tracked person's willingness to pay is confirmed.

In addition, in places where an unspecified number of people visit, robots can recognize the people who are the target of services and provide information and support to visitors depending on the situation, as well as provide security services, evacuation guidance in case of fire, etc. Systems have been proposed that allow services to be provided. This system consists of a humanoid robot equipped with an imaging device, a speech device, a voice recognition device, etc., an action recognition sensor that recognizes the actions of an unspecified number of humans, which consists of an imaging device, a floor sensor, a voice recognition device, etc., and a data It consists of a processing device. In this system, a robot is moved to a position where the active behavior of a service candidate selected from an unspecified number of people using a behavior recognition sensor can be determined. Then, data for determining active behavior is acquired by the robot sensor, and compared with behavioral pattern data of the service recipient prepared in advance, it is determined whether the service candidate is the service recipient or not, and the service is provided.

JP2010-257451A Japanese Patent Application Publication No. 2015-130155 Unexamined Japanese Patent Publication No. 2016-57908 Japanese Patent Application Publication No. 2008-142876

When automatically calling out or displaying a message on a display to a suspicious person automatically detected from an image to prevent shoplifting, etc., it is necessary to do so without causing discomfort to other customers. There is.

However, in the conventional technology, if there are other people around the person to whom information is being output, such as a call or display, it may not be possible to appropriately convey to whom the information is being output. For example, when a voice message addressed to a target person is output, if there are other people near the target person, it is not easy to determine which person the message was sent to. Therefore, if the person to whom the person should address cannot determine that he or she has been addressed, the effect of the address will not be obtained. In addition, for people other than the target person, since the person is addressed unrelated to that person, they hear a voice whose intention is unclear, which makes them feel uncomfortable.

With the conventional technology for determining whether or not a service target candidate is a service target, if there is a person near the service target, it is clear which person the system is providing the service to. It may not be possible. Furthermore, in stores and the like, the aisles are narrow, making it difficult for the robot to reliably approach the person receiving the service, and as mentioned above, the person receiving the service is likely to be unclear.

As one aspect, the disclosed technology aims to output information in such a way that it can be easily recognized that the information is being output to a specific target person.

As one aspect, the disclosed technique includes an extraction unit that extracts feature information including appearance features of a plurality of people detected from a captured image. The disclosed technology also provides a method for identifying a first person to whom information is to be outputted among the plurality of people detected from the captured image based on the feature information extracted by the extraction unit. has a department. Furthermore, the disclosed technique provides information including, among the appearance features extracted by the extraction unit, an appearance feature that the first person specified by the identification unit has and that the second person does not have. It has an output section that outputs. The second person is a person other than the first person among the plurality of people detected from the captured image.

One aspect is that information can be output in such a way that it can be easily recognized that the information is being output to a specific target person.

1 is a diagram showing a schematic configuration of an information output system according to the present embodiment. FIG. 2 is a functional block diagram of an information processing device. It is a diagram showing an example of a person DB (Database). FIG. 6 is a diagram for explaining output of a voice message including appearance characteristics possessed only by a target person. It is a diagram showing an example of an output information DB. FIG. 1 is a block diagram showing a schematic configuration of a computer that functions as an information output device according to the present embodiment. 3 is a flowchart illustrating an example of feature extraction processing. It is a flowchart which shows an example of target person identification processing. It is a flowchart which shows an example of a call-out process. FIG. 3 is a diagram for explaining handing over of personal information between imaging devices.

Hereinafter, an example of an embodiment according to the disclosed technology will be described with reference to the drawings. In this embodiment, an example will be described in which an information output device according to the disclosed technology is applied to automatically call out to a suspicious person in a store.

As shown in FIG. 1, the information output system 100 according to this embodiment includes an information output device 10, an imaging device 32, and an output device 34. A plurality of each of the imaging devices 32 and the output devices 34 are installed in the store, and are connected to the information output device 10 via a network. The imaging device 32 outputs an image captured in an imaging range (range indicated by a broken line in FIG. 1). The output device 34 is, for example, a directional speaker, and is configured so that the range of output audio (hereinafter referred to as the "output range", indicated by the dashed line in FIG. 1) is limited to a specific range. Output audio to. As shown in FIG. 1, each of the imaging device 32 and the output device 34 is arranged so that the output range of the output device 34 is included within the imaging range of the imaging device 32.

Functionally, the information output device 10 includes an extraction section 12, a specification section 14, and an output section 16, as shown in FIG. Further, a person DB (Database) 22 and an output information DB 24 are stored in a predetermined storage area of the information output device 10.

When the video output from the imaging device 32 is input, the extraction unit 12 decomposes the video into captured images in units of frames, and detects a person from each captured image. A general method such as DPM (Deformable Part Models) can be used to detect a person. The extraction unit 12 uses the captured images input before the current time and the tracking results of the person to associate the person detected from the past captured image with the person detected from the current captured image. This allows you to track people. For example, KCF (Kernelized Correlation Filters) can be used to track a person.

The extraction unit 12 assigns a person ID that is the same as the person ID assigned to the past person to a person who has been matched with a person detected from a past captured image, that is, a person who has been able to be tracked. do. Further, the extraction unit 12 assigns a new person ID to a person who could not be associated with a person detected from a past captured image, that is, a person who could not be tracked.

Furthermore, the extraction unit 12 extracts appearance characteristics and state characteristics for each person detected from the captured image. Note that in the following description, appearance features and state features are simply referred to as "feature amounts" when they are described without distinction.

Appearance characteristics are features related to the person's appearance that can be recognized from the surroundings, such as the person's gender, type of clothing, color of clothing, presence or absence of a hat, presence or absence of glasses, and presence or absence of a beard. The extraction unit 12 extracts appearance features by classifying a person's belongings into classes using a method such as deep learning, for example.

The state feature is a feature indicating the state of the person, such as the movement of the person, such as the speed of movement, direction of movement, and direction of the face, and the position of the person. For example, the extraction unit 12 extracts a three-dimensional image calculated based on the coordinate position of a predetermined point of the person (for example, a point at the feet) on the captured image and the position and orientation of the imaging device 32 that have been calibrated in advance. The x and y coordinates of the world coordinate system are extracted as the position of the person. Further, the extraction unit 12 extracts the moving speed and moving direction from changes in the position of the person in the captured images at multiple times. Further, the extraction unit 12 extracts a person's face from within the area on the captured image of the detected person using the Viola-Jones method or the like. Then, the extraction unit 12 learns face images at various angles in advance, compares the face image extracted from the captured image with the learned result, and extracts the closest angle as the face orientation.

Note that the appearance feature is a feature used to distinguish the person to be addressed, which will be described later, from surrounding people, and the appearance feature and state feature are features used to specify the person to be addressed. In this embodiment, the target person is, for example, a suspicious person who is likely to commit a fraudulent act such as shoplifting or robbery. In the case of a suspicious person, characteristics tend to appear in the color and shape of clothing, belongings and their positions, direction of the face, etc., so it is sufficient to extract characteristic quantities that can identify these characteristics as appearance characteristics. Note that the appearance characteristics and state characteristics are not limited to the examples of this embodiment, and other characteristics may be adopted.

The extraction unit 12 stores the feature amount extracted for each person detected from the captured image in the person DB 22. FIG. 3 shows an example of the person DB 22. In the example of FIG. 3, the person DB 22 includes "object identification results," "appearance features," and "state features" in association with "person ID," which is identification information for each person. It is stored as personal information. A plurality of "state characteristics" are stored as "data 0", "data 1", . . . for each time. “Target identification result” is a result of identification by the identification unit 14, which will be described later, as to whether or not the person is a target person.

The person detected from the currently captured image is matched with the person detected from the previous captured image through tracking. Therefore, if the same person ID is already stored in the person DB 22, the extraction unit 12 converts the currently extracted feature amount into data obtained by incrementing the maximum data number of the stored feature amounts for the same person ID by one. Store it in association with the number. Furthermore, when a new person ID is assigned, the extraction unit 12 creates a new line corresponding to the new person ID, associates it with data 0, and stores the extracted feature amount.

The specifying unit 14 specifies, based on at least one of the appearance characteristics and the state characteristics extracted by the extracting unit 12, a target person to whom information is to be output from among the people detected from the captured image. Specifically, the identifying unit 14 selects, as the target person, a person in which at least one of the position, face orientation, movement speed, and appearance characteristics of the person detected from the captured image satisfies predetermined conditions as characteristics of the target person. Identify.

As a specific example of identifying a suspicious person as a target person, for example, the identifying unit 14 determines, for each person information stored in the person DB 22, that the appearance characteristics are, for example, wearing black clothes, sunglasses, a mask, a hat, etc. It is specified by whether or not it matches a predetermined condition such as Further, for example, the identifying unit 14 may determine that the change in the position of the person for multiple times included in the state feature reciprocates within the same range, or that the change in the face direction included in the state feature occurs within a predetermined period of time. It can also be specified by whether it indicates that the vehicle has swung left and right a predetermined number of times or more. Further, the identifying unit 14 may identify a person who is looking in a direction other than the direction of movement (whose face is facing) as a suspicious person based on the temporal change in the direction of the face and the direction of movement.

As described above, in order to identify a target person by integrating feature amounts for a predetermined time, such as when identifying a target person based on temporal changes in state characteristics, the identifying unit 14 stores person information in the person DB 22. It may be possible to specify whether or not a person is a target person in order from a person for whom a predetermined period of time has elapsed since the person was targeted. Furthermore, the identification of the target person is performed at regular intervals because the accuracy improves as the amount of person information accumulated in the person DB 22 increases over time. Therefore, the identification unit 14 selects a person who has passed a certain period of time since the initial detection time ("time" of "data 0") among the people indicated by the person information stored in the person DB 22, or who has been identified since the previous identification process. The identification process may be performed by extracting a person for whom a certain period of time has elapsed. Whether a certain period of time has elapsed since the previous identification process can be determined by recording the time of the identification process for each person ID, and checking the elapsed time from that time.

The specifying unit 14 writes the result of specifying whether or not the person is a target person in the "target specifying result" column in association with each person ID in the person DB 22. For example, the identifying unit 14 writes "1" as the target identification result when the person indicated by the person ID is identified as the target person, and writes "0" when the person is not identified as the target person.

The output unit 16 controls the output device 34 to output a voice message calling out to the person when the person identified as the target person by the identifying unit 14 enters the output range. At this time, the output unit 16 outputs external appearance features extracted by the extraction unit 12 that are possessed by the target person identified by the identification unit 14 but not possessed by surrounding people around the target person. Include in call voice message. If there is no single appearance feature that only the target person has, a combination of appearance features that only the target person has may be included in the voice message for greeting. Furthermore, the output unit 16 identifies people other than the target person who are present in the output range into which the target person has entered, as peripheral people.

Specifically, the output unit 16 converts, among the person information stored in the person DB 22, the person information whose latest data of the feature value is within a certain period of time from the current time into the person information of the person currently in the store. Extract as. The output unit 16 extracts, from the extracted person information, person information for which the “target identification result” is “1”, that is, the “position” included in the state characteristics of the person information of the person identified as the target person. The output unit 16 compares the extracted position with an output range predetermined from the installation position of the output device 34, output intensity, etc., and determines whether the subject has entered the output range.

Further, the output unit 16 specifies, from the person information stored in the person DB 22, a person indicated by the person information whose output range includes "position" included in the latest status feature, as a person around the target person. Note that the output unit 16 calculates the distance between the target person and each person from the "position" included in the person information extracted as the person information of the person currently in the store and the "position" of the target person. However, persons whose distance is within a predetermined value may be identified as nearby persons.

Further, the output unit 16 may exclude a person who is expected to go out of the output range from the surrounding people during the period from the start to the end of the output of the voice message for calling out. Furthermore, the output unit 16 may add a person who is predicted to enter the output range to the surrounding people from the start to the end of the output of the voice message for calling out.

For example, the output unit 16 uses the moving speed determined from the time change of the "position" included in the person information, the "moving direction", etc. to calculate the position of each person several seconds from now until the end of the call. Predict. Then, the output unit 16 may determine whether to identify the person as a surrounding person by comparing the predicted position of each person with the output range or calculating the distance from the target person's position. .

As shown in FIG. 4, the output unit 16 compares the "appearance characteristics" included in the person information of the target person and surrounding persons (in the example of FIG. 4, persons A, B, and C), and selects only the target person. Identify the appearance characteristics of. In the example of FIG. 4, "coat (color)", "trousers (color)", and "gender" are used as appearance characteristics. Among these, "pants: black" and "gender: male" are not appearance features possessed only by the subject, but "jacket: red" is an appearance feature possessed only by the subject.

Further, if the output unit 16 cannot identify the appearance feature that only the target person has using only one type of appearance feature, the output unit 16 identifies the appearance feature that only the target person has using a combination of multiple types of appearance features. For example, in the example of FIG. 4, assume that person A also has the appearance characteristic of "coat: red." In this case, the appearance feature "coat: red" alone cannot distinguish the target person from person A, but if the appearance feature "coat: red" and "gender: male" only the target person It is a combination of appearance characteristics.

The output unit 16 generates a voice message for greeting based on the appearance feature or combination of appearance features that only the specified target person has and the information stored in the output information DB 24.

FIG. 5 shows an example of the output information DB 24. In the example of FIG. 5, the output information DB 24 includes a template table 24A and an insertion content table 24B. The template table 24A is a table that stores templates of voice messages depending on the purpose of the call. In the example of FIG. 5, a plurality of "templates" are stored in the template table 24A in association with "ID" which is identification information of the template. Contents corresponding to the appearance characteristics of the subject are inserted into the XXXXXXX portion of each template stored in the template table 24A. The insertion content table 24B stores "appearance characteristics" and "insertion contents" to be inserted into the XXXXXXX portion of the template in accordance with the appearance characteristics in association with each other.

The output unit 16 acquires the insertion contents corresponding to the external appearance characteristics possessed only by the specified target person from the insertion contents table 24B, inserts the acquired insertion contents into the template selected from the template table 24A, and then inserts the acquired insertion contents into the template selected from the template table 24A. Generate voice messages. Templates may be selected from the template table 24A at random, or may be selected according to a predetermined rule, such as selecting a template according to the location of the target person (for example, a sales floor).

For example, in the example shown in Figure 4 above, if "coat: red" is specified as an appearance characteristic that only the target person has, the message "Customers wearing red jackets, please be careful as the items on the shelves are easily breakable. ” message will be generated. In addition, if "coat: red" and "gender: male" are identified as a combination of appearance characteristics possessed only by the target person, "Male customers wearing red jackets, please contact the store staff if there is a product you are looking for." Please contact me.'' message is generated.

The output unit 16 specifies the output device 34 corresponding to the output range in which the target person exists, and outputs the generated voice message for greeting. As a result, the designated output device 34 outputs a voice message calling out to the user. As shown in Figure 4, the output voice message includes appearance characteristics that only the target person has, so the target person who listens to the voice message is aware that the output voice message is a message for him/herself. can be recognized. Additionally, surrounding people can recognize that the voice message is not addressed to them because the appearance features included in the voice message do not apply to them.

The information output device 10 can be realized, for example, by a computer 40 shown in FIG. The computer 40 includes a CPU (Central Processing Unit) 41, a memory 42 as a temporary storage area, and a nonvolatile storage section 43. The computer 40 also includes an input/output device 44 such as an input section and a display section, and an R/W (Read/Write) section 45 that controls reading and writing of data to and from a storage medium 49 . The computer 40 also includes a communication I/F (Interface) 46 connected to a network such as the Internet. The CPU 41, memory 42, storage section 43, input/output device 44, R/W section 45, and communication I/F 46 are connected to each other via a bus 47.

The storage unit 43 can be realized by an HDD (Hard Disk Drive), an SSD (Solid State Drive), a flash memory, or the like. An information output program 50 for causing the computer 40 to function as the information output device 10 is stored in the storage unit 43 as a storage medium. The information output program 50 includes an extraction process 52, a specific process 54, and an output process 56. Furthermore, the storage unit 43 includes an information storage area 60 in which information constituting each of the person DB 22 and the output information DB 24 is stored.

The CPU 41 reads the information output program 50 from the storage unit 43, expands it into the memory 42, and sequentially executes the processes included in the information output program 50. The CPU 41 operates as the extraction unit 12 shown in FIG. 2 by executing the extraction process 52. Further, the CPU 41 operates as the specifying unit 14 shown in FIG. 2 by executing the specifying process 54. Further, the CPU 41 operates as the output unit 16 shown in FIG. 2 by executing the output process 56. Further, the CPU 41 reads information from the information storage area 60 and develops each of the person DB 22 and the output information DB 24 in the memory 42 . Thereby, the computer 40 that has executed the information output program 50 will function as the information output device 10. Note that the CPU 41 that executes the program is hardware.

Note that the functions realized by the information output program 50 can also be realized by, for example, a semiconductor integrated circuit, more specifically, an ASIC (Application Specific Integrated Circuit).

Next, the operation of the information output system 100 according to this embodiment will be explained. While the information output system 100 is in operation, an image captured by the imaging device 32 is input to the information output device 10 . Then, in the information output device 10, the feature extraction process shown in FIG. 7, the target person identification process shown in FIG. 8, and the calling process shown in FIG. 9 are executed in parallel. Note that the feature extraction process, the target person identification process, and the calling process are examples of the information output method of the disclosed technology. Each of the feature extraction processing, target person identification processing, and calling out processing will be described in detail below.

First, the feature extraction process shown in FIG. 7 will be described.

In step S12, the extraction unit 12 determines whether a captured image has been acquired. If a video is input from the imaging device 32, the extraction unit 12 decomposes the input video into captured images in units of frames, obtains one frame worth of captured images, and proceeds to step S14. If no video is input, the determination in this step is repeated.

In step S14, the extraction unit 12 detects a person from the acquired captured image using a method such as DPM.

Next, in step S16, the extraction unit 12 uses the captured images input before the current time and the tracking results of the person to identify the person detected from the past captured image and the person detected from the current captured image. The person is tracked by associating it with the person. The extraction unit 12 assigns the same person ID as the person ID assigned to the past person to a person who has been associated with a person detected from a past captured image. Further, the extraction unit 12 assigns a new person ID to a person who cannot be associated with a person detected from a previously captured image.

Next, in step S18, the extraction unit 12 determines the appearance characteristics of each person detected from the captured image, such as the person's gender, type of clothing, color of clothing, presence or absence of a hat, presence or absence of glasses, and presence or absence of a beard. Extract.

Next, in step S20, the extraction unit 12 extracts, for each person detected from the captured image, the person's movement such as the person's moving speed, moving direction, and face orientation, and state characteristics such as the person's position.

Next, in step S22, the extraction unit 12 updates the person DB 22. Specifically, for the person who was able to be tracked, the extraction unit 12 uses the data number obtained by incrementing the largest data number of the stored feature amounts by one for the same person ID as the person ID assigned in step S14 above. The feature amounts extracted in steps S18 and S20 are stored in association with each other. Furthermore, for a new person, the extraction unit 12 creates a new row corresponding to the person ID assigned in step S14 above, and stores the feature amount extracted in steps S18 and S20 in association with data 0. do. The process then returns to step S12.

Next, the target person identification process shown in FIG. 8 will be explained.

In step S32, the specifying unit 14 selects one piece of person information stored in the person DB 22 in order from the beginning.

Next, in step S34, the specifying unit 14 determines whether a predetermined time has elapsed since the previous specifying process for the selected person information. If the predetermined time has elapsed, the process moves to step S36, and if the predetermined time has not elapsed, the process returns to step S32.

In step S36, the identifying unit 14 determines whether the person indicated by the selected person information is the target person based on whether the feature amount included in the selected person information satisfies predetermined conditions as the characteristics of the target person. Identify.

Next, in step S38, if the identifying unit 14 identifies in step S36 that the person indicated by the person information selected in step S32 is the target person, the identifying unit 14 associates the person with the corresponding person ID in the person DB 22. and write “1” in the “Target identification result” column. In addition, when the identifying unit 14 identifies that the person indicated by the person information selected in step S32 is not the target person, the identifying unit 14 sets “0” in the “target identification result” column in association with the corresponding person ID in the person DB 22. ”. The process then returns to step S32.

Next, the calling process shown in FIG. 9 will be explained.

In step S<b>42 , the output unit 16 selects, among the person information stored in the person DB 22 , person information whose latest feature value data is within a certain period of time from the current time, person information of a person currently in the store, In other words, it is extracted as personal information to be processed.

Next, in step S44, the output unit 16 selects one person's information from the person's information extracted in step S42.

Next, in step S46, the output unit 16 determines whether the "target identification result" of the person information selected in step S44 is "1", so that the person indicated by the selected person information is the target person ( Here, it is determined whether the person is a suspicious person or not. If the person is the target person, the process moves to step S48, and if the person is not the target person, the process moves to step S56.

In step S48, the output unit 16 compares the "position" included in the state characteristics of the person information selected in step S44 with the output range predetermined from the installation position and output intensity of the output device 34, and Determine whether or not has entered the output range. If the subject has entered the output range, the process moves to step S50, and if the subject has not entered the output range, the process moves to step S56.

In step S50, the output unit 16 specifies, from the person information extracted in step S42, the person indicated by the person information whose output range includes the "position" in the latest state feature, as a person around the target person. In addition, the output unit 16 excludes people who are predicted to go outside the output range from the surrounding people between the start and end of the output of the voice message for calling out, and excludes people who are predicted to go outside the output range, and excludes people who are predicted to enter the output range. Add the person to the surrounding people list.

Next, in step S52, the output unit 16 compares the "appearance features" included in the personal information of the target person and the surrounding people, and identifies the appearance features or combinations of appearance features that only the target person has.

Next, in step S54, the output unit 16 acquires insertion contents corresponding to the appearance characteristics possessed only by the target person identified in step S52, from the insertion contents table 24B of the output information DB 24. Further, the output unit 16 inserts the acquired insertion content into the template selected from the template table 24A, and generates a voice message for greeting. Then, the output unit 16 specifies the output device 34 corresponding to the output range in which the target person is present, and outputs the generated voice message for greeting. As a result, the designated output device 34 outputs a voice message calling out to the user.

Next, in step S56, the output unit 16 determines whether all the personal information extracted in step S42 has been selected. If there is unselected personal information, the process returns to step S44; if all the information has been selected, the process returns to step S42.

As explained above, according to the information output system according to the present embodiment, the information output device identifies the target person to whom the user is addressing, and the appearance that only the target person has and that other people around the target person do not have. The features are included in the voice message and output. Thereby, the target person who has heard the voice message can easily recognize that the output voice message is a message for him/herself, and the surrounding people can easily recognize that the output voice message is not a message for him/herself.

In addition, although the said embodiment demonstrated the example which calls out to a suspicious person by a voice message, it is not limited to this. For example, the information may be output to the target person by displaying a message on a display. In this case, the range in which the display can be viewed is set as the output range, and it is sufficient to determine whether the target person has entered the output range, identify people around the target person, and the like.

Furthermore, in the embodiment described above, the case where the target person is a suspicious person has been described, but the present invention is not limited to this. This embodiment is effective when it is desired to output information to a specific person in a place where an unspecified number of people are present. For example, a person who is considering products at a store or looking for a destination at a facility such as a station can be identified as a target audience using information such as movement and line of sight, and messages and guidance for sales promotion can be sent to them. You can also output messages such as

Further, in the above embodiment, FIG. 1 shows a case where a voice message calling out to a target person specified from the imaging range of one imaging device is output within the output range corresponding to the imaging range. , but not limited to. For example, as shown in FIG. 10, a person detected from an image captured by one imaging device is tracked in images captured by another imaging device based on the person's moving direction, moving speed, placement of the imaging device, etc. , person information stored in the person DB may also be taken over. In this case, it is possible to output the voice message calling out to the target person in an output range other than the output range corresponding to the imaging range in which the target person is detected. This makes it possible to adopt a more effective method for outputting information to the target person, such as outputting a voice message to greet the target person when the target person enters an output range with few surrounding people.

Further, depending on the positional relationship between the person and the imaging device, some external appearance features of the person may not be extracted, such as when the lower body of the person is hidden behind a shelf. In that case, only the appearance features that can be extracted from each of the target person and surrounding persons may be used to specify the appearance features that only the target person has.

Furthermore, the appearance characteristics to be used may be changed depending on the characteristics of the location where the output range is set. Specifically, in the case where the appearance features of many people existing in the output range are assumed in advance, appearance features other than those appearance features may be used. For example, in an output range set for a cosmetics department where it is assumed that there are many female customers, characteristics other than gender may be used as appearance characteristics. Furthermore, the appearance features to be used may be changed depending on the time of day.

Further, in the above embodiment, a mode has been described in which the information output program is stored (installed) in the storage section in advance, but the present invention is not limited to this. The program according to the disclosed technology can also be provided in a form stored in a storage medium such as a CD-ROM, DVD-ROM, or USB memory.

Regarding the above embodiments, the following additional notes are further disclosed.

(Additional note 1)
an extraction unit that extracts feature information including appearance features of a plurality of people detected from the captured image;
a specifying unit that specifies a first person whose information is to be outputted among the plurality of people detected from the captured image based on the feature information extracted by the extracting unit;
Among the appearance features extracted by the extraction unit, the first person identified by the identification unit has them, and among the plurality of people detected from the captured image, the first person an output unit that outputs information including appearance characteristics that are not possessed by a second person other than the person;
Information output device including.

(Additional note 2)
The information output device according to supplementary note 1, wherein the output unit outputs information including a combination of appearance characteristics that the first person has and the second person does not have.

(Additional note 3)
The information output device according to Supplementary Note 1 or 2, wherein the output unit outputs the information when the first person enters the range of the information to be output.

(Additional note 4)
The information output device according to any one of Supplementary Notes 1 to 3, wherein the output unit identifies a person existing within a range of output information as the second person.

(Appendix 5)
When the feature information includes a state feature indicating the position and movement of each of the plurality of people, the output unit determines the reachable range from the start to the end of outputting the information based on the state feature. A person who is expected to go outside is excluded from the second person, and a person who is expected to enter the reachable range from the start to the end of the output of the information is selected as the second person. The information output device described in additional note 4.

(Appendix 6)
When the feature information includes a state feature indicating the position and movement of each of the plurality of persons, the output unit may output a state feature indicating the position of the first person included in the state feature among the plurality of types of appearance features. The information output device according to any one of Supplementary Notes 1 to 5, which changes the appearance characteristics to be used accordingly.

(Appendix 7)
The appearance characteristics include at least one of the gender of the first person and the second person, the type of clothing, the color of clothing, the presence or absence of a hat, the presence or absence of glasses, and the presence or absence of a beard. The information output device according to any one of the items.

(Appendix 8)
The identifying unit is configured to identify, as the first person, a person in which at least one of the position, face orientation, moving speed, and appearance characteristics of the person detected from the captured image satisfies a predetermined condition as a characteristic of the first person. The information output device according to any one of Supplementary Notes 1 to 7 for identifying a person.

(Appendix 9)
Extract feature information including appearance features of multiple people detected from captured images,
Based on the extracted feature information, identify a first person from among the plurality of people detected from the captured image, and whose information is to be output;
Among the extracted appearance features, the identified first person has them, and among the plurality of people detected from the captured image, a second person other than the first person does not have them. An information output method in which a computer performs processing that includes outputting information including appearance characteristics.

(Appendix 10)
The information output method according to appendix 9, wherein information including a combination of appearance characteristics that the first person has and that the second person does not have is output.

(Appendix 11)
The information output method according to appendix 9 or 10, wherein the information is output when the first person enters the range of the information to be output.

(Appendix 12)
The information output method according to any one of Supplementary Notes 9 to 11, wherein a person existing within the reach of the output information is specified as the second person.

(Appendix 13)
When the characteristic information includes a state feature indicating the position and movement of each of the plurality of people, based on the state feature, it is possible for the plurality of people to go out of the reachable range from the start to the end of outputting the information. In supplementary note 12, the predicted person is excluded from the second person, and the person who is predicted to invade the reachable range is added to the second person from the start to the end of the output of the information. Information output method described.

(Appendix 14)
When the feature information includes a state feature indicating the position and movement of each of the plurality of persons, the plurality of types of appearance features are used depending on the position of the first person included in the state feature. The information output method according to any one of Supplementary notes 9 to 13, which changes external appearance characteristics.

(Appendix 15)
The appearance characteristics include at least one of the gender of the first person and the second person, the type of clothing, the color of clothing, the presence or absence of a hat, the presence or absence of glasses, and the presence or absence of a beard. The information output method according to any one of the items.

(Appendix 16)
A supplementary note specifying, as the first person, a person in which at least one of the position, face orientation, movement speed, and appearance characteristics of the person detected from the captured image satisfies predetermined conditions as characteristics of the first person. The information output method according to any one of Items 9 to Supplementary Note 15.

(Appendix 17)
Extract feature information including appearance features of multiple people detected from captured images,
Based on the extracted feature information, identify a first person from among the plurality of people detected from the captured image, and whose information is to be output;
Among the extracted appearance features, the identified first person has them, and among the plurality of people detected from the captured image, a second person other than the first person does not have them. An information output program that causes a computer to perform processing that includes outputting information including appearance characteristics.

(Appendix 18)
The information output program according to appendix 17, which outputs information including a combination of appearance characteristics that the first person has and that the second person does not have.

(Appendix 19)
The information output program according to appendix 17 or 18, which outputs the information when the first person enters the range of the information to be output.

(Additional note 20)
Extract feature information including appearance features of multiple people detected from captured images,
Based on the extracted feature information, identify a first person from among the plurality of people detected from the captured image, who is a target for outputting information;
Among the extracted appearance features, the identified first person has them, and among the plurality of people detected from the captured image, a second person other than the first person does not have them. A storage medium that stores an information output program for causing a computer to perform processing including outputting information including appearance characteristics.

10 Information output device 12 Extraction unit 14 Specification unit 16 Output unit 22 Person DB
24 Output information DB
24A Template table 24B Insertion content table 32 Imaging device 34 Output device 40 Computer 41 CPU
42 Memory 43 Storage unit 49 Storage medium 50 Information output program 100 Information output system

Claims (10)

an extraction unit that extracts feature information including appearance features of a plurality of people detected from the captured image;
a specifying unit that specifies a first person whose information is to be outputted among the plurality of people detected from the captured image based on the feature information extracted by the extracting unit;
Among the appearance features extracted by the extraction unit, the first person identified by the identification unit has them, and among the plurality of people detected from the captured image, the first person an output unit that outputs information including external appearance characteristics that are not possessed by a second person other than the person , to an output range included in the imaging range of the imaging device that captured the captured image ;
Information output device including. The information output device according to claim 1, wherein the output unit outputs information including a combination of appearance features that the first person has and the second person does not have. The information output device according to claim 1 or 2, wherein the output unit outputs the information when the first person enters the range of the information to be output. The information output device according to any one of claims 1 to 3, wherein the output unit specifies a person existing within a range of the output information as the second person. When the feature information includes a state feature indicating the position and movement of each of the plurality of people, the output unit determines the reachable range from the start to the end of outputting the information based on the state feature. A person who is predicted to go outside is excluded from the second person, and a person who is predicted to enter the reachable range from the start to the end of the output of the information is selected as the second person. 5. The information output device according to claim 4. When the feature information includes a state feature indicating the position and movement of each of the plurality of persons, the output unit may output a state feature indicating the position of the first person included in the state feature among the plurality of types of appearance features. The information output device according to any one of claims 1 to 5, wherein the appearance feature to be used is changed accordingly. The appearance characteristics include at least one of the first person's and the second person's gender, type of clothing, color of clothing, presence or absence of a hat, presence or absence of glasses, and presence or absence of a beard. 6. The information output device according to any one of 6. The identifying unit identifies, as the first person, a person in which at least one of the position, face orientation, moving speed, and appearance characteristics of the person detected from the captured image satisfies predetermined conditions as characteristics of the first person. The information output device according to any one of claims 1 to 7, which identifies a person as a person. Extract feature information including appearance features of multiple people detected from captured images,
Based on the extracted feature information, identify a first person from among the plurality of people detected from the captured image, and whose information is to be output;
Among the extracted appearance features, the identified first person has them, and among the plurality of people detected from the captured image, a second person other than the first person does not have them. An information output method in which a computer performs processing including outputting information including external appearance characteristics to an output range included in an imaging range of an imaging device that captured the captured image . Extract feature information including appearance features of multiple people detected from captured images,
Based on the extracted feature information, identify a first person from among the plurality of people detected from the captured image, and whose information is to be output;
Among the extracted appearance features, the identified first person has them, and among the plurality of people detected from the captured image, a second person other than the first person does not have them. An information output program for causing a computer to execute processing including outputting information including appearance characteristics to an output range included in an imaging range of an imaging device that has captured the captured image .