JP6531435B2 - INFORMATION PROCESSING APPARATUS, INFORMATION PROCESSING METHOD, AND PROGRAM - Google Patents

Description

  The present invention relates to an information processing apparatus, an information processing method, and a program.

  Conventionally, a person is detected based on image information captured by an imaging device, and a predetermined range of an image corresponding to the detected person size is extracted as a target image for face recognition processing, and face recognition processing is performed on the extracted target image There is known a technology for performing (see, for example, Patent Document 1).

JP, 2013-155924, A

  As described in Patent Document 1, by limiting the image range in which the face recognition process is performed, the processing time of the face recognition process can be shortened and the responsiveness can be improved. However, the technology described in Patent Document 1 is not efficient because it is necessary to detect a person each time before the face recognition process in order to limit the range of the target image of the face recognition process.

  An object of the present invention is to efficiently reduce the processing time of face recognition.

In order to solve the above-mentioned subject, the information processor of the invention according to claim 1 is,
Imaging means,
Face recognition means for performing face recognition on the captured image acquired by the imaging means;
Based on the history of face recognition results by the face recognition unit, the range of the captured image to be the target of face recognition by the face recognition unit and the frequency of the face recognition process for the range of the captured image to be the target of the face recognition are determined and a determination means for,
The determination means calculates, based on the history of face recognition results by the face recognition means, for each of the areas where the face is recognized by the face recognition means, the frequency with which the face is recognized from the areas is calculated. A range of a captured image to be subjected to face recognition by the face recognition unit such that each area in which a face is recognized by the face recognition is subject to face recognition by the face recognition unit at a frequency at which the face is recognized from that area; The frequency of face recognition processing for the range of the captured image to be subjected to the face recognition is determined .

  According to the present invention, it is possible to efficiently reduce the processing time of face recognition.

It is a block diagram showing functional composition of a digital signage device in this embodiment. It is a figure which shows schematic structure of the screen part of FIG. It is a figure which shows typically the captured image image | photographed with the imaging part of the digital signage apparatus installed in the store. It is a flowchart which shows the content output process performed by the control part of FIG.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following embodiments, the case where the information processing apparatus according to the present invention is applied to the digital signage apparatus 1 will be described as an example. The present invention is not limited to the illustrated example.

[Configuration of Digital Signage Device 1]
FIG. 1 is a block diagram showing the main control configuration of the digital signage device 1. FIG. 2 is a front view showing a schematic configuration of the screen unit of FIG. The digital signage device 1 is, for example, a device which is installed in a store or the like and outputs contents such as a product description to a customer.

  As shown in FIG. 1, the digital signage apparatus 1 includes a projection unit 21 for emitting image light of content, and a screen unit 22 for receiving the image light emitted from the projection unit 21 on the back and projecting the light on the front. There is.

First, the projection unit 21 will be described.
The projection unit 21 includes a control unit 23, a projector 24, a storage unit 25, a communication unit 26, and a clock unit 30. The projector 24, the storage unit 25, the communication unit 26, and the timer unit 30 are connected to the control unit 23 as shown in FIG. 1.

  The control unit 23 executes various programs stored in the program storage unit 251 of the storage unit 25 to perform predetermined calculations and control of each unit, and a central processing unit (CPU) and a memory serving as a work area at the time of program execution. (All are not shown) and. The control unit 23 executes content output processing to be described later in cooperation with the program stored in the program storage unit 251 of the storage unit 25. The control unit 23 functions as a face recognition unit and a determination unit.

  The projector 24 is a projection device that converts image data output from the control unit 23 into image light and emits the light toward the screen unit 22. The projector 24 performs display operation by individually turning on / off each tilt angle of a plurality of (for XGA, 1024 horizontal pixels × 768 vertical pixels) small mirrors arranged in an array, for example. A DLP (Digital Light Processing) (registered trademark) projector using a DMD (digital micro mirror device), which is a display element that forms an optical image by the reflected light, is applicable.

  The storage unit 25 is configured of an HDD (Hard Disk Drive), a non-volatile semiconductor memory, or the like. As shown in FIG. 1, the storage unit 25 is provided with a program storage unit 251. The program storage unit 251 includes a system program executed by the control unit 23, a processing program for executing various processes including content output processing to be described later, a face recognition program (face recognition engine), and a gender / age estimation program. Application programs such as (sex / age estimation engine), data required to execute these programs, and the like are stored.

Further, the storage unit 25 is provided with a content storage unit 252, a face recognition history storage unit 253, a face recognition setting storage unit 254, and the like.
The content storage unit 252 stores content data of content matching the combination of gender and age in association with the combination of gender and age. In addition, content data of general-purpose content (referred to as general-purpose content) corresponding to any gender and age is also stored. The content data is, for example, composed of moving image data composed of a plurality of frame images and audio data corresponding to each frame image.

  The face recognition history storage unit 253 stores and stores history information of face recognition results obtained by face recognition processing performed on the captured image acquired by the imaging unit 34. In the present embodiment, when face recognition processing is performed on the captured image acquired by the imaging unit 34, a face (referred to as a first time face) not recognized in the previous face recognition processing is recognized. The coordinates of the face area are stored as history information of face recognition results.

The face recognition setting storage unit 254 stores face recognition setting information indicating the range and frequency of a captured image to be subjected to face recognition by face recognition processing. Specifically, face recognition setting information indicating which range of the captured image area that can be acquired by the imaging unit 34 and how often the face recognition is performed is stored.
For example, as shown in FIG. 3, the area A of the captured image is at a frequency of 5 times in 10 times, the area B is at a frequency of 3 times in 10 times, and the area C is at a frequency of 1 in 10 times When face recognition is performed at a frequency of once in 10 times, the coordinates of each of areas A to D and the area (range) to be subjected to face recognition in 10 times of face recognition processing as in “ABACABABAD” The order is stored as face recognition setting information.
By default, face recognition setting information is stored in which face recognition is performed on the entire image of the captured image each time the captured image is acquired.

  In addition, the storage unit 25 stores, for example, the counting timing of history information stored in the face recognition history storage unit 253.

  The communication unit 26 includes a modem, a router, a network card, and the like, and communicates with an external device connected to a communication network such as a LAN.

  The clock unit 30 is configured by an RTC (Real Time Clock) or the like, acquires the current date and time, and outputs the current date to the control unit 23.

Next, the screen unit 22 will be described.
As shown in FIG. 2, the screen unit 22 is provided with an image forming unit 27 and a pedestal 28 for supporting the image forming unit 27.

  For example, the image forming unit 27 has a film-like Fresnel lens laminated on a single light transmitting plate 29 formed in a human shape such as an acrylic plate, which is disposed to be substantially orthogonal to the irradiation direction of the image light. It is a screen to which the film screen 291 for back projection and the film 292 printed beforehand are stuck. The projector 24 projects an image on the area to which the film screen 291 of the image forming unit 27 is attached. An output unit is configured by the image forming unit 27 and the projector 24 described above.

  The pedestal 28 is provided with a button-type operation unit 32, an audio output unit 33 such as a speaker for outputting audio, and an imaging unit 34.

  The operation unit 32 includes various operation buttons, detects a pressing signal of the operation button, and outputs the signal to the control unit 23.

The imaging unit 34 is an imaging unit that captures an image of a space facing the digital signage device 1 and generates a captured image. Although not shown, the imaging unit 34 includes a camera including an optical system and an imaging element, and an imaging control unit that controls the camera.
The optical system of the camera is directed in such a direction that the person in front of the digital signage device 1 can be photographed. The imaging device is, for example, an image sensor such as a charge coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS). Then, the imaging device converts the optical image having passed through the optical system into a two-dimensional image signal.
The operation unit 32, the audio output unit 33, and the imaging unit 34 are connected to the control unit 23, as shown in FIG.

[Operation of Digital Signage Device 1]
Next, the operation of the digital signage device 1 will be described.

  FIG. 4 shows a flowchart of the content output process performed in the digital signage device 1. The content output process is executed by the cooperation of the control unit 23 and the program stored in the program storage unit 251 when the digital signage device 1 is powered on.

First, the control unit 23 causes the image forming unit 27 and the audio output unit 33 to output content (step S1). Specifically, the control unit 23 reads the content data of the content to be output from the content storage unit 252, sequentially outputs the frame image to the projector 24, and causes the projector 24 to project the frame image. Further, the read audio data is output to the audio output unit 33 to output the audio of the content.
Note that, at the beginning of power-on, for example, general-purpose content is played back.

  Next, the control unit 23 causes the imaging unit 34 to perform imaging and acquires a captured image (step S2).

  Next, the control unit 23 reads face recognition setting information from the face recognition setting storage unit 254, and performs face recognition processing on the captured image acquired in step S2 based on the face recognition setting information (step S3). Specifically, the control unit 23 specifies a range to be a target of face recognition from the captured image acquired this time based on face recognition setting information, cuts out an image of the range specified from the captured image, and generates a face recognition program. hand over. Then, the face recognition program performs face recognition processing on the image in the extracted range to recognize the face of the person. The face recognition processing can be performed using a known image processing technology such as a method using a Haar-like feature described in JP 2012-53813 A, for example.

  Next, the control unit 23 determines whether or not a face has been recognized as a result of the face recognition process (step S4). If it is determined that the face could not be recognized (step S4; NO), the control unit 23 returns to step S1. Here, when content other than general purpose content is currently to be output, it is preferable to switch the output target to general purpose content.

On the other hand, when it is determined that the face has been recognized (step S4; YES), the control unit 23 determines whether the first face not recognized in the previous face recognition process is recognized (step S5). Here, the control unit 23 assigns an ID to the face recognized from the captured image and manages it in the memory, and adds the same ID to the captured image acquired thereafter until the face is no longer recognized. I manage. When a face not managed by ID is recognized, it is identified as the first face.
If it is determined that the first face is not recognized (step S5; NO), the control unit 23 returns to step S1.

  If it is determined that the first face is recognized (step S5; YES), the control unit 23 stores the coordinates of the first recognized face region in the face recognition history storage unit 253 as history information of the face recognition result ( Step S6). For example, upper left coordinates and lower right coordinates of a rectangular range circumscribing the first recognized face are stored as history information of face recognition results. By storing the history information when the first face is recognized, it is possible to prevent the history information for the same face (person) from being stored many times.

  Next, the control unit 23 reads out the gender / age estimation program, and estimates the gender and age of the recognized person (step S7). For estimation of gender and age based on face images, for example, known image processing techniques such as human attribute estimation techniques based on wrinkle features, stain features, and lip features described in JP-A-2012-53813 It can be done using

  Next, the control unit 23 switches the setting of the content to be output to the content corresponding to the gender and the age estimated in step S7 (step S8).

  Next, the control unit 23 determines whether the count timing of the face recognition history has come (step S9). If it is determined that the count timing of the face recognition history has not arrived (step S9; NO), the control unit 23 returns to the process of step S1 and outputs the set content.

  If it is determined that the count timing of the face recognition history has come (step S9; YES), the control unit 23 counts the history information stored in the face recognition history storage unit 253 (step S10). Specifically, the history information stored in the face recognition history storage unit 253 is summed up, and in the captured image region that can be acquired by the imaging unit 34, for each of the regions where the first face is recognized, Is counted and the frequency of face recognition is obtained from the area. When counting the number of times the face is recognized from the area for each area where the face is recognized, the coordinates of the areas do not have to completely match. For example, in the case where a plurality of partially overlapping areas exist in the history information, the plurality of areas may be handled together as one area, and the frequency may be acquired.

  Then, the control unit 23 determines the range and frequency of a captured image to be subjected to face recognition processing by face recognition processing based on the information acquired by aggregation, and sets the range and frequency in the face recognition setting storage unit 254 (step S11). Specifically, each area in which the first face is recognized is a target of face recognition so that it is a target of face recognition in the face recognition process at a frequency equivalent to the frequency of the face recognition in the past from that area. Set the range and frequency of the captured image. Then, the process returns to step S1.

  FIG. 3 is a view schematically showing a captured image captured by the imaging unit 34 of the digital signage device 1 installed in the store. In the store, the flow of traffic when the customer approaches the digital signage device 1 is limited to several lines. Therefore, when the area where the first face is recognized is summed up, it is assumed that it will be concentrated in a specific range. For example, in the case of the captured image area shown in FIG. 3, it is assumed that there is a high possibility that the first face is recognized in areas A and B at the back of the passage. In addition, it is assumed that a person who comes back from behind the digital signage device 1 is likely to be recognized in the C area. Therefore, in the present embodiment, an area where a face is recognized for the first time in the captured image area is added up every predetermined period (at every aggregation timing), and each recognized area is recognized as a face from that area. The range and frequency of a captured image to be subjected to face recognition are determined so that the frequency is to be subjected to face recognition in face recognition processing.

  For example, as a result of tabulation in step S10, the frequency of recognition of a face from area A shown in FIG. 3 is 50%, the frequency of recognition of a face from area B is 30%, and the frequency of recognition of a face from area C is 10 %, When the frequency of face recognition from other areas is 10%, A area is 5 times in 10 times, B area is 3 times in 10 times, C area is 1 in 10 times It is determined that the D area is to be recognized as a face once every ten times. Then, based on the frequency determined for each area, it is determined that face recognition is to be performed in the order of "ABACABABAD", and the coordinates and the order of each area are stored in the face recognition setting storage unit 254 as face recognition setting information.

  As a result, the frequency of performing the face recognition process on the entire captured image area can be reduced, and the face recognition can be performed with emphasis on the range where the face is recognized frequently. It is possible to improve the quality and shorten the processing time.

  Further, as shown in FIG. 3, the size of the image range to be face recognition target is not uniform, and changes depending on the position of the image range. For example, since the A area and the B area are areas where the face of a person at a position away from the digital signage device 1 is captured, the size of the recognized face is small. Therefore, the size of the target range of face recognition determined based on the A area and the B area becomes smaller (for example, it becomes an area of 70 × 50 pixels). On the other hand, since the area C is an area in which the face of a person in the vicinity of the digital signage device 1 is captured, the size of the face to be recognized is large. Therefore, the size of the target range of face recognition determined based on the C area becomes large (for example, an area of 150 × 100 pixels). That is, since the size of the range of the captured image to be subjected to face recognition is determined based on the size of the face area recognized in the range in the past, there is no waste corresponding to the size of the face that can be recognized An area of a size can be set as a target range of face recognition.

  When the digital signage device 1 is physically moved, the history information stored in the face recognition history storage unit 253 can not be used. Therefore, when it is detected that the digital signage device 1 has been moved, the control unit 23 deletes the history information stored in the face recognition history storage unit 253, and the history information of the area where the face is recognized again is used. accumulate. Whether or not the digital signage device 1 is physically moved can be detected by an acceleration sensor, for example, with the pedestal 28 or the like of the digital signage device 1 including an acceleration sensor.

  As described above, according to the digital signage device 1, the control unit 23 performs the face recognition result when the face recognition process is performed on the captured image acquired by the imaging unit 34 according to the face recognition program. Accumulate and store in 253. When a predetermined timing arrives, the control unit 23 tabulates history information of face recognition results stored in the face recognition history storage unit 253, and a captured image to be subjected to face recognition processing based on the tabulated result And the frequency is set in the face recognition setting storage unit 254. Then, when performing the face recognition process thereafter, the control unit 23 specifies an image range to be subjected to the face recognition process from the captured image based on the face recognition setting information stored in the face recognition setting storage unit 254. Then, an image in the specified range is cut out and passed to a face recognition program to perform face recognition processing.

  Therefore, the range and frequency of the captured image to be subjected to the face recognition process can be determined based on the past face recognition result. Therefore, as in the prior art, without detecting a person each time before the face recognition process It becomes possible to efficiently shorten the processing time of face recognition and improve the responsiveness.

In addition, based on the history information of the face recognition result stored in the face recognition history storage unit 253, the control unit 23 calculates the frequency at which the face is recognized from that area for each area where the face is recognized by the face recognition processing. A range of a captured image to be subjected to face recognition so that each area in which a face is recognized by face recognition processing is calculated and becomes a target of face recognition by face recognition processing at a frequency at which the face is recognized from that area Determine the frequency.
Therefore, it is possible to reduce the frequency of performing the face recognition processing on the entire captured image, and to perform the face recognition in a focused manner in the range where the face is frequently recognized.

  Further, the control unit 23 determines the size of the range of the captured image to be subjected to the face recognition by the face recognition process based on the size of the area in which the face is recognized by the face recognition process in the past. Therefore, it is possible to set an area of a size without waste that matches the size of a face to be recognized as a target range of face recognition.

  In addition, by storing the history of the face recognition result when the first face that was not recognized last time is recognized by the face recognition processing is stored in the face recognition history storage unit 253, the storage of unnecessary history is suppressed. Can.

  The contents of the description in the above embodiment are a preferred example of the digital signage apparatus according to the present invention, and the present invention is not limited to this.

  For example, in the above embodiment, the captured image of the entire range of the captured image area that can be acquired by the imaging unit 34 is acquired, and the control unit 23 determines coordinate information of a face recognition target area in the acquired captured image as a face recognition program The face of the determined imaging range is determined at a determined frequency, but the coordinate information of the face recognition target area is passed to the imaging unit 34, and the imaging unit 34 determines the determined imaging range. The image may be taken at the specified frequency. Thereby, the memory amount used for storing a captured image can be suppressed. Moreover, it is possible to omit the process of cutting out the area of the face recognition target from the captured image.

  Moreover, in the said embodiment, although the case where content was a moving image (with audio | voice) was demonstrated to an example, it is not limited to this, For example, it is good also as only an image.

  In the above embodiment, although the present invention is applied to a digital signage apparatus that displays an image by projecting an image from a projector onto a screen, the present invention has been described by way of example. The same effect can be obtained even when applied to an apparatus, and is not limited to this example.

  In addition, the detailed configuration and the detailed operation of the digital signage apparatus can be appropriately modified without departing from the scope of the invention.

Although several embodiments of the present invention have been described, the scope of the present invention is not limited to the above-described embodiments, but includes the scope of the invention described in the claims and the equivalents thereof.
In the following, the invention described in the claims initially attached to the request for this application is appended. The item numbers of the claims described in the appendix are as in the claims attached at the beginning of the application for this application.
[Supplementary note]
<Claim 1>
Imaging means,
Face recognition means for performing face recognition on the captured image acquired by the imaging means;
A determination unit configured to determine the range and frequency of a captured image to be subjected to face recognition by the face recognition unit based on a history of face recognition results by the face recognition unit;
An information processing apparatus comprising:
<Claim 2>
The determination means calculates, based on the history of face recognition results by the face recognition means, for each of the areas where the face is recognized by the face recognition means, the frequency with which the face is recognized from the areas is calculated. Range and frequency of captured images to be subjected to face recognition by the face recognition unit such that each area in which a face is recognized by the face recognition target is subject to face recognition by the face recognition unit at a frequency at which the face is recognized from that area The information processing apparatus according to claim 1, wherein
<Claim 3>
The determination means is a captured image to be subjected to face recognition by the face recognition means based on the size of the area in which the face is recognized by the face recognition means obtained from the history of face recognition results by the face recognition means. The information processing apparatus according to claim 1, wherein the size of the range is determined.
<Claim 4>
4. The information according to any one of claims 1 to 3, wherein the history of the face recognition result by the face recognition means is a history of the face recognition result when the face not recognized by the face recognition means is recognized. Processing unit.
<Claim 5>
A face recognition step of performing face recognition on a captured image acquired by the imaging unit;
A determination step of determining a range and a frequency of a captured image to be subjected to face recognition by the face recognition step based on a history of face recognition results by the face recognition step;
Information processing method including:
<Claim 6>
A computer provided in an information processing apparatus including an imaging unit;
Face recognition means for performing face recognition on the captured image acquired by the imaging means;
Determining means for determining the range and frequency of a captured image to be subjected to face recognition by the face recognition means based on the history of face recognition results by the face recognition means;
Program to function as.

Reference Signs List 1 digital signage device 21 projection unit 22 screen unit 23 control unit 24 projector 25 storage unit 251 program storage unit 252 content storage unit 253 face recognition history storage unit 254 face recognition setting storage unit 26 communication unit 27 image forming unit 28 pedestal 29 light transmission Board 30 Timekeeping unit 32 Operation unit 33 Voice output unit 34 Imaging unit

Claims (5)

Imaging means,
Face recognition means for performing face recognition on the captured image acquired by the imaging means;
Based on the history of face recognition results by the face recognition unit, the range of the captured image to be the target of face recognition by the face recognition unit and the frequency of the face recognition process for the range of the captured image to be the target of the face recognition are determined and a determination means for,
The determination means calculates, based on the history of face recognition results by the face recognition means, for each of the areas where the face is recognized by the face recognition means, the frequency with which the face is recognized from the areas is calculated. A range of a captured image to be subjected to face recognition by the face recognition unit such that each area in which a face is recognized by the face recognition is subject to face recognition by the face recognition unit at a frequency at which the face is recognized from that area; An information processing apparatus that determines the frequency of face recognition processing for a range of a captured image to be subjected to the face recognition . The determination means is a captured image to be subjected to face recognition by the face recognition means based on the size of the area in which the face is recognized by the face recognition means obtained from the history of face recognition results by the face recognition means. The information processing apparatus according to claim 1, wherein the size of the range is determined. The information processing apparatus according to claim 1, wherein the history of the face recognition result by the face recognition unit is a history of the face recognition result when the face not recognized by the face recognition unit is recognized. A face recognition step of performing face recognition on a captured image acquired by the imaging unit;
Based on the history of face recognition results in the face recognition process, the range of the captured image to be subjected to the face recognition in the face recognition process and the frequency of the face recognition process for the range of the captured image to be the face recognition are determined and a determination step of,
The determination step calculates, based on the history of face recognition results obtained by the face recognition step, the frequency with which a face is recognized from each of the areas where the face is recognized by the face recognition step, and the face recognition step A range of a captured image to be subjected to face recognition in the face recognition step such that each area in which the face is recognized is a target of face recognition in the face recognition step at a frequency at which the face is recognized from that area; An information processing method for determining the frequency of face recognition processing for a range of a captured image to be subjected to the face recognition . A computer provided in an information processing apparatus including an imaging unit;
Face recognition means for performing face recognition on the captured image acquired by the imaging means;
Based on the history of face recognition results by the face recognition unit, the range of the captured image to be the target of face recognition by the face recognition unit and the frequency of the face recognition process for the range of the captured image to be the target of the face recognition are determined and determining means for it to function,
The determination means calculates, based on the history of face recognition results by the face recognition means, for each of the areas where the face is recognized by the face recognition means, the frequency with which the face is recognized from the areas is calculated. A range of a captured image to be subjected to face recognition by the face recognition unit such that each area in which a face is recognized by the face recognition is subject to face recognition by the face recognition unit at a frequency at which the face is recognized from that area; A program for determining the frequency of face recognition processing for the range of a captured image to be subjected to the face recognition .

Images