JP3734793B2 - Human detection device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a person detection device, and more particularly to an apparatus for detecting a person with higher accuracy without overlooking a person hidden by a previous person.
[0002]
[Prior art]
As a conventional device of this type, first, a pedestrian moving up and down the stairs is photographed by a plurality of imaging devices installed on the stairs with depression angles. The image is input to the association unit, and parallax data is acquired. The parallax data is converted into distance data in the parallax distance converter. The recognition unit separates pedestrians using distance data and recognizes their positions. In the counting unit, the position of the recognized pedestrian is tracked in time and counted when passing through the counting line. As a result, pedestrians are recognized and tracked from a stereo image with a small number of blind spots, so that the number of pedestrians, the moving direction, the degree of congestion, and the like can be accurately measured even when the pedestrians are crowded (for example, see Patent Document 1) ).
[0003]
[Patent Document 1]
Japanese Patent Laid-Open No. 10-334207
[Problems to be solved by the invention]
Conventional devices such as those described above photograph pedestrians going up and down stairs and detect people from them, but in a flat place (e.g. hotel lobby, elevator hall, corridor, etc.) The person hidden by the previous person could not be detected accurately without missing.
[0005]
The present invention has been made to solve the above-described problems, and in particular, more accurately detects a plurality of persons even on a flat place without missing a person who is concealed by a previous person. An object of the present invention is to provide a human detection device capable of performing the above.
[0006]
[Means for Solving the Problems]
In view of the above object, the present invention associates a plurality of cameras installed at positions over which a plurality of persons can be seen with a position on the image indicating the same observation point in space on the image from each camera. An inter-image correlation means; a distance calculation means for calculating a three-dimensional distance of the observation point in the space from the camera from the parallax of the corresponding corresponding points and the distances between the plurality of cameras; and the three-dimensional A three-dimensional position calculation means for calculating the position of the observation point in the three-dimensional space from the distance, and the obtained three-dimensional position of each observation point is projected onto a specific plane to create a vote value distributed on the plane. Concealment with a high probability that concealment will occur by decreasing the value each time the voting value obtained increases gradually toward the predetermined maximum value as the position from the specific plane projection means and the camera moves away. Set the area A concealment area setting means for setting a threshold for object detection, and a person detection means for determining that a person is detected when the obtained vote value reaches the threshold for which the concealment area is set. It is in the human detection device that is characterized.
[0007]
DETAILED DESCRIPTION OF THE INVENTION
Embodiment 1 FIG.
FIG. 1 is a diagram for explaining the principle of a person detection apparatus according to the present invention, showing a person detected in one longitudinal section taken by a camera and a vote value histogram of these persons, and FIG. 2 according to the present invention. It is a figure which shows an example of a structure of a person detection apparatus main body. In FIG. 2, reference numerals 117 to 125 relate to the third embodiment described later. FIG. 1 shows an elevator hall as an example. A plurality of cameras 101 (only one is shown here), typically two cameras, installed at the top of the three-way frame of the elevator or in the vicinity thereof. 101 (stereo camera) captures an area where the person A gathers. An image obtained by each camera is input to a person detection apparatus main body constituted by a computer via an interface 103 shown in FIG.
[0008]
The inter-image association unit 105 obtains the positions of both points (referred to as corresponding points) that are considered to observe the same point in the three-dimensional space between images input from the stereo camera. The distance on the image when the position of the corresponding point of the other image is relatively viewed from the position of the corresponding point of one image is called parallax. Then, the positional relationship between the plurality of cameras 101 is calibrated in advance, and the distance calculation means 107 calculates the observation point in the space that is the origin of the corresponding point from the parallax d and the distance D (baseline) between the two cameras. A three-dimensional distance, that is, a distance Z = a (D / d) (a is a coefficient) from the line connecting both cameras 101 to the observation point can be calculated. If the three-dimensional distance can be calculated, the position of each observation point in the three-dimensional space can be calculated by the existing method by the three-dimensional position calculation means 109.
[0009]
Next, consider projecting this three-dimensional position onto a specific plane. An example of the specific plane projection unit 111 is shown below. In the case of person detection, assuming that the person is in the state of a pedestrian, it may be considered that an object extending vertically in the vertical direction is present on the floor surface. Therefore, a specific plane is used as a floor surface, and a three-dimensional point (each observation point) is projected on the floor surface. If the number of three-dimensional points (observation points) observed from the camera during projection is large, the value of the point on the projection also increases. This operation is called voting. Further, a value obtained as a result of voting on a specific surface is referred to as a vote value. A voting value histogram 10 which is a voting result obtained based on an image showing the person A is shown on the lower side of FIG. However, the voting value histogram 10 is shown downward, and the voting value increases on the lower side. A curve B indicates a vote value, and a sawtooth wave line C indicates a threshold for determining a person to be described later. In this way, the vote value becomes large at the place on the floor where the person stands, and the person can be detected by detecting the portion with the large vote value.
[0010]
On the other hand, however, when a large number of people gather, the back person may be hidden (hidden) by the front person and a sufficient observation point may not be obtained. In that case, a large vote value does not appear on the projection plane, and a detection leak occurs.
[0011]
Therefore, the hidden area is set by the hidden area setting means 113. An example of concealment area setting by the concealment area setting unit 113 will be described. First, consider that a person is detected in order from a position close to the camera 101 on the projection plane. Since concealment does not occur at a position close to the camera and a high vote value can be expected, a person can be detected by dividing the vote value with a large (maximum) fixed threshold T as shown by 10 in FIG. When the vote value reaches the threshold value, the side after detecting a person is estimated as a concealment region with a high probability of concealment, and the threshold value is lowered to increase the detection sensitivity here. From the detected three-dimensional observation point position of the person from the three-dimensional position calculation means 109, the approximate height L of the person is grasped. As shown in FIG. 1, the height L and the average height Lm are compared, and the difference Ld (+ side only) is calculated. The value of the difference Ld is multiplied by a fixed value b, and the threshold value is lowered by that value (−bLD). In addition, using the relationship between the camera and the specific plane from the lowered point, the threshold value is raised linearly toward the point where the concealment region ends (for example, the right end in FIG. 1) and gradually recovers toward the maximum value. Go. Therefore, the threshold value C used for the determination at the time of person detection is the maximum value T until the first person detection, and then the height of the person is detected every time a person whose vote value reaches the threshold value is detected (average height) It is lowered in size), and after being lowered, it becomes a saw-like shape that gradually increases to return to the maximum value.
[0012]
If the threshold adjustment according to the concealment area can be performed by the concealment area setting unit 113, the vote value is obtained by the person detection unit 115 considering the concealment degree by comparison with this threshold (for example, determination of whether or not the threshold is reached). By detecting that has reached the threshold, it is possible to detect a person without overlooking. Also, there may be a case where almost no vote value is obtained in an area where concealment is large. In that case, returning to the input image, from the knowledge of the position of the concealment area and the average height of the person, if it is assumed that there is a person, the position of the face is back calculated, The detection performance may be further improved by detecting only the face.
[0013]
In particular, the large concealment is based on the physical fact that the camera is observing the person from a horizontal direction. There is a high possibility of being caught. In such a case, the existing technique can sufficiently detect the face and is particularly effective.
[0014]
Embodiment 2. FIG.
In the first embodiment, the concealment region is premised on the assumption that the reliability of the image from the camera is low and that the rear side of the object that tends to be a concealment element extending vertically from the floor surface is the concealment region. Although the concealment area is set based on the comparison with the threshold value of the vote value in the setting means 113, in this embodiment, the reliability of the image from the camera is high, and for example, the concealment area extends vertically from the floor surface. The concealment region is set based on the three-dimensional position of each observation point from the three-dimensional position calculation unit 109 in the concealment region setting unit 113a on the premise that it is possible to limit to not only those. FIG. 3 is a diagram showing another example of the configuration of the main body of the person detection apparatus according to the present invention, and the same or corresponding parts as those of the above-described embodiment are denoted by the same reference numerals and description thereof is omitted. Reference numerals 117 to 125 relate to the third embodiment described later.
[0015]
The hidden area setting means 113a sets a hidden area. An example of concealment area estimation by the concealment area setting unit 113a will be described. First, consider that persons are detected in order from a position close to the camera 101 on the projection plane. Since concealment does not occur at a position close to the camera and a high vote value can be expected, a person can be detected by dividing the vote value with a large fixed threshold T as shown by 10 in FIG. Based on the three-dimensional position of each observation point from the three-dimensional position calculation means 109, a concealment region having a high probability of concealment is estimated, and the threshold value is lowered to increase the detection sensitivity here. Then, the approximate height L of the person is grasped from the three-dimensional observation point position of the detected person from the three-dimensional position calculation means 109 corresponding to each concealment region. As shown in FIG. 1, the height L and the average height Lm are compared, and the difference Ld (+ side only) is calculated. The value of the difference Ld is multiplied by a fixed value b, and the threshold value is lowered by that value (−bLD). Further, the threshold value is raised so as to recover linearly from the lowered point toward the point where the concealment region ends (for example, the right end in FIG. 1) using the relationship between the camera and the specific plane. Therefore, the threshold value C used for the determination at the time of person detection is the maximum value T until the first person detection, and the magnitude according to the height of the person every time a person is detected (exceeding the average height). After being lowered, it becomes a saw-like shape that gradually increases to return to the maximum value.
[0016]
If the threshold adjustment according to the concealment area can be performed by the concealment area setting unit 113a, the voting plane is obtained by the person detection unit 115a considering the concealment degree by comparison with this threshold (for example, determination of whether or not the threshold is reached). By detecting the peak of the vote value from the projection unit 111, a person can be detected without missing.
[0017]
In addition, regarding the first and second embodiments, in the above description, the threshold is lowered based on the detected height of the person exceeding the average height, so that the detection sensitivity is more appropriate according to the height of the person. However, even if any person is detected, the detection sensitivity in the concealment region can be increased by uniformly lowering the threshold value by a predetermined value, and there is a corresponding effect, and the configuration can be simplified.
[0018]
Embodiment 3 FIG.
In the first and second embodiments, the person is detected mainly using the distance information. However, in this embodiment, not only the distance but also the pattern information, silhouette, color histogram, etc. of the person area on the image By using in combination, a more accurate person detection device can be provided. In this embodiment, in FIG. 2 and FIG. 3, a plurality of feature quantity detection means 117, an integration means 119, a person feature quantity storage unit 121, and a person determination means 123 are provided. These operations are common in FIGS.
[0019]
As shown in FIG. 4, the feature amount of the distance, face pattern, person silhouette, and color histogram is detected in the human region on the image from the camera 110 by the plural feature amount detecting means 117, and the human region is used using these information. Think about detecting. At this time, the distance is processed as described in the above embodiment. However, if the distance is still not detected due to a lack of distance information detection means or noise on the image, it is supplemented with other information. And there is a request to detect. At this time, it is possible to determine whether or not the region is a human region using the remaining face pattern, human silhouette, color histogram, and the like.
[0020]
For example, in this case, the face pattern, the person silhouette, and the color histogram are input to the integration unit 119, and the integration unit 119 determines the degree of matching of the face pattern with the person, the degree of matching of the person silhouette with the person, The degree of coincidence of the color histogram with a person is calculated, and if the degree of coincidence of each is higher than a certain level, it is determined as a person region. Note that the degree of coincidence can be easily calculated by the existing means such as the face pattern detection unit 117a, the person silhouette detection unit 117b, the color histogram detection unit 117c, and the like, and thus is not particularly mentioned here. As an example of the integration unit 119, the case where the degree of coincidence of each feature amount is compared with a certain threshold value has been described. However, the present invention is not limited to this. For example, a combination of feature value values from several person images may be learned in advance using a learning algorithm such as a neural network, and the combination unit 119 may be used. It should be noted that each of the face pattern, person silhouette, and color histogram used for detection of the feature amount of the face pattern, person silhouette, and color histogram in the multiple feature amount detection unit 117 and for calculation of each matching degree in the integration unit 119 is provided. The model and template are stored in the person feature amount storage unit 121. Then, the person determination unit 123 adds the determination result of the integration unit 119 to the determination result of the person detection unit 115 to determine all as persons.
[0021]
In FIG. 4, it is assumed that a part of the person area to be detected is currently concealed by the already detected left person. If it does so, the matching degree to the person obtained from the person silhouette detection means 117b may fall. In the worst case, there is a risk that a person cannot be detected just by being partially concealed. If, for example, information such as concealment from the concealment area setting unit 113 indicates that the silhouette cannot be used in advance, it is only necessary to determine whether the person is a person using only the face pattern and the color histogram. . As described above, the feature amount selecting unit 125 may be provided that calculates the reliability of each feature amount and determines whether or not to use it for the determination.
[0022]
In addition, regarding each said embodiment, although the apparatus attached to the three-way frame of an elevator is assumed in the said description, if it installs with the same depression angle at the same height, it is necessary to limit to the waiting detection of an elevator. No. For example, it can be applied to indoor and outdoor human flow measurement.
[0023]
【The invention's effect】
As described above, according to the present invention, between a plurality of cameras installed at positions where a plurality of persons can be seen and images on the images from the respective cameras are associated with positions on the image indicating the same observation point in space. An associating means; a distance calculating means for calculating a three-dimensional distance of the observation point in the space from the camera from a parallax of the corresponding corresponding points and a distance between the plurality of cameras; A three-dimensional position calculation means for calculating the position of the observation point in a three-dimensional space, and a specific plane for projecting the obtained three-dimensional position of each observation point onto a specific plane and creating a vote value distributed on the plane A concealment area with a high probability that concealment will occur by gradually decreasing toward the predetermined maximum value as the position from the projection means and the camera moves away, and decreasing the value each time the obtained vote value reaches this value. Set person A human detection apparatus comprising: a concealment area setting means for setting a threshold for exit; and a person detection means for determining that a person has been detected when an obtained vote value has reached the threshold for which the concealment area is set. Therefore, it is possible to detect a plurality of persons even on a flat place with higher accuracy without overlooking a person hidden by the previous person.
[Brief description of the drawings]
FIG. 1 is a diagram for explaining the principle of a person detection apparatus according to the present invention.
FIG. 2 is a diagram showing a configuration of a person detection device main body according to an embodiment of the present invention.
FIG. 3 is a diagram showing a configuration of a person detection device main body according to another embodiment of the present invention.
FIG. 4 is a diagram for explaining a plurality of feature quantity detection means, feature quantity selection means, and integration means according to the present invention.
[Explanation of symbols]
103 interface, 105 image-to-image association means, 107 distance calculation means, 109 three-dimensional position calculation means, 111 specific plane projection means, 113, 113a concealment area setting means, 115, 115a human detection means considering concealment degree, 117 plural Feature quantity detection means, 119 integration means, 121 person feature quantity storage unit, 123 person determination means, 125 feature quantity selection means.

Claims (5)

A plurality of cameras installed in a position where a plurality of persons can be seen,
On the images from the respective cameras, an image-to-image association means for associating positions on the image indicating the same observation point in space,
Distance calculating means for calculating a three-dimensional distance of the observation point on the space from the camera from the parallax of the corresponding corresponding point and the distance between the plurality of cameras;
3D position calculation means for calculating the position of the observation point in a 3D space from the 3D distance;
Specific plane projection means for projecting the obtained three-dimensional position of each observation point onto a specific plane and creating vote values distributed on the plane;
Human detection with a concealment area that has a high probability of concealment by decreasing the value each time the obtained vote value reaches this value gradually increases as the position from the camera increases. A hidden area setting means for setting a threshold for
Human detection means for determining human detection when the obtained vote value has reached the threshold for which the concealment area is set;
A person detection apparatus comprising: A plurality of cameras installed in a position where a plurality of persons can be seen,
On the images from the respective cameras, an image-to-image association means for associating positions on the image indicating the same observation point in space,
Distance calculating means for calculating a three-dimensional distance of the observation point on the space from the camera from the parallax of the corresponding corresponding point and the distance between the plurality of cameras;
3D position calculation means for calculating the position of the observation point in a 3D space from the 3D distance;
Specific plane projection means for projecting the obtained three-dimensional position of each observation point onto a specific plane and creating vote values distributed on the plane;
A concealment region having a high probability of concealment is obtained from a position in the three-dimensional space of each observation point obtained from the three-dimensional position calculation means, the value decreases in each concealment region, and then reaches a maximum value as the position from the camera moves away. A concealment area setting means for setting a threshold for human detection in which a concealment area that gradually increases toward the person is set;
Human detection means for determining human detection when the obtained vote value has reached the threshold value in consideration of the concealment area;
A person detection apparatus comprising: The person detection apparatus according to claim 1, wherein the concealment area setting unit changes an amount of lowering the threshold according to a person's height obtained on an image from the camera. A plurality of feature amount detecting means for detecting a region including a plurality of types of feature amounts of different persons on the image of the camera;
An integration means for determining a person area if the degree of coincidence with each predetermined model of each feature amount in the detected area has reached a predetermined rule;
A person determination unit that adds the determination result of the integration unit to the determination result of the person detection unit and determines all as persons;
The person detection device according to claim 1, further comprising: The image processing apparatus further includes a feature amount selection unit that selects the feature amount for determining the person region in the integration unit according to a situation of a region including a plurality of different types of feature amounts of a person in the image from the camera. The person detecting device according to claim 4, wherein

Images