JP5687082B2 - Moving object tracking device - Google Patents

Description

  The present invention relates to a moving object tracking apparatus that tracks a plurality of moving objects by image processing.

  As a conventional technique for tracking a moving object by image processing, one using a template matching process is known. In the template matching process, if the entire image is set as the search range, the processing load increases. Therefore, the search range is around the position of the moving object of interest at the previous time or around the predicted position at the current time based on past motion or the like. Set. Then, the degree of coincidence between the input image and the template is examined within the limited search range, and it is determined that there is a moving object that is focused on the most coincident position.

  Here, when there are a plurality of tracking target moving objects in the tracking area, when the moving objects are close to each other, the search ranges of those templates may overlap, and the position of the person other than the person of interest is the most. May match the template. In this case, there is a mix-up that starts tracking people other than those who should be tracked.

  As a countermeasure against this, there is a conventional technique in which when a candidate position of a template is close to another template, the degree of coincidence is re-evaluated by correcting the templates to be separated (Patent Document 1).

JP 2009-48428 A

  However, in the prior art, it is difficult to control the correction of the template position when the distance between three or more objects approaches. For example, when attention is paid to one of three objects, two distances to the other two objects are used as an index. In this case, if one distance is separated, the other distance may be closer, and even if both distances are increased, there is no guarantee that the direction will approach the true target object position. As a result, the corrected template position falls into the local minimum, causing an error and sometimes causing a mistake in the person.

  The likelihood of such errors increases as more objects are approached. In addition, as the number of objects approaching increases, the number of corrections increases according to the combination, and there is a problem that real-time tracking becomes difficult.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide a moving object tracking device capable of accurately tracking these positions even when objects approach each other.

  A moving object tracking device according to the present invention tracks a plurality of objects moving in the space using time-series images obtained by photographing a predetermined space, and each object in the past from the attention time A storage unit that stores past position information and reference information for extracting image features of the object; a position prediction unit that predicts a candidate position of a destination of the object at the target time from the past position; The presence setting unit that sets each object as a target object, and sets a presence level representing the probability of existence of the other object in a distribution range of the candidate positions of the other objects excluding the target object among the plurality of objects, An evaluation value corresponding to the degree to which the image feature is extracted from the portion corresponding to the candidate position of the target object in the image is set in the candidate position by comparing the image at the target time with the reference information. Calculated reduced in accordance with the abundance that, and a determining object position determination unit the destination location of the target object from the evaluation value of the candidate position.

  In the moving object tracking device according to another aspect of the invention, the position predicting unit predicts a plurality of candidate positions for each object, and the presence setting unit is centered on each of the candidate positions of the other object. A presence function representing the probability of existence of the other object is set in each range having the size of the other object, and the presence degree is calculated by adding the value of the presence function for each position in the space. In the moving object tracking device, the presence level setting unit weights the presence level function of the candidate position for which the evaluation value is calculated by the object position determination unit according to the evaluation value, and calculates the presence level. It can be set as the structure to calculate.

  In the moving object tracking device according to still another aspect of the present invention, the presence level setting unit may include the candidate position of the other object for the other object whose position is determined by the object position determination unit. Instead, the range for setting the presence is determined using the determined destination position.

  In the moving object tracking device according to another aspect of the invention, the position predicting unit predicts a plurality of candidate positions for each object, and the presence setting unit distributes the plurality of candidate positions for each other object. A probability density function according to the above is set, and the value of the probability density function of each other object is added for each position in the space to calculate the presence level.

  According to the present invention, by reducing the evaluation value as the movement destination position with respect to the candidate position of the movement destination of the object of interest according to the presence of other objects at the position, the movement destination positions of a plurality of approaching objects are preferable. Can be separated. That is, the position of each object can be accurately determined even when a plurality of objects approach, particularly when three or more objects approach.

It is a block block diagram of the moving object tracking apparatus which concerns on embodiment of this invention. It is a perspective view which shows an example of a three-dimensional model typically. It is a typical perspective view of the monitoring space where four persons exist as a plurality of moving objects. It is a schematic diagram explaining the example of a presence map corresponding to arrangement | positioning of the moving object of FIG. It is a schematic diagram explaining the method of producing an individual presence map from a predicted position. It is a schematic diagram explaining the method of producing the synthetic | combination presence map for object position determination of an attention object from an individual presence map. It is a schematic diagram explaining the other preparation method of the synthetic | combination presence map for object position determination. It is a general | schematic flowchart of the tracking process of the moving object tracking apparatus which concerns on embodiment of this invention. It is a schematic diagram explaining the process example by the moving object tracking apparatus which concerns on embodiment of this invention. It is a schematic diagram explaining the other method of creating an individual presence map from a predicted position.

  Hereinafter, a moving object tracking device 1 according to an embodiment of the present invention (hereinafter referred to as an embodiment) will be described with reference to the drawings. The moving object tracking device 1 uses a person moving in the monitoring space as a tracking target (hereinafter referred to as a moving object), and processes the monitoring image obtained by imaging the monitoring space to track the moving object. The moving object tracking device 1 is configured to be able to track a plurality of moving objects existing in the monitoring space. The monitoring space is not limited to being indoors but may be outdoors, and the moving object may be a person other than a person such as a vehicle.

[Configuration of moving object tracking device]
FIG. 1 is a block diagram of a moving object tracking device 1 according to the embodiment. The moving object tracking device 1 includes an imaging unit 2, a setting input unit 3, a storage unit 4, a control unit 5, and an output unit 6. The imaging unit 2, the setting input unit 3, the storage unit 4, and the output unit 6 are connected to the control unit 5.

  The imaging unit 2 is a monitoring camera, is installed so as to face the monitoring space, and images the monitoring space at a predetermined time interval. The captured monitoring images of the monitoring space are sequentially output to the control unit 5. In order to grasp the position and movement of a person who moves along a reference plane such as the floor surface or the ground surface, the imaging unit 2 is basically installed at a height that allows a bird's-eye photography, for example, in this embodiment, The moving object tracking device 1 is used for indoor monitoring, and the imaging unit 2 is installed on the ceiling. The time interval at which the monitoring image is captured is 1/5 second, for example. Hereinafter, the unit of time recorded at the time interval of imaging is referred to as time.

  The setting input unit 3 is an input means for the administrator to perform various settings for the control unit 5, and is a user interface device such as a touch panel display, for example.

  The storage unit 4 is a storage device such as a ROM (Read Only Memory) or a RAM (Random Access Memory). The storage unit 4 stores various programs and various data, and inputs / outputs such information to / from the control unit 5. Various data includes a three-dimensional model 40, a camera parameter 41, a predicted position 42, an object position 43, an abundance map 44, and a background image.

  The three-dimensional model 40 is data describing a state in which a moving object model that approximates a three-dimensional shape of a moving object is arranged in a virtual space that simulates a monitoring space (real space). In the present embodiment, the monitoring space and the virtual space are represented by a right-handed orthogonal coordinate system including the X, Y, and Z axes, and the vertically upward direction is set to the positive direction of the Z axis. A reference surface such as a floor surface is horizontal and is defined by an XY plane represented by Z = 0. The arrangement of the moving object model in the virtual space can be set at an arbitrary position.

  FIG. 2 is a perspective view schematically showing an example of the three-dimensional model 40. The reference plane 100, the camera position 101, and the moving object model 103 of the virtual space obtained by discretizing the monitoring space into N × M × K position coordinates. An arrangement example is shown.

  The moving object model is, for example, data describing a partial model representing a three-dimensional shape for each of a plurality of constituent parts constituting the moving object, and an arrangement relationship between the partial models. The moving object to be monitored by the moving object tracking device 1 is a standing person. In this embodiment, for example, a spheroid approximating the three-dimensional shape of the human head torso and leg is represented by Z Set the moving object model stacked in the axial direction. The height from the reference plane to the center of the head is represented by H, and the maximum width of the trunk (the trunk minor axis diameter) is represented by W. In this embodiment, in order to simplify the description, the height H and the width W are standard human sizes and are common to any moving object. The center of the head is set as the representative position of the moving object. Note that the moving object model may be further simplified and approximated by one spheroid. Data relating to the three-dimensional shape of the moving object model is stored in the storage unit 4 prior to the tracking process.

  The data stored as a moving object model in the storage unit 4 is the feature information of each moving object extracted from the monitoring image (for example, image features such as a color histogram) as reference information for individually characterizing the moving object to be tracked. including. The feature quantity of the moving object is extracted from the image area corresponding to the detected position and stored in the storage unit 4 when a moving object to be tracked is newly detected in the input image. Note that the data related to the shape of the moving object model is also used as reference information when extracting the image features of the moving object and is compared with the monitoring image.

  The camera parameter 41 includes information regarding a projection condition when the imaging unit 2 captures a monitoring image that projects the monitoring space. For example, external parameters such as the installation position and imaging direction of the imaging unit 2 in the actual monitoring space, focal length, angle of view, lens distortion, and other lens characteristics of the imaging unit 2, and internal parameters such as the number of pixels of the imaging element are included. These parameters obtained by actual measurement or the like are input in advance from the setting input unit 3 and stored in the storage unit 4. The three-dimensional model 40 can be projected onto the coordinate system of the monitoring image (imaging surface of the imaging unit 2; xy coordinate system) by a coordinate transformation formula in which the camera parameter 41 is applied to a camera model such as a known pinhole camera model.

  The predicted position 42 (hypothesis) is information regarding the predicted value (predicted position) of the position of the moving object at each time. In order to determine the position (object position) of a moving object stochastically, a large number of predicted positions are set for each moving object (the number is represented by α, for example, 200 per moving object). Specifically, the predicted position 42 is time-series data in which an identifier of a moving object is associated with an index (0 to α-1) of a predicted position at each time and its position coordinates (XYZ coordinate system).

  The object position 43 is information regarding the position of the moving object at each time, and specifically is time-series data in which the identifier of the moving object is associated with the position coordinates (XYZ coordinate system). That is, the position of each moving object determined at each time is sequentially added to the object position 43, and information on the past position of each moving object is held.

  Note that the processing can be speeded up as a configuration in which the predicted position 42 and the object position 43 are specified by the horizontal coordinate (XY coordinate system) of the monitoring space. In the present embodiment, the configuration will be described as an example for easy understanding.

  The presence map 44 is data in which a presence indicating the degree (probability) that each moving object may exist at each position in the monitoring space is associated with the position and the identification code of the moving object. The degree of presence is set with the predicted position 42 or the object position 43 as the center position. When moving objects occupy a part of the monitoring space, and the occupied space of multiple moving objects exists exclusively, the presence of any moving object is determined when determining the position of another moving object By using this as a penalty value, an exclusive action is produced between objects.

  FIG. 3 is a schematic perspective view of a monitoring space where four persons (moving objects # 1 to # 4) are present as a plurality of moving objects. FIG. 4 is a schematic diagram for explaining an example of the presence map 44, and shows an example corresponding to the arrangement of the moving objects in FIG. The abundance map 44 has an X axis and a Y axis that represent positions corresponding to the reference plane, and a P axis that represents the abundance perpendicular to the X and Y axes. The presence map 44 includes an individual presence map for each moving object and a combined presence map obtained by combining a plurality of individual presence maps. 4A shows an individual presence map 201 of the moving object # 1, an individual presence map 202 of the moving object # 2, an individual presence map 203 of the moving object # 3, and an individual presence map 204 of the moving object # 4. Is shown. In the individual abundance maps 201 to 204, the cloud-shaped regions shown at the upper left indicate the distribution ranges of significant abundances. FIG. 4B shows a composite presence map 210 obtained by combining these four individual presence maps 201 to 204.

Each individual abundance map is an output value of a function obtained by adding and combining a plurality of two-dimensional normal distribution probability density functions having XY coordinates as variables (FIG. 4A shows an example in this case), or It is represented by an output value of a probability density function of one two-dimensional normal distribution having XY coordinates as variables. In this embodiment, since the reference plane is discretized into N × M, the individual abundance map is actually XY in the range of X = [0, N−1] and Y = [0, M−1]. The output values of the above functions at the coordinates are arranged as individual abundances, and the collection codes of the individual abundances are associated with the moving object identification codes. The individual presence map may be data in which a set of the average value μ and variance value σ ^{2 of} each two-dimensional normal distribution and the weight ω of each distribution is associated with an identification code of the moving object.

The composite presence map is obtained by superimposing a plurality of individual presence maps, and is data in which composite presences are arranged in the same XY coordinate range as the individual presence map. The composite abundance map may be data composed of a set of the average value μ and variance value σ ^{2 of} each added two-dimensional normal distribution and the weight ω of the distribution.

  The control unit 5 is configured using an arithmetic device such as a CPU (Central Processing Unit), a DSP (Digital Signal Processor), or an MCU (Micro Control Unit), and reads and executes a program from the storage unit 4 to change a pixel extraction unit. 50, functions as a position prediction unit 51, a presence calculation unit 52, an object position determination unit 53, and an abnormality determination unit 54.

  The change pixel extraction unit 50 extracts change pixels from the monitoring image newly input from the imaging unit 2, and outputs information on the extracted change pixels to the object position determination unit 53. Therefore, the change pixel extraction unit 50 performs a difference process between the newly input monitoring image and the background image, and extracts a pixel whose difference is equal to or greater than a preset difference threshold as a change pixel. The change pixel is an image feature of the moving object extracted by comparing with the monitoring image using the background image as reference information. Note that the changed pixels may be extracted by a correlation process between the newly input monitoring image and the background image instead of the difference process, or the background model is learned instead of the background image and the difference process with the background model is performed. The change pixel may be extracted by.

  The position predicting unit 51 (hypothesis setting unit) performs motion prediction of the moving object using past position information (object position or predicted position) of the moving object, and a plurality of movement destination candidate positions at the current time of the moving object. Find the predicted position.

  First, the position prediction unit 51 performs motion prediction from the object position of each moving object determined in the past or the predicted position of each moving object set in the past, and the moving object at the time when a new monitoring image is input. Is predicted, and the predicted position (predicted position) is output to the presence calculation unit 52 and the object position determination unit 53. As described above, α predicted positions are set for each moving object. A method of determining a position (object position) of a moving object in a probabilistic manner by sequentially setting a large number of predicted positions in this manner is called a particle filter or the like, and the set predicted position is called a hypothesis or the like. The predicted position can be set in the xy coordinate system of the monitoring image, but in the present embodiment, it is set in the XYZ coordinate system of the monitoring space. Motion prediction is performed by applying a predetermined motion model or applying a predetermined prediction filter to past position data.

  The presence calculation unit 52 (presence setting unit) creates the presence map 44 with reference to the predicted position 42 set by the position prediction unit 51 and / or the object position 43 determined by the object position determination unit 53. The created presence map 44 is stored in the storage unit 4. In addition, when the identification code of the target object and the candidate position of the target object are input, the presence level calculation unit 52 outputs the presence level of other objects excluding the target object at the candidate position from the level map 44.

Two methods (A1) and (A2) for creating the individual presence map will be described below.
(A1) This method is a method of creating an individual presence map from the predicted position 42. The presence degree calculation unit 52 reads the predicted position 42 of the moving object to be created from the storage unit 4 and calculates the individual presence degree of the moving object by this method. This method can be used in a state where the likelihood has not yet been calculated for each predicted position. FIG. 5 is a schematic diagram illustrating an example of creating an individual presence map by this method. For each predicted position of the moving object to be created (the center of the small circle shown on the reference plane 100 in FIG. 5A), the predicted position is an average value μ, and 3σ is two minutes of the width of the moving object. A probability density function of a two-dimensional normal distribution in which a variance value σ ² matching 1 (W / 2) is set (a mountain function in FIG. 5B) is created. A significant value as the presence level is set within a range corresponding to the size of the moving object. Then, normalization is performed by dividing the output value in each XY coordinate of the function obtained by adding and synthesizing these two-dimensional normal distributions with a weight of 1.0, and XY coordinates after the normalization. The value in is defined as an individual abundance map of moving objects to be created. The cloud-shaped region shown on the reference plane 100 in FIG. 5C is a 3σ range of the two-dimensional normal distribution for each predicted position of the moving object to be created, and the range where the individual abundance is substantially set. Equivalent to. FIG. 5D is a schematic graph showing a change in individual abundance along the alternate long and short dash line in FIG.
(A2) This method is a method of creating an individual presence map from the object position 43. The presence degree calculation unit 52 reads the object position 43 of the moving object to be created from the storage unit 4 and calculates the individual presence degree of the moving object by this method. Create a probability density function of a two-dimensional normal distribution in which the object position of the moving object to be created is the average value μ, and 3σ is set to a variance value σ ² that matches one half (W / 2) of the width of the moving object. To do. A significant value as the presence level is set within a range corresponding to the size of the moving object. And the output value in each XY coordinate of the said function is defined as the separate presence degree map of the moving object of creation object.

  The presence calculation unit 52 of the present embodiment creates an individual presence map by the method (A1) for a moving object whose object position has not yet been determined, and for the moving object whose object position has been determined. Updates the individual presence map by the method (A2).

  Next, a method for creating a composite presence map for determining the object position of the object of interest from the individual presence map will be described. FIG. 6 is a schematic diagram illustrating the method by taking as an example the generation of a composite presence map for other objects # 2 to # 4 used for determining the object position of the object of interest # 1. First, the presence calculation unit 52 adds values at positions corresponding to each other in the individual presence maps 201 to 204 of all objects, and divides the added value at each position by the maximum value in the XY coordinates of the addition result. The composite abundance map 210 of all objects is created by normalization. Next, the individual abundance of the position in the individual abundance map 201 of the object of interest # 1 is subtracted from the abundance of each position in the synthesis abundance map 210 of all objects, and the other objects # 2 to # 4 are obtained. A composite abundance map 400 is created. When subtracting the individual presence map 201 of the object of interest from the composite presence map 210, the individual presence map 201 is normalized in the same way as when the composite presence map 210 is created.

  FIG. 7 is a schematic diagram for explaining another method of creating a composite abundance map for object position determination, and shows an example in which the moving object # 1 is an object of interest. In this method, the presence calculation unit 52 corresponds to each of the individual presence maps 202 to 204 of the remaining other objects # 2 to # 4 except for the attention object # 1 among the moving objects # 1 to # 4. A combined presence map 401 for determining the object position of the object of interest is created by adding values between positions and normalizing the addition result of each position by the maximum value in the XY coordinates of the addition result.

  The composite abundance map 400 created by the method described with reference to FIG. 6 includes the composite abundance map 401 created by the method described with reference to FIG. 7 and the essence of the composite abundance distribution in the XY coordinates. The characteristics are the same, but there is a difference in normalization. Specifically, in the method described with reference to FIG. 6, a penalty value at the time of object position determination is set on a scale common to all objects by creating and normalizing a composite abundance map 210 of all objects once. It becomes easy.

  The above-described combined presence degree map for determining the object position of the target object is created based on the candidate positions of other objects excluding only the target object among a plurality of moving objects, and the presence degree of the other object is calculated as the target time. It corresponds to the abundance function given at each position in the monitoring space. When the identification code of the target object and its candidate position are input, the presence level calculation unit 52, based on the combined presence map for object position determination corresponding to the target object, the presence of another object at the candidate position Output degrees.

  It should be noted that the composite presence 1.0 can be overwritten at a position where it is known in advance that a fixture or the like is installed and no moving object can exist.

  The object position determination unit 53 receives a monitoring image and a candidate position where an object can exist in the monitoring space, and evaluates an image feature of a portion corresponding to each candidate position in the monitoring image to calculate an evaluation value. Then, the object position of the moving object is determined based on the evaluation value of each candidate position, and the determination result is stored in the storage unit 4. The candidate position is a predicted position input from the position prediction unit 51.

  The object position determination unit 53 determines an object position for each moving object from an evaluation value calculation unit 530 (likelihood calculation unit) that evaluates each predicted position in consideration of a penalty value and evaluation of each predicted position. A calculation unit 531 is included. Hereinafter, the evaluation value calculation unit 530 and the object position calculation unit 531 will be described in detail.

  The evaluation value calculation unit 530 extracts the feature amount of the moving object from the region corresponding to the predicted position of each moving object in the monitoring image, and the likelihood as the object position of the predicted position according to the degree of feature amount extraction. (Evaluation value) is calculated and output to the object position calculation unit 531. At this time, the evaluation value calculation unit 530 inputs the identification code of the target object and its candidate position to the presence level calculation unit 52 and calculates the level of presence of other objects other than the target object at the candidate position. To correct the likelihood of the predicted position.

If the XY coordinate of the predicted position is (X, Y), the likelihood before correction for the predicted position is L0 (X, Y), and the presence of another object at the position is P (X, Y), the predicted position The likelihood L (X, Y) after correction for is calculated by subtracting the presence as a penalty value from the likelihood before correction as in the following equation (1).
L (X, Y) = L0 (X, Y) -P (X, Y) (1)

  Incidentally, the value range of the abundance is set to [0, 1] by the above-described normalization and coincides with the value range [0, 1] of the likelihood. When the range of abundance is not normalized to [0, 1], the subtraction result on the right side of equation (1) can be less than 0, but L (X, Y) in that case is defined as 0.

Further, the likelihood L (X, Y) after correction is subtracted from the upper limit value Pmax of the presence level (here, it is normalized to “1”) as shown in the following equation (2). The calculated value may be calculated as a correction coefficient by multiplying the likelihood before correction.
L (X, Y) = L0 (X, Y) × {1-P (X, Y)} (2)

  Here, the degree of existence is defined in the range of [0, Pmax]. On the other hand, the penalty value itself is allowed to be set to exceed Pmax, and L (X, Y) is defined as 0 when P (X, Y)> Pmax at the correction stage shown in Equation (2). You may decide.

The likelihood L0 before correction is calculated as follows. The evaluation value calculation unit 530 generates the three-dimensional model 40 in which the moving object model is virtually arranged at the predicted position. Then, the camera parameter 41 is used to project the three-dimensional model 40 onto the imaging surface of the imaging unit 2 to obtain a non-hidden area of the moving object model, and the similarity, inclusion degree, and The degree of edge extraction is calculated, and the likelihood corresponding to these weighted addition values is calculated. Here, the non-hidden region is a region in which the moving object model of interest appears as an image on the monitoring image without being hidden by other moving objects. In addition, by including a model of an installation such as a fixture in the virtual space, a non-hidden area that further considers concealment by the set object can be obtained.
(1) The feature amount extracted from the non-hidden region at the past object position of each moving object is stored in the storage unit 4 as reference information of the moving object. As the predicted position is closer to the position where the moving object actually exists, the feature amount of the background and other moving objects will not be mixed, so the similarity between the feature amount extracted from the non-hidden region and the reference information becomes high, The similarity tends to decrease as the distance increases. Therefore, the feature quantity in the non-hidden region is extracted from the monitoring image, and the similarity between the extracted feature quantity and the reference information is calculated. As the feature amount here, for example, various image feature amounts such as an edge distribution, a color histogram, or both of them can be used.
(2) The non-obscured region is overlapped with the changed pixel extracted by the changed pixel extraction unit 50, and the ratio (inclusion degree) of the changed pixel included in the non-hidden region is obtained. Inclusion degree increases as the predicted position is closer to the position where the moving object actually exists, and tends to decrease as the distance increases.
(3) Extract an edge from a portion corresponding to the contour of the non-hidden region in the monitoring image. The closer the predicted position is to the position where the moving object actually exists, the more the edge extraction degree (for example, the sum of the extracted edge strengths) becomes higher because the contour of the non-hidden region matches the edge position. The degree tends to be low.

  The likelihood corresponding to any one of the above-described similarity, inclusion, and edge extraction may be calculated, or the likelihood may be calculated according to the weighted addition value of two of these. It may be calculated.

  Instead of using the likelihood as an evaluation value, the above-described similarity, inclusion, edge extraction, or a combination of two or more of these can be used as the evaluation value.

The object position calculation unit 531 determines the position (object position) of the moving object from each predicted position of the moving object and the likelihood calculated for each predicted position, and the determination result is stored in the storage unit 4 for each moving object. Accumulate in series. When all the likelihoods are less than a predetermined lower limit (likelihood lower limit), it is determined that there is no object position, that is, disappeared. The following (1) to (3) are examples of the object position calculation method.
(1) For each moving object, a weighted average value of predicted positions weighted by likelihood is calculated, and this is set as the object position of the moving object.
(2) For each moving object, a predicted position where the maximum likelihood is calculated is obtained and set as the object position.
(3) For each moving object, an average value of predicted positions at which likelihoods equal to or higher than a preset likelihood threshold value are calculated is set as the object position. Here, the likelihood threshold> the likelihood lower limit value.

  The abnormality determination unit 54 refers to the time-series object positions accumulated in the storage unit 4, determines that a suspicious movement that stays for a long time or a suspicious movement that deviates from the normal flow line is abnormal, and determines the abnormality. And output an abnormal signal to the output unit 6.

  The output unit 6 includes a sound output means for outputting a warning sound, a display means for displaying a monitoring image determined to be abnormal, or a communication means for transmitting to a security company center device via a communication line. When an abnormality signal is input from the unit 54, the fact that an abnormality has occurred is output to the outside.

[Operation of moving object tracking device]
Next, the tracking operation of the moving object tracking device 1 will be described. FIG. 8 is a schematic flowchart of the tracking process of the moving object tracking device 1.

  When the tracking process is started, the control unit 5 receives a monitoring image every time the imaging unit 2 images the monitoring space (S1). Hereinafter, the time when the latest monitoring image is input is called the current time, and the latest monitoring image is called the current image.

  The current image is compared with the background image by the change pixel extraction unit 50, and the change pixel extraction unit 50 extracts the change pixel (S2). Here, the isolated change pixel is excluded from the extraction result as being caused by noise. Immediately after the start of the operation without a background image, the current image is stored in the storage unit 4 as a background image, and there is no change pixel for convenience.

  Further, the position predicting unit 51 sets α predicted positions for each moving object being tracked based on the motion prediction (S3). Note that the predicted positions of the moving object determined to be a new appearance in step S18, which will be described later, cannot be predicted, and therefore α predicted positions are set in a wider range centering on the appearance position. Further, the predicted position of the moving object determined to be lost in step S18 described later is deleted.

  If the change pixel is not extracted in step S2 and the predicted position is not set in step S3 (there is no moving object being tracked) (if “YES” in S4), the control unit 5 performs step. Returning to S1, the input of the next monitoring image is awaited.

  On the other hand, if “NO” in the step S4, the processes of the steps S5 to S18 are performed. The presence calculating unit 52 creates an individual presence map of the moving object by the method (A1) described above based on the predicted position of each moving object, and overwrites and saves it in the storage unit 4 (S5). Further, the abundance calculating unit 52 creates a combined abundance map of all objects, and overwrites and saves it in the storage unit 4 (S6).

  The controller 5 determines the front-rear relationship of the moving object (S7). Specifically, the distance between the center of gravity (average value) of the predicted position and the camera position is calculated for each moving object, and a contextual list in which the identifiers of the objects are arranged in ascending order of the distance is created. Then, the control unit 5 sets each moving object being tracked as a target object sequentially from the front one based on the context list (S8). Subsequently, the control unit 5 sequentially sets each predicted position of the target object as the target position (S9). However, the predicted position outside the field of view of the monitoring image is excluded from the target position setting targets, the object region at the predicted position is not estimated, and the likelihood is set to zero.

  The evaluation value calculation unit 530 arranges the moving object model at the position of interest in the virtual space, and generates the three-dimensional model 40 on which the moving object model is arranged. Then, the camera parameter 41 is used to project the three-dimensional model 40 onto the imaging surface of the imaging unit 2 to obtain a non-hiding area of the moving object model, and this is estimated as a non-hiding area of the moving object in real space (S10). . Then, based on the estimated non-hidden region, a pre-correction likelihood L0 corresponding to the target position is calculated (S11).

  For example, as described above, the presence calculation unit 52 creates a combined presence map corresponding to the target object from the total presence map of all objects and the individual presence map of the target object. The evaluation value calculation unit 530 acquires the presence level P of other objects at the position of interest using the composite location level map for object position determination created by the presence level calculation unit 52 (S12). The evaluation value calculation unit 530 reads out the presence P of the other objects at the target position from the combined presence map of all the objects, and reads the combined presence of all objects at the target position from the target object individual presence map. It can also be obtained by reading the individual abundance and subtracting the individual abundance from the read combined abundance. The evaluation value calculation unit 530 calculates the corrected likelihood L by performing likelihood correction based on the above-described equation (1) using the presence degree P of the other object (S13).

  The control part 5 repeats step S9-S13, when the prediction position where the likelihood is not calculated remains (in the case of "NO" in S14). When the likelihood is calculated for all α predicted positions (in the case of “YES” in S14), the object position calculation unit 531 calculates each predicted position (X, Y) of the target object and each of the predicted positions. The object position of the object of interest is calculated using the obtained likelihood L (X, Y) (S15). The object position calculated for the current time is added in association with the object position of the object of interest stored in the storage unit 4 one hour before. In the case of a newly appearing moving object, a new identifier is assigned and registered. If the likelihood at all predicted positions is less than the lower limit of likelihood, it is determined that there is no object position.

  The presence level calculation unit 52 updates the individual presence level map of the target object using the method (A2) using the object position calculated in step S15, and uses the updated individual presence level map to update all the presence levels. The composite presence map of the object is also updated (S16). Thereby, in the object position determination process (steps S8 to S16) of a certain target object, the object position determination result of another moving object processed before that is reflected. Since the presence level based on the object position can be expected to have higher accuracy than the presence level based on the predicted position, the update of the above-described presence level map improves the accuracy of the subsequent object position determination, and thus loop processing (steps S8 to S17). ) Is repeated once for all moving objects, it can be expected that a suitably converged state is obtained for the positional relationship between the moving objects. Further, by determining the object position from the object in front, the concealment state of the object behind by the object in front is accurately evaluated, and the accuracy of the likelihood calculated in step S11 is improved. This also improves the object position determination accuracy.

  When an unprocessed moving object remains ("NO" in S17), the control unit 5 repeats the process of determining the object position for the moving object (steps S8 to S16). On the other hand, when the determination of the object position is completed for all the moving objects (in the case of “YES” in S17), the appearance and disappearance of the object are determined (S18). Specifically, the object position calculation unit 531 combines the object regions estimated for each object position, detects a change pixel outside the combination region among the change pixels extracted by the change pixel extraction unit 50, Among the detected change pixels, adjacent change pixels are labeled. If the label is of a size that can be regarded as a moving object, a new appearance is recorded in the storage unit 4 together with the label position (appearance position). If there is a moving object without an object position, the fact that the moving object has disappeared is recorded in the storage unit 4. When the above processing is completed, the processing returns to step S1 in order to perform processing on the monitoring image at the next time.

  FIG. 9 is a schematic diagram for explaining an example of processing performed by the moving object tracking device 1, in which graphs of likelihood L, L0 and abundance P are superimposed on a cross-sectional view of a monitoring space cut out in parallel with the Y axis. It is. Here, in order to simplify the explanation, an example of two moving objects (persons # 1 and # 2) in the monitoring space is taken, and in FIG. 9, a person one hour before is indicated by a solid line, a person at the current time is indicated by a dotted line, The figures show that the persons # 1 and # 2 have moved in the Y-axis direction and are close to each other at the current time. Further, as a graph, a graph 70 approximately representing the distribution of likelihood L0 before correction calculated for each predicted position of the person # 1 on the straight line in the Y-axis direction along the cross section, other than the person # 1 The graph 71 approximately representing the distribution of the abundance P of the person # 2 that is the other object, and the distribution of the likelihood L after correction calculated for each predicted position of the person # 1 A graph 72 is shown. Note that the reference lines (levels with a value of “0”) of these graphs 70 to 72 are made to coincide with the straight lines indicating the reference plane Z = 0. Further, the positive direction of L (and L0) is upward and the positive direction of P is downward, corresponding to the likelihood L after correction being generated by subtracting the penalty value P from the likelihood L0 before correction. Is set. When there are three or more persons, the presence level P of other objects other than the person # 1 is calculated by adding the presence levels of all persons other than the person # 1.

  In the graph 70 of the likelihood L0 before correction, not only one distribution peak position appears at the position of the person # 1, but also a distribution peak appears at the position of the person # 2, and the position of the person # 2 The peak is higher. For this reason, if the determination is made based on the likelihood before correction, the person position of the person # 1 is erroneously determined to be the position of the person # 2.

  In this regard, the moving object tracking device 1 corrects the likelihood L0 of the person # 1 with the presence degree P of the person # 2, and determines the position of the person # 1 using the likelihood L after the correction. The presence level P of the person # 2 has a peak at the position of the person # 2 as indicated by the graph 71. In the corrected likelihood L (thick line graph 72) obtained by subtracting this abundance P from the likelihood L0, the peak of the position of the person # 2 is lowered and smaller than the peak of the position of the person # 1. That is, the maximum peak position of the likelihood L distribution is corrected to the position of the person # 1. By using the likelihood L after the correction, the moving object tracking device 1 uses the corrected peak position to the position of the true person # 1 that is very close to the peak position as the object position of the person # 1. Can be determined.

Note that (A3) described below is an alternative method of (A1), which is a method of creating an individual presence map from the predicted position 42, and is a method of creating an individual presence map from the object position 43. (A4) or (A5) described below can also be used as an alternative method of (A2).
(A3) The distribution of the predicted position is approximated by function for each moving object, and the individual existence degree at each position of the XY coordinates is calculated by the function. Specifically, the presence calculating unit 52 calculates the average value and variance of the predicted position for each moving object, and the distribution of the predicted position using a two-dimensional normal distribution probability density function in which the calculated average value and variance are set. Approximate. Then, the value of the function at each value in the monitoring space is set as the individual presence degree of the object at each position. FIG. 10 is a schematic diagram showing a state in which the distribution of predicted positions is approximated by a two-dimensional normal distribution. A large number of small circles on the XY plane indicate predicted positions, respectively, and a two-dimensional normal distribution (average value μ, variance value). σ ² ) is represented by a mountain shape having a peak in the P-axis direction at the average position μ and a circle on the XY plane indicating a range of 3σ. Compared with (A1), this method cannot express a complicated shape, so the accuracy is low. However, since the processing amount is small, high-speed processing is possible.
(A4) The likelihood of the predicted position is assigned as a weight ω to the two-dimensional normal distribution corresponding to each predicted position in (A1) above. Then, a composite function obtained by weighting and adding each two-dimensional normal distribution set at each predicted position with the likelihood is used as the individual presence map created from the object position for the moving object to be created. At this time, the presence calculation unit 52 performs normalization by dividing each individual presence by the maximum value of the total individual presence.
(A5) A two-dimensional normal distribution in which each predicted position of the moving object to be processed is weighted with likelihood, an average value and variance using the weighted predicted position are calculated, and the calculated average value and variance are set This function is an individual abundance map.

  As the number of moving objects having interrelationships increases beyond three, the combination of two of them increases, so that the positional relationship is determined by decomposing into a two-body problem that is relatively easy to handle. The technique becomes difficult. In this regard, the moving object tracking device 1 according to the present invention, when determining the position of the target object, refers to a composite presence map, that is, a so-called monitoring presence space, regardless of the number of objects other than the target object. By replacing it with a single “field”, even if a large number of moving objects approach, it is possible to quickly perform position determination with suitable accuracy while suppressing trial and error.

  In the above embodiment, the position prediction unit 51 sets a large number of predicted positions for each moving object, and the object position determination unit 53 calculates these likelihoods and performs integrated determination. In another embodiment, the position predicting unit 51 sets one predicted position and an error range centered on each moving object, and the object position determining unit 53 performs a search error while shifting the predicted position within the error range. An evaluation value at each place in the range is calculated, and a position where the highest evaluation value is calculated is determined as an object position. In this case, the presence calculation unit 52 may calculate the individual presence by setting the center predicted position as an average and setting a two-dimensional normal distribution function or the like that is 3σ on the circumference of the error range.

  DESCRIPTION OF SYMBOLS 1 Moving object tracking apparatus, 2 Imaging part, 3 Setting input part, 4 Storage part, 5 Control part, 6 Output part, 40 3D model, 41 Camera parameter, 42 Predicted position, 43 Object position, 44 Presence map, 50 Change pixel extraction unit, 51 Position prediction unit, 52 Presence calculation unit, 53 Object position determination unit, 54 Abnormality determination unit, 100 Reference plane, 101 Camera position, 103 Moving object model, 201-204 Individual presence map, 210 All Object composite presence map 400, 401 Composite presence map for object position determination of target object 530 Evaluation value calculation unit 531 Object position calculation unit.

Claims (3)

A moving object tracking device for tracking a plurality of objects moving in the space using time-series images of a predetermined space,
A storage unit that stores information on the past position of each object in the past from the attention time, and reference information for extracting an image feature of the object;
A position prediction unit that predicts a plurality of candidate positions of the movement destination at the time of interest of each object from the past position;
A presence setting unit that sets each object as a target object, and sets a presence level indicating the probability of existence of the other object in a distribution range of the candidate positions of the other objects excluding the target object among the plurality of objects;
An evaluation value corresponding to the degree to which the image feature is extracted from the portion corresponding to the candidate position of the target object in the image is set in the candidate position by comparing the image at the target time with the reference information. An object position determination unit that calculates a lower position according to the degree of presence and determines a destination position of the target object from the evaluation value of the candidate position;
Equipped with a,
The presence level setting unit sets a presence function representing the probability of existence of the other object in each range having the size of the other object centered on each candidate position of the other object, and the object position determination unit The presence function of the candidate position where the evaluation value is calculated is weighted according to the evaluation value, and the presence function is calculated by adding the weighted presence function value for each position in the space. To do,
A moving object tracking device. The moving object tracking device according to claim 1,
The presence degree setting unit uses the determined destination position instead of the candidate position of the other object for the other object whose destination position is determined by the object position determination unit. A moving object tracking device characterized in that a range in which presence is set is determined. A moving object tracking device for tracking a plurality of objects moving in the space using time-series images of a predetermined space,
A storage unit that stores information on the past position of each object in the past from the attention time, and reference information for extracting an image feature of the object;
A position prediction unit that predicts a plurality of candidate positions of the movement destination at the time of interest of each object from the past position;
A presence setting unit that sets each object as a target object, and sets a presence level indicating the probability of existence of the other object in a distribution range of the candidate positions of the other objects excluding the target object among the plurality of objects;
An evaluation value corresponding to the degree to which the image feature is extracted from the portion corresponding to the candidate position of the target object in the image is set in the candidate position by comparing the image at the target time with the reference information. An object position determination unit that calculates a lower position according to the degree of presence and determines a destination position of the target object from the evaluation value of the candidate position;
With
The presence degree setting unit sets a probability density function corresponding to the distribution of the plurality of candidate positions for each of the other objects, and adds the value of the probability density function of each of the other objects for each position in the space. Calculating the presence level,
A moving object tracking device.

Images