JP6340769B2 - Object position estimation apparatus, object position estimation method, and program - Google Patents

Description

The present invention relates to an object position estimation device, an object position estimation method, and a program.

  Currently, a technique for estimating the position of an object on a moving image or tracking the object on a moving image using a particle filter (particle filter) is known. For example, Patent Document 1 discloses an imaging device that detects a person from a captured image using a particle filter and tracks the area of the detected head portion of the person.

  Here, when the position of the object on the moving image is estimated using the particle filter, a very large number of random numbers are used during the process. The imaging apparatus disclosed in Patent Document 1 employs a technique for generating a random number every time a random number is required during processing.

JP 2010-193333 A

  However, in order to generate random numbers, it is generally necessary to perform complicated calculations. For this reason, as disclosed in Patent Document 1, if a method of generating a random number every time a random number is required during the process is employed, a large processing load is applied to generate the random number. Therefore, the imaging apparatus disclosed in Patent Document 1 cannot execute the process of estimating the position of the object on the moving image at high speed. For this reason, there is a demand for a technique that can execute a process for estimating the position of an object on a moving image at high speed.

The present invention has been made in view of such a situation, and provides an object position estimation device, an object position estimation method, and a program suitable for executing processing for estimating the position of an object on a moving image at high speed. The purpose is to do.

In order to achieve the above object, one aspect of the object position estimation apparatus of the present invention is:
And at least two first random number sequence having a number of samples of the random number, and at least two second random number sequence has a random number of the number of samples, and the third random number sequence having at least two uniform random numbers, A random number sequence generating means for generating
At least the two of the first random number sequence is generated, at least the two of the second random number sequence is generated, and the generated third random number sequence, a random number sequence storing means for storing in advance,
A temporary sample point setting means for setting a temporary sample point having the same coordinates as the sample point as the weight assigned to the sample point increases,
Based on the third random number sequence stored in advance, the first random number sequence is selected from at least two first random number sequences stored in advance, and the selected one random number Temporary sample point selection means for selecting temporary sample points for the number of samples from the set temporary sample points based on the first random number sequence;
Temporary sample point moving means for moving a selected temporary sample point based on the second random number sequence stored in advance;
Sample point setting means for setting a sample point having the same coordinates as the temporary sample point after movement;
Weight giving means for newly giving a larger weight as the degree of similarity between the color at the sample point after movement and the color at the sample point before movement is higher;
An object position estimating means for estimating the position of the object based on the coordinates of the set sample points and the newly assigned weight;
Equipped with a,
And wherein a call.
In order to achieve the above object, one aspect of the object position estimation apparatus of the present invention includes:
At least two first random number sequences having random numbers for the number of samples, at least two second random number sequences having random numbers for the number of samples, and a fourth random number sequence having at least two uniform random numbers; A random number sequence generating means for generating
Random number sequence storage means for storing in advance the generated at least two first random number sequences, the generated at least two second random number sequences, and the generated fourth random number sequence;
A temporary sample point setting means for setting a temporary sample point having the same coordinates as the sample point as the weight assigned to the sample point increases,
Temporary sample point selection means for selecting temporary sample points for the number of samples from the set temporary sample points based on the first random number sequence stored in advance;
Based on the fourth random number sequence stored in advance, two second random number sequences are selected from at least two second random number sequences stored in advance, and the two selected two random number sequences are selected. Temporary sample point moving means for moving the selected temporary sample point based on the second random number sequence;
Sample point setting means for setting a sample point having the same coordinates as the temporary sample point after movement;
Weight giving means for newly giving a larger weight as the degree of similarity between the color at the sample point after movement and the color at the sample point before movement is higher;
An object position estimating means for estimating the position of the object based on the coordinates of the set sample points and the newly assigned weight;
Comprising
It is characterized by that.
In order to achieve the above object, one aspect of the object position estimation method of the present invention includes:
And at least two first random number sequence having a number of samples of the random number, and at least two second random number sequence has a random number of the number of samples, and the third random number sequence having at least two uniform random numbers, A random number sequence generation step for generating
At least the two of the first random number sequence is generated, at least the two of the second random number sequence is generated, and the random number sequence storing step of storing the generated third random number sequence, previously,
A temporary sample point setting step for setting a larger number of temporary sample points having the same coordinates as the sample points as the weight given to the sample points is larger;
Based on the third random number sequence stored in advance, the first random number sequence is selected from at least two first random number sequences stored in advance, and the selected one random number A temporary sample point selection step of selecting temporary sample points corresponding to the number of samples from the set temporary sample points based on the first random number sequence;
A temporary sample point moving step of moving the selected temporary sample point based on the second random number sequence stored in advance;
A sample point setting step for setting a sample point having the same coordinates as the temporary sample point after movement;
A weighting step for newly giving a larger weight as the degree of similarity between the color at the sample point after movement and the color at the sample point before movement is higher;
An object position estimation step for estimating the position of the object based on the coordinates of the set sample points and the newly assigned weights;
Including,
And wherein a call.
In order to achieve the above object, one aspect of the object position estimation method of the present invention includes:
At least two first random number sequences having random numbers for the number of samples, at least two second random number sequences having random numbers for the number of samples, and a fourth random number sequence having at least two uniform random numbers; A random number sequence generation step for generating
A random number sequence storage step for storing in advance the generated at least two first random number sequences, the generated at least two second random number sequences, and the generated fourth random number sequence;
A temporary sample point setting step for setting a larger number of temporary sample points having the same coordinates as the sample points as the weight given to the sample points is larger;
A temporary sample point selecting step for selecting temporary sample points for the number of samples from the set temporary sample points based on the first random number sequence stored in advance;
Based on the fourth random number sequence stored in advance, two second random number sequences are selected from at least two second random number sequences stored in advance, and the two selected two random number sequences are selected. A temporary sample point moving step of moving the selected temporary sample point based on the second random number sequence;
A sample point setting step for setting a sample point having the same coordinates as the temporary sample point after movement;
A weighting step for newly giving a larger weight as the degree of similarity between the color at the sample point after movement and the color at the sample point before movement is higher;
An object position estimation step for estimating the position of the object based on the coordinates of the set sample points and the newly assigned weights;
including,
It is characterized by that.
In order to achieve the above object, one aspect of the program of the present invention is:
Computer
And at least two first random number sequence having a number of samples of the random number, and at least two second random number sequence has a random number of the number of samples, and the third random number sequence having at least two uniform random numbers, Random number sequence generation means for generating
At least two of the first random number sequence is generated, at least the two of the second random number sequence is generated, the generated third random number sequence and the random number sequence storing means for storing in advance,
Temporary sample point setting means for setting a larger number of temporary sample points having the same coordinates as the sample points as the weight given to the sample points is larger,
Based on the third random number sequence stored in advance, the first random number sequence is selected from at least two first random number sequences stored in advance, and the selected one random number Temporary sample point selection means for selecting temporary sample points corresponding to the number of samples from the set temporary sample points based on the first random number sequence;
Temporary sample point moving means for moving the selected temporary sample point based on the second random number sequence stored in advance;
Sample point setting means for setting a sample point having the same coordinates as the temporary sample point after movement,
Weight giving means for newly giving a larger weight as the degree of similarity between the color at the sample point after movement and the color at the sample point before movement is higher;
Object position estimating means for estimating the position of the object based on the coordinates of the set sample points and the newly assigned weights;
To function as a,
And wherein a call.
In order to achieve the above object, one aspect of the program of the present invention is:
Computer
At least two first random number sequences having random numbers for the number of samples, at least two second random number sequences having random numbers for the number of samples, and a fourth random number sequence having at least two uniform random numbers; Random number sequence generation means for generating
Random number sequence storage means for storing in advance the generated at least two first random number sequences, the generated at least two second random number sequences, and the generated fourth random number sequence;
Temporary sample point setting means for setting a larger number of temporary sample points having the same coordinates as the sample points as the weight given to the sample points is larger,
Temporary sample point selection means for selecting temporary sample points for the number of samples from the set temporary sample points based on the first random number sequence stored in advance;
Based on the fourth random number sequence stored in advance, two second random number sequences are selected from at least two second random number sequences stored in advance, and the two selected two random number sequences are selected. Temporary sample point moving means for moving the selected temporary sample point based on the second random number sequence;
Sample point setting means for setting a sample point having the same coordinates as the temporary sample point after movement,
Weight giving means for newly giving a larger weight as the degree of similarity between the color at the sample point after movement and the color at the sample point before movement is higher;
Object position estimating means for estimating the position of the object based on the coordinates of the set sample points and the newly assigned weights;
Function as
It is characterized by that.

  According to the present invention, processing for estimating the position of an object on a moving image can be executed at high speed.

It is a block diagram which shows the structure of the object position estimation apparatus which concerns on embodiment of this invention. It is a block diagram for demonstrating the function of the object position estimation apparatus which concerns on embodiment of this invention. It is a figure which shows the random number sequence for selection. (A) is a figure which shows the random number sequence for horizontal axis movement. (B) is a figure which shows the random number sequence for a vertical axis | shaft movement. (A) is a figure which shows the random number sequence for selection random number sequences. (B) is a diagram showing a random number sequence for selecting a random number sequence for horizontal axis movement. (C) is a diagram showing a random number sequence for selecting a random number sequence for moving the vertical axis. It is a figure which shows the sample point set to the set frame image. It is a figure which shows a mode that a temporary sample point is set for every sample point. It is a figure which shows the temporary sample point set for every sample point. It is a figure which shows the selected temporary sample point. It is a figure which shows the sample point set to the setting object frame image. It is a flowchart which shows the object position estimation process which the object position estimation apparatus which concerns on embodiment of this invention performs.

  Hereinafter, an object position estimation apparatus 100 according to an embodiment of the present invention will be described with reference to the drawings.

  The object position estimation device 100 is a device that estimates the position of an object to be tracked on a moving image acquired from the imaging device 200. As shown in FIG. 1, the object position estimation apparatus 100 physically includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a flash memory 14, an RTC (Real Time Clock) 15, touch screen 16, and interface 17. Each component included in the object position estimation apparatus 100 is connected to each other via a bus.

  The CPU 11 controls the overall operation of the object position estimation apparatus 100. The CPU 11 operates according to a program stored in the ROM 12 and uses the RAM 13 as a work area.

  The ROM 12 stores a program and data for controlling the overall operation of the object position estimation apparatus 100.

  The RAM 13 functions as a work area for the CPU 11. That is, the CPU 11 temporarily writes programs and data in the RAM 13 and refers to these programs and data as appropriate.

  The flash memory 14 is a nonvolatile memory that stores various types of information. For example, the flash memory 14 stores various random number sequences and moving images. Note that the object position estimation apparatus 100 may include a hard disk or the like instead of the flash memory 14.

  The RTC 15 is a time measuring device. The RTC 15 incorporates a battery, for example, and continues timing while the power of the object position estimation apparatus 100 is off. The RTC 15 includes an oscillation circuit including a crystal oscillator, for example.

  The touch screen 16 detects a touch operation performed by the user and supplies a signal indicating the detection result to the CPU 11. The touch screen 16 displays an image based on the image signal supplied from the CPU 11 or the like. As described above, the touch screen 16 functions as a user interface of the object position estimation apparatus 100.

  The interface 17 communicates with the imaging device 200 by ad hoc communication or network communication. The interface 17 controls the operation of the imaging device 200 in accordance with instructions from the CPU 11. The interface 17 acquires a moving image from the imaging device 200 and supplies the moving image to the flash memory 14 or the touch screen 16. The interface 17 may be an interface for serial communication such as USB (Universal Serial Bus) or IEEE (Institute of Electrical and Electronic Engineers) 1394, or a LAN (Local Area Network) interface such as a NIC (Network Interface Card). It may be.

  The imaging apparatus 200 captures an object to be tracked and generates a moving image. The object to be tracked may be an artificial object such as an airplane, a car, or a ball, or may be a natural object such as a star, an animal, a person, or an insect. The imaging apparatus 200 includes an interface for communicating with the object position estimation apparatus 100. The imaging device 200 generates a moving image according to an instruction from the object position estimation device 100. The imaging apparatus 200 supplies the generated moving image to the object position estimation apparatus 100. The imaging device 200 is, for example, a digital camera, a video camera, or a smartphone.

  Next, basic functions of the object position estimation apparatus 100 will be described with reference to FIG. As shown in FIG. 2, the object position estimation apparatus 100 includes a random number sequence generation unit 101, a random number sequence storage unit 102, a temporary sample point setting unit 103, a temporary sample point selection unit 104, a temporary sample point moving unit 105, and a sample point setting. Unit 106, weight assigning unit 107, object position estimating unit 108, and moving image acquiring unit 109.

  The random number sequence generation unit 101 generates a selection random number sequence (first random number sequence) and a moving random number sequence (second random number sequence). The selection random number sequence is a random number sequence for selecting temporary sample points corresponding to the number of samples (number of sample points) from the set temporary sample points. Therefore, the selection random number sequence has random numbers corresponding to the number of samples. In the present embodiment, the random number is a pseudo-random number. The random number sequence for movement is a random number for moving the selected temporary sample point. Therefore, the random number sequence for movement has random numbers for the number of samples.

  Here, generally, in order to generate a random number, it is necessary to perform a complicated operation. For this reason, in order to generate a random number, a long processing time is required. For this reason, for example, if the object position estimation apparatus 100 generates random numbers at any time immediately before using random numbers, the object position estimation process cannot be executed at high speed. Therefore, in the present embodiment, the object position estimation device 100 generates a random number at a timing that does not decrease the processing speed of the object position estimation process. Specifically, for example, immediately after the power of the object position estimation apparatus 100 is turned on, the random number sequence generation unit 101 generates a selection random number sequence and a movement random number sequence in advance. In addition, it is preferable that the random number sequence generation unit 101 generates random numbers continuously in order to generate random numbers corresponding to the number of samples at high speed. The random number sequence generation unit 101 includes, for example, a CPU 11 and an RTC 15.

  The random number sequence storage unit 102 stores the selection random number sequence and the movement random number sequence generated by the random number sequence generation unit 101. The random number sequence storage unit 102 includes, for example, a CPU 11 and a flash memory 14. Then, the CPU 11 stores the generated selection random number sequence and movement random number sequence in the flash memory 14 or the like.

  The temporary sample point setting unit 103 sets a temporary sample point having the same coordinates as the sample point for each of the sample points for the number of samples set in the set frame image. Here, the sample point is a point indicating the position of the estimated object on the moving image or the frame image constituting the moving image. Here, the set frame image is a frame image in which sample points have been set among the frame images constituting the moving image. A weight is assigned to the sample point. In the present embodiment, the weight given to the sample point is a likelihood indicating the likelihood as a point indicating the position of the object at the sample point.

  Here, the temporary sample point setting unit 103 sets a larger number of temporary sample points as the weight given to the sample points is larger. The temporary sample point is a temporary sample point for setting a new sample point in the setting target frame image. The sample point and the temporary sample point are represented by, for example, coordinates on the frame image (X axis coordinates, Y axis coordinates). The temporary sample point setting unit 103 includes, for example, a CPU 11.

  The temporary sample point selection unit 104, based on the selection random number sequence stored in the random number sequence storage unit 102, out of the temporary sample points set by the temporary sample point setting unit 103 is the number of temporary sample points. Select. The temporary sample point selection unit 104 can select one temporary sample point based on one random number included in the selection random number sequence. Further, the temporary sample point selection unit 104 can select a temporary sample point while allowing duplication. That is, the temporary sample point selection unit 104 can select one temporary sample point from the temporary sample points set by the temporary sample point setting unit 103 a plurality of times. The temporary sample point selection unit 104 includes, for example, a CPU 11.

  The temporary sample point moving unit 105 moves each of the temporary sample points selected by the temporary sample point selecting unit 104 based on the moving random number sequence stored in the random number sequence storage unit 102. For example, the temporary sample point moving unit 105 can move one temporary sample point based on one random number included in the moving random number sequence. Note that moving the temporary sample point means changing the coordinates of the temporary sample point according to the value of the random number. Further, the temporary sample point moving unit 105 may move the temporary sample points according to only the random number value, or may move the temporary sample points according to the random number value and the predicted equation of motion.

  This equation of motion is an equation of motion for predicting the position of the object on the frame image. This equation of motion may be a predicted equation of motion before the object position estimation process is executed, or is predicted based on the estimated locus of the object position during the execution of the object position estimation process. The equation of motion may be The temporary sample point moving unit 105 includes, for example, a CPU 11.

  The sample point setting unit 106 sets the same sample points as the temporary sample points after being moved by the temporary sample point moving unit 105 in the setting target frame image. It should be noted that the setting target frame image is a frame image for which a sample point is to be set among frame images constituting a moving image, and is a frame image for which a sample point is about to be set. Note that setting the sample point in the setting target frame image based on the sample point set in the set frame image is called resampling.

  In the present embodiment, the setting target frame image is a frame image next to the latest set frame image among the frame images constituting the moving image, and becomes the latest set frame image when the sample point is set. Shall. That is, in this embodiment, sample points are set for all the frame images constituting the moving image. Note that, by resampling, a sample point follows an object that moves over time on a moving image that is an aggregate of frame images on the moving image (on the frame image). The sample point setting unit 106 includes, for example, a CPU 11.

  The weight assigning unit 107 assigns a weight to each of the sample points set by the sample point setting unit 106. The weight given to the sample point is given according to the probability (likelihood) as a point indicating the position of the object at the sample point, for example. For example, the weighting unit 107 sets the color at the sample point corresponding to the sample point set in the set frame image and the set point frame image to the sample point set in the setting target frame image. The higher the degree of similarity with the color at the sample point, the greater the weight. The sample point corresponding to the sample point set in the setting target frame image is a temporary sample having the same coordinates as the sample point set in the setting target frame image among the sample points set in the set frame image. This is a sample point set to the coordinates before the point is moved.

  Here, an aggregate of pixels representing an object (or a part of the object) is predicted to have a high probability that there is no difference in luminance value (high color similarity). Therefore, assuming that the sample point set in the set frame image appropriately indicates the position of the object, when the corresponding sample point set in the setting target frame image appropriately indicates the position of the object, It is considered that the color similarity is increased. In other words, the above-described high degree of color similarity means that there is a high probability that the resampling method is appropriate according to the movement of the object.

  Therefore, the weight assigning unit 107 assigns a greater weight to the sample points as the color similarity is higher. On the other hand, the temporary sample point setting unit 103 sets a larger number of temporary sample points having the same coordinates as the sample points as the weight assigned to the sample points is larger. As this process is repeated, the number of sample points that appropriately indicate the position of the object increases, and the number of sample points that do not properly indicate the position of the object decreases. As a result, the sample points set on the moving image (on the frame image) appropriately indicate the position of the object with high probability. That is, the sample point is updated so as to appropriately follow the position of the object on the moving image (on the frame image). The weight assigning unit 107 includes, for example, a CPU 11.

  The object position estimation unit 108 is based on the coordinates of the sample points set in the setting target frame image and the weights given to the sample points set in the setting target frame image by the weighting unit 107. Thus, the position of the object on the moving image is estimated. For example, the object position estimation unit 108 obtains the coordinates of the center of gravity of all the sample points by weighting the weight given to the sample points and obtaining the average value of the coordinates of all the sample points. Then, the object position estimation unit 108 estimates the coordinates of the center of gravity of all the sample points as the coordinates of the center of the object. The object position estimation unit 108 includes, for example, the CPU 11.

  The moving image acquisition unit 109 acquires a moving image from the imaging device 200. The moving image acquisition unit 109 can supply frame images constituting the acquired moving image one by one to the weighting unit 107 and the like. The moving image acquisition unit 109 includes, for example, a flash memory 14 and an interface 17.

  Here, the selection random number sequence may have a uniform random number corresponding to the number of samples. The uniform random number is a random number such that all real numbers existing within a finite interval appear with the same probability. Here, the selection random number sequence is used to select temporary sample points corresponding to the number of samples from the set temporary sample points. Therefore, the uniform random number included in the selection random number sequence is, for example, a natural number equal to or less than the upper limit value of the number of provisional sample points to be set.

  Further, the moving random number sequence may have normal random numbers corresponding to the number of samples. The normal random number is a random number such that all real numbers existing within a finite interval appear with a probability according to a normal distribution. When actually estimating the position of an object, it is difficult to consider that the object moves the maximum distance assumed, that is, the object always keeps moving at the maximum speed assumed, and in the next frame, a distance shorter than the maximum distance is assumed. There is a high possibility of moving. Therefore, if a normal random number with an expected value of 0 is used, the probability that a value close to 0 will be increased, and “the possibility of moving to the assumed farthest position is low and the possibility of moving to a closer position is high” is said. The situation can be reflected. Also, other random numbers may be used as long as such a probability distribution. Here, the random number sequence for movement is used to move the selected temporary sample point. Accordingly, the normal random number included in the moving random number sequence is, for example, a natural number having an absolute value equal to or less than the upper limit value of the moving amount (number of pixels) of the temporary sample point.

  Here, the random number sequence generation unit 101 selects at least two selection random number sequences, at least two transfer random number sequences, and at least two selection random number sequence selection random number sequences (third Random number sequence) and a moving random number sequence selection random number sequence (fourth random number sequence) having at least two uniform random numbers can be generated.

  Each of the at least two selection random number sequences has a random number corresponding to the number of samples. Each of the at least two moving random number sequences has a random number corresponding to the number of samples. Here, each of the at least two uniform random numbers included in the selection random number sequence selection random number sequence is, for example, a natural number whose upper limit is the number of selection random number sequences. Each of the at least two uniform random numbers included in the moving random number sequence selection random number sequence is, for example, a natural number whose upper limit is the number of moving random number sequences.

  On the other hand, the random number sequence storage unit 102 includes at least two selection random number sequences generated by the random number sequence generation unit 101, at least two transfer random number sequences, a selection random number sequence selection random number sequence, and a transfer random number sequence selection. The random number sequence can be stored.

  Here, the temporary sample point selection unit 104 is based on the selection random number sequence selection random number sequence stored in the random number sequence storage unit 102, and at least two selection random numbers stored in the random number sequence storage unit 102. One selection random number sequence can be selected from the sequence. Then, the temporary sample point selection unit 104 selects temporary sample points corresponding to the number of samples from the temporary sample points set by the temporary sample point setting unit 103 based on the selected single random number sequence for selection. can do.

  The temporary sample point moving unit 105 also includes at least two moving random number sequences stored in the random number sequence storage unit 102 based on the moving random number sequence selection random number sequence stored in the random number sequence storage unit 102. Two random numbers for movement can be selected from the list. Then, the temporary sample point moving unit 105 can move each of the temporary sample points selected by the temporary sample point selecting unit 104 based on the two selected moving random number sequences. Here, the temporary sample point moving unit 105 moves each of the temporary sample points in the X-axis direction based on, for example, one of the selected two moving random number sequences, and moves the other of the temporary sample points. Each of the temporary sample points can be moved in the Y-axis direction based on the moving random number sequence.

  Next, the random number sequence for selection will be described with reference to FIG. As described above, the selection random number sequence is used to select temporary sample points corresponding to the number of samples from the set temporary sample points.

  FIG. 3 shows an example in which the number of selection random number sequences is M and the number of random numbers included in the selection random number sequence is N. Here, M is the number (number of patterns) of a random number sequence prepared in advance, and is a natural number. N is the number of samples (the number of sample points set in one frame image), which is a natural number. Here, Rs [m, n] represents the nth random number in the mth selection random number sequence. Note that Rs [m, n] is a natural number having, for example, the upper limit value of the number of provisional sample points to be set. Further, m is a natural number equal to or less than M, and n is a natural number equal to or less than N.

  Next, the moving random number sequence will be described with reference to FIG. 4A shows a random number sequence for horizontal axis movement, and FIG. 4B shows a random number sequence for vertical axis movement. The random number sequence for horizontal axis movement is used to determine the amount of movement of the selected temporary sample point in the horizontal axis direction (X-axis direction). On the other hand, the random number sequence for vertical axis movement is used to determine each movement amount of the selected temporary sample point in the vertical axis direction (Y-axis direction).

  FIG. 4A shows an example in which the number of random number sequences for horizontal axis movement is M and the number of random numbers included in the random number sequence for horizontal axis movement is N. Here, Rmx [m, n] represents the nth random number in the mth horizontal axis moving random number sequence. Note that Rmx [m, n] is an integer whose upper limit value is the upper limit value of the number of pixels that can be moved in the horizontal axis direction, for example.

  FIG. 4B shows an example in which the number of random number sequences for vertical axis movement is M, and the number of random numbers included in the random number sequence for vertical axis movement is N. Here, Rmy [m, n] represents the nth random number in the mth vertical axis moving random number sequence. Note that Rmy [m, n] is an integer whose upper limit value is the upper limit value of the number of pixels movable in the vertical axis direction, for example.

  Next, a random number sequence selection random number sequence will be described with reference to FIG. 5A shows a selection random number sequence selection random number sequence, FIG. 5B shows a horizontal axis movement random number sequence selection random number sequence, and FIG. 5C shows a vertical axis movement. The random number sequence for random number is shown. The selection random number sequence is used to select one selection random number sequence from among the M selection random number sequences. The horizontal axis movement random number sequence selection random number sequence is used to select one horizontal axis movement random number sequence from among the M horizontal axis movement random number sequences. The vertical axis moving random number sequence selection random number sequence is used to select one vertical axis moving random number sequence from among the M vertical axis moving random number sequences.

  FIG. 5A illustrates an example in which the number of random numbers included in the selection random number sequence selection random number sequence is G. Here, G is a natural number. Rss [g] represents the g-th random number in the selection random number sequence selection random number sequence. Here, Rss [g] is, for example, a natural number having M as an upper limit value. Note that g is a natural number equal to or less than G.

  FIG. 5B shows an example in which the number of random numbers included in the random number sequence for selecting a random number sequence for moving on the horizontal axis is G. Rmxs [g] indicates the g-th random number in the random number sequence for selecting a horizontal axis moving random number sequence. Here, Rmxs [g] is, for example, a natural number with M as an upper limit value.

  FIG. 5C illustrates an example in which the number of random numbers included in the random number sequence for selecting a vertical axis moving random number sequence is G. Rmys [g] represents the g-th random number in the random number sequence for selecting a vertical axis moving random number sequence. Here, Rmys [g] is, for example, a natural number with M as an upper limit value.

  In addition, before starting the main process of the object position estimation process, the CPU 11 performs M selection random number sequences each including N random numbers in the preprocess of the object position estimation process. M random number sequences for moving along the horizontal axis, M random numbers for moving along the vertical axis each including N random numbers, a random number sequence for selecting random numbers including G random numbers, and G random numbers A random number sequence for selecting a random number sequence for moving the horizontal axis and a random number sequence for selecting a random number sequence for moving the vertical axis including G random numbers are generated and stored in the flash memory 14 or the like.

  Next, the sample points set in the set frame image will be described with reference to FIG.

The sample point is a point indicating the estimated position of the object to be followed on the moving image. However, whether the object position estimation apparatus 100, and immediately after the object position estimation process is started, immediately after lost sight of the object in the middle of the object position estimating process, there is an object in Ri per any of the moving picture May not be estimated accurately. In such a case, as shown in FIG. 6, sample points are set in the set frame image at positions that do not depend on the position of the object.

  In FIG. 6, Ps [1], Ps [2],..., Ps [10] indicate the first, second,. FIG. 6 shows an example in which the number of sample points is 10 (an example where N = 10) for easy understanding. The sample points shown in FIG. 6 do not accurately indicate the position of the object 600. However, as will be described later, the sample point gradually indicates the position of the object 600 by appropriately updating the position of the sample point each time re-sampling is performed. Note that the number of samples remains unchanged even if resampled.

  Here, the temporary sample points set for each of the sample points set in the set frame image will be described. FIG. 7 is a diagram illustrating a state in which the number of temporary sample points corresponding to the likelihood given to the sample points is set for each of the sample points set in the set frame image. The likelihood given to the sample point is a concept indicating the likelihood that the sample point is a point indicating the position of the object on the set frame image. Thus, in this embodiment, likelihood is adopted as the weight given to the sample points.

  FIG. 7 shows that the number of temporary sample points set for a sample point increases as the likelihood assigned to the sample point increases. As described above, when the number of temporary sample points having the same coordinates as the sample points having a high likelihood as a point indicating the position of the object is large, these temporary sample points are easily selected by a selection process described later. As a result, it is expected that the likelihood given to the sample points increases as a whole, and the position of the object is accurately indicated by the sample points.

  In the present embodiment, in order to facilitate understanding, the likelihood is a natural number, and the number of temporary sample points indicated by the likelihood is set. For example, since the likelihood given to the first sample point (Ps [1]) is 1, one temporary sample point (Psp [1]) is set for the first sample point. Is done. On the other hand, since the likelihood given to the 10th sample point (Ps [10]) is 6, for the 10th sample point, 6 temporary sample points (Psp [15] −Psp [ 20]) is set.

  Each of Psp [1], Psp [2],..., Psp [20] represents a temporary sample point. Here, the coordinates of the temporary sample points set for the sample points are the same as the coordinates of the sample points. For example, the coordinates of Ps [1] and Psp [1] are the same. Further, the coordinates of Ps [10] and the coordinates of Psp [15] -Psp [20] are the same.

  FIG. 8 is a diagram showing temporary sample points set for each sample point. FIG. 8 shows a state where temporary sample points set for each sample point are set in a setting target frame image for easy understanding. Further, in FIG. 8, temporary sample points having the same coordinates are displayed adjacent to each other without being overlapped for easy understanding. For example, Psp [3], which is the third temporary sample point, and Psp [4], which is the fourth temporary sample point, have the same coordinates but are displayed without being overlapped so that they can be easily seen.

  Next, a method for selecting temporary sample points for the number of samples from the set temporary sample points will be described. FIG. 9 shows a state where temporary sample points for the number of samples are selected from the set temporary sample points. In FIG. 9, 20 temporary sample points (Psp [1] -Psp [20]) to 10 temporary sample points (Psp [3], Psp [5], Psp [6], Psp [10], In this example, Psp [11], Psp [13], Psp [14], Psp [15], Psp [16], and Psp [17] are selected.

  Here, the temporary sample points are selected based on the selection random number sequence and the selection random number sequence selection random number sequence. First, one random number is selected from the G random numbers included in the selection random number sequence. In this embodiment, it is assumed that a random number having an element number indicated by a counter variable for selecting a random number is selected from a random number sequence for selecting a random number sequence for selection. As the counter variable, for example, an initial value is 1, which is incremented by 1 each time a random number is selected, and a variable set to 1 after G can be adopted.

  One random number selected from the random number sequence for selection random number sequence is a natural number of M or less. Accordingly, one selection random number sequence is selected from the M selection random number sequences based on one random number selected from the selection random number sequence selection random number sequence. Next, one temporary sample point is selected based on each of the N random numbers included in the selected selection random number sequence, and a total of N temporary sample points are selected. In this embodiment, N = 10.

  Here, the random number included in the selection random number sequence is a natural number equal to or less than the upper limit value of the number of temporary sample points to be set. On the other hand, the number of temporary sample points to be set may change depending on the likelihood. Therefore, one temporary sample point is appropriately selected from one random number. For example, assuming that one selected random number is Rs [m, n] and the number of set temporary sample points is Nsp, one selected temporary sample point is Psp [(Rs [m, n]. −1)% Nsp + 1]. Here, “%” is a remainder operator.

  For example, when Rs [m, n] that is one selected random number is 25 and Nsp that is the number of set temporary sample points is 20, Psp [5] that is the fifth temporary sample point is Selected. As a result of the processing described above, temporary sample points corresponding to the number of samples are selected.

  Next, a method for moving each of the selected temporary sample points will be described.

  The temporary sample point moves based on the random number sequence for moving the horizontal axis, the random number sequence for moving the vertical axis, the random number sequence for selecting the horizontal axis moving random number sequence, and the random number sequence for selecting the vertical axis moving random number sequence. First, one random number is selected from the G random numbers included in the random number sequence for selecting the horizontal axis movement random number sequence, and from among the G random numbers included in the random number sequence for selecting the random number sequence for moving the vertical axis. One random number is selected. In the present embodiment, it is assumed that the random number of the element number indicated by the counter variable described above is selected from the random number sequence for selecting the horizontal axis moving random number sequence and the random number sequence for selecting the vertical axis moving random number sequence. That is, in this embodiment, a counter variable for selecting a random number from the random number sequence for selection random number sequence, a counter variable for selecting a random number from the random number sequence for moving the horizontal axis, and a vertical axis movement The counter variable for selecting a random number from the random number sequence for random number selection is shared.

  Note that one random number selected from the random number sequence for selecting the horizontal axis moving random number sequence is a natural number equal to or less than M, and one random number selected from the random number sequence for selecting the vertical axis moving random number sequence is M The following natural numbers. Accordingly, one horizontal axis moving random number sequence is selected from among the M horizontal axis moving random number sequences based on one random number selected from the moving random number sequence selecting random number sequence, and M One vertical axis moving random number sequence is selected from among the vertical axis moving random number sequences. Next, based on each of the N random numbers included in the selected horizontal axis movement random number sequence, one temporary sample point moves in the horizontal axis direction and is included in the selected vertical axis movement random number sequence. On the basis of each of the N random numbers, one temporary sample point moves in the vertical axis direction. As a result, each of the N temporary sample points moves in the horizontal axis direction and the vertical axis direction.

  Note that the amount of movement of the temporary sample points is determined such as how many pixels in the horizontal axis direction and how many pixels in the vertical axis direction. Accordingly, the random number included in the random number sequence for horizontal axis movement is preferably a number in the range from -Nx to + Nx, where Nx is the maximum number of pixels that can be moved in the horizontal axis direction. Further, the random number included in the vertical axis moving random number sequence is preferably a number in the range from -Ny to + Ny, where Ny is the maximum number of pixels that can be moved in the vertical axis direction.

  Here, the motion equation of the tracking target object may be predictable or not predictable. As a case where the equation of motion can be predicted, for example, in addition to the case where the motion of the object can be predicted before the object position estimation process is started, the object position is estimated based on the locus of the object position during the object position estimation process. A case where motion can be predicted is conceivable. On the other hand, as a case where the equation of motion is not predictable, for example, when the object movement cannot be predicted immediately after the start of the object position estimation process, or the object position is lost during the object position estimation process, The case where it becomes impossible to predict is considered.

  Here, when the equation of motion can be predicted, the temporary sample point moves in the horizontal axis direction and the vertical axis direction according to the equation of motion, and then moves in the horizontal axis direction and the vertical axis direction according to a random number. In this case, for example, the likelihood given to the sample points existing in the background position is reduced, and as a result, it can be expected that the sample points are gathered near the object.

  On the other hand, when the equation of motion is not predictable, the temporary sample point moves in the horizontal axis direction and the vertical axis direction according to a random number. Even in this case, since the likelihood given to the sample point moved together with the object on the moving image is high, it can be expected that the sample point moved together with the object survives. In particular, the luminance when the position of the pixel of interest in the background portion is shifted (for example, the difference between the luminance values of adjacent pixels in the background portion) shifts the position of the pixel of interest in the object portion. When the value is relatively higher than the change in the value (for example, the difference between the luminance values of adjacent pixels in the object portion), it can be expected that the sample points gather near the object. That is, when the color variation of the object portion is smaller than the color variation of the background portion, it can be expected that sample points gather near the object even if the equation of motion is not predictable.

  Here, each of the temporary sample points after movement is set as a sample point for the setting target frame image. FIG. 10 shows ten temporary sample points after movement (Psp [3], Psp [5], Psp [6], Psp [10], Psp [11], Psp [13], Psp [14], Psp. [15], Psp [16], Psp [17]) are 10 sample points (Ps [1], Ps [2], Ps [3], Ps [4], Ps) for the setting target frame image. [5], Ps [6], Ps [7], Ps [8], Ps [9], Ps [10]).

  As is clear from the comparison between FIG. 6 and FIG. 10, when the resampling process is executed so that the sample points based on the assigned sample points having a high likelihood are likely to survive, the sample points are located near the object. Is expected to gather. Therefore, according to the object position estimation process in which such resampling process is repeatedly performed, it can be expected to accurately grasp the position of the object.

  Here, a weight is given to each of the sample points set in the setting target frame image. For example, the weight given to a certain sample point set in the setting target frame image includes the color in the set frame image of this sample point and the set frame of the sample point on the set frame image corresponding to this sample point. The higher the degree of similarity with the color in the image, the higher the setting.

  Note that the method for defining the similarity can be adjusted as appropriate. For example, when the frame image is represented by three components of R (Red), G (Green), and B (Blue), the smaller the average value of the luminance values obtained for each of the three components, The degree of similarity may be increased. Alternatively, among the three components, the similarity may be higher as the difference in the luminance value of the component having the largest luminance value difference is smaller. Or you may make it a similarity become high, so that the difference of the luminance value of the component to notice is small among three components.

  Next, the object position estimation process executed by the object position estimation apparatus 100 will be described using the flowchart shown in FIG. When the power is turned on, the object position estimation apparatus 100 executes an object position estimation process shown in FIG.

  First, the CPU 11 executes an initialization process (step S101). Specifically, the CPU 11 initializes coordinates and likelihood for each of the N sample points. For example, the CPU 11 determines the coordinates of each of the N sample points so that the N sample points are arranged at equal intervals in the horizontal axis direction and the vertical axis direction over the entire region on the moving image. To do. In addition, the CPU 11 gives the same likelihood to each of the N sample points. The CPU 11 stores the coordinates and likelihood of each of the N sample points in the RAM 13.

  Further, the CPU 11 initializes a counter variable stored in the RAM 13 to 1. Further, the CPU 11 causes the imaging device 200 to start imaging via the interface 17. On the other hand, the imaging apparatus 200 starts imaging in accordance with an instruction from the CPU 11. Thereafter, the imaging apparatus 200 supplies a moving image generated by imaging to the interface 17. Here, the interface 17 supplies and supplies the supplied moving image to the flash memory 14.

  When completing the process in step S101, the CPU 11 generates a random number sequence (step S102). More specifically, the CPU 11 performs M selection random number sequences, M horizontal axis movement random number sequences, and M vertical lengths by various calculations or using values acquired from the RTC 15. An axis movement random number sequence, a selection random number sequence selection random number sequence, a horizontal axis movement random number sequence selection random number sequence, and a vertical axis movement random number sequence selection random number sequence are generated. The CPU 11 can generate a normal random number from the uniform random number by using the Box = Mueller method.

  When completing the process in step S102, the CPU 11 stores the random number sequence generated in step S102 (step S103). Specifically, the CPU 11 selects M random number sequences for selection, M random number sequences for horizontal axis movement, M random number sequences for vertical axis movement, a random number sequence for selection random number sequence, A random number sequence for selecting an axial movement random number sequence and a random number sequence for selecting a vertical axis moving random number sequence are stored in the flash memory 14.

  When completing the process in step S103, the CPU 11 acquires a setting target frame image (step S104). Specifically, the CPU 11 acquires the latest frame image among the frame images constituting the moving image stored in the flash memory 14. That is, in the present embodiment, the setting target frame image is the latest frame image. The CPU 11 stores the acquired setting target frame image in the RAM 13 as appropriate.

  When completing the process in step S104, the CPU 11 sets a temporary sample point (step S105). For example, for each of the N sample points set in the set frame image (latest set frame image), the CPU 11 increases the number of samples and samples as the likelihood assigned to the sample points is higher. Set a temporary sample point with the same coordinates as the point. If there is no set frame image, the CPU 11 sets a temporary sample point based on the coordinates and likelihood of the sample point set in the initialization process in step S101.

  When completing the process of step S105, the CPU 11 selects a temporary sample point (step S106). Specifically, first, the CPU 11 acquires a random number of an element number indicated by a counter variable among random numbers included in the random number sequence for selection random number sequence. Then, the CPU 11 acquires a selection random number sequence indicated by the acquired random number from among the M selection random number sequences. Here, the CPU 11 selects one temporary sample point from the set temporary sample points based on each of the N random numbers included in the acquired selection random number sequence. That is, the CPU 11 selects N temporary sample points while allowing duplication based on N random numbers.

  When completing the process of step S106, the CPU 11 moves the temporary sample point (step S107). Specifically, first, the CPU 11 acquires a random number of an element number indicated by a counter variable among random numbers included in a random number sequence for selecting a horizontal axis movement random number sequence. Then, the CPU 11 acquires a horizontal axis movement random number sequence indicated by the acquired random number from the M horizontal axis movement random number sequences. Further, the CPU 11 obtains a random number having an element number indicated by the counter variable among the random numbers included in the random number sequence for selecting the vertical axis moving random number sequence. Then, the CPU 11 acquires the vertical axis moving random number sequence indicated by the acquired random number from the M vertical axis moving random number sequences.

  Here, the CPU 11 moves the selected N temporary sample points in the horizontal axis direction based on the N random numbers included in the acquired random number sequence for horizontal axis movement. Further, the CPU 11 moves the selected N temporary sample points in the vertical axis direction based on the N random numbers included in the acquired vertical axis moving random number sequence. When the motion equation of the object can be predicted, the CPU 11 further moves each of the selected N temporary sample points in the movement amount, the horizontal axis direction, and the vertical axis direction according to the motion equation.

  CPU11 will set the temporary sample point after a movement to a sample point, if the process of step S107 is completed (step S108). Specifically, the CPU 11 sets each of the N moved temporary sample points for the setting target frame image.

  CPU11 will give likelihood to a sample point, if the process of step S108 is completed (step S109). Specifically, the CPU 11 sets, for each of the N sample points set in the setting target frame image, the color at the sample point set in the setting target frame image and the set frame image (the latest set image). The higher the degree of similarity with the color at the sample point corresponding to the sample point set in the setting target frame image set in the frame image), the higher likelihood is given.

  When completing the process of step S109, the CPU 11 increments the counter variable (step S110). Specifically, the CPU 11 increments the value of the counter variable by 1 when the value of the counter variable stored in the RAM 13 is not M, and sets the value of the counter variable to 1 when the value of the counter variable is M. To do.

  When completing the process in step S110, the CPU 11 determines whether the sample point is appropriate (step S111). The method for determining whether or not the sample point is appropriate can be adjusted as appropriate. For example, the CPU 11 can determine whether or not the sample point is appropriate based on N likelihoods given to the N sample points set in the setting target frame image. For example, the CPU 11 determines that the average value of N likelihoods is lower than the threshold value, the highest likelihood among the N likelihoods is lower than the threshold value, or the threshold value among the N likelihoods. If the number of the highest likelihoods is less than the threshold, it can be determined that the sample point is not appropriate, and in other cases, it can be determined that the sample point is appropriate.

  When determining that the sample point is appropriate (step S111: YES), the CPU 11 estimates the position of the object (step S112). The method for estimating the position of the object can be adjusted as appropriate. For example, the CPU 11 obtains a weighted average of the coordinates of N sample points set in the setting target frame image, and uses the obtained coordinates as the center of gravity of the N sample points. When calculating the weighted average, the coordinates of the sample points are weighted based on the likelihood given to the sample points. Then, the CPU 11 can estimate the center of gravity of the obtained N sample points as the position of the object (typically, the center position of the object).

  Further, the CPU 11 can store the history of the position of the object in the RAM 13 or the like. In this case, the CPU 11 can estimate an equation of motion indicating the movement of the object based on the history of the position of the object stored in the RAM 13.

  On the other hand, when determining that the sample point is not appropriate (step S111: NO), the CPU 11 initializes the sample point and the likelihood (step S113). For example, the CPU 11 can initialize the coordinates of each of the N sample points and the likelihood of each of the N sample points, similarly to the initialization process in step S101. Then, the CPU 11 sets N sample points whose coordinates and likelihood are initialized in the setting target frame image. As a typical example of determining that the sample point is not appropriate, there may be a case where the object moves unexpectedly, a case where the object cannot be captured normally, and a normal moving image is not generated.

  When completing the process of step S112 or step S113, the CPU 11 returns the process to step S104 and continues resampling with the latest frame image as a new setting target frame image. Thereafter, the CPU 11 repeatedly executes the processing from step S104 to step S113.

  According to the present embodiment, various random number sequences are generated in the preprocessing of the object position estimation process (for example, the processes in steps S101 to S103), and the main process of the object position estimation process (for example, in steps S104 to S113). In the processing, various random number sequences are not generated. Therefore, an improvement in processing speed in the main processing of the object position estimation processing can be expected.

  Further, according to the present embodiment, a plurality of selection random number sequences, a horizontal axis movement random number sequence, a vertical axis movement random number sequence, and the like are prepared, and a random number sequence for selecting these random number sequences is prepared. Therefore, random numbers rich in variations are provided based on a relatively small number of random number sequences.

  The temporary sample points selected based on the selection random number sequence are selected from temporary sample points whose total number changes according to the likelihood given to the sample points. For this reason, even if the random number provided by the selection random number sequence is fixed to some extent, it is considered that the possibility that the selected temporary sample points are biased is low.

  Further, the temporary sample points that move based on the horizontal axis movement random number sequence or the vertical axis movement random number sequence are temporary sample points that are randomly selected from the set temporary sample points. For this reason, the random number provided by the horizontal axis movement random number sequence or the vertical axis movement random number sequence is fixed to some extent, and even if the movement amount is biased, the coordinates of the temporary sample points to be moved change. It is considered that there is a low possibility that the amount of movement depends on the position of the temporary sample point.

(Modification)
The present invention is not limited to the one disclosed in the above embodiment.

  In the present invention, which part of the configuration, function, and operation described in the above embodiment is adopted is arbitrary. Further, in the present invention, in addition to the configuration, function, and operation described above, further configuration, function, and operation may be employed.

  In the above-described embodiment, the example in which the weight given to the sample point is determined based on the similarity obtained by comparing the colors of the points has been described. In the present invention, the weight to be given to the sample points may be determined based on the similarity obtained by comparing the colors of the regions.

  In this case, for example, the weight assigning unit 107 assigns, to each of the sample points set in the setting target frame image by the sample point setting unit 106, a color in a specific region centered on the sample point set in the setting target frame image. The higher the degree of similarity with the color in the specific area centered on the sample point corresponding to the sample point set in the set target frame image set in the set frame image, the higher the weight. Thus, by comparing colors in units of regions, it can be expected to reduce the influence of noise and the like. Note that comparing colors in units of regions is, for example, comparing average values (simple average values or weighted average values) of luminance values of pixels included in each region for each primary color.

  The specific area may be set wider as the number of samples is smaller. Here, it can be expected that the position of the object is accurately estimated as the number of samples increases. On the other hand, the smaller the number of samples, the higher the object position estimation process can be expected. In addition, it is expected that the larger the specific area, the easier it is to estimate the position of the object even with a small number of samples. On the other hand, the narrower the specific area, the higher the object position estimation process can be expected. Therefore, it is expected that the execution speed of the object position estimation process can be increased within a range in which the position of the object can be appropriately estimated by setting the specific region wider as the number of samples is smaller. Note that the specific region can be, for example, an aggregate of points whose difference in coordinates (X coordinate and Y coordinate) with a sample point is a predetermined value or less.

  Note that when the frame image is in RGB format, HSV (Hue Saturation Lightness) format, or the like, pixel information is provided in units of one pixel. Therefore, when the frame image is in the RGB format, HSV format, or the like, the specific area can be freely set in units of one pixel. On the other hand, when the frame image is in YUV format or the like, pixel information is shared by two adjacent pixels. Therefore, when the frame image is in YUV format or the like, it is preferable to set the specific area in units of two pixels.

  In the above-described embodiment, an example has been described in which sample points are arranged in a grid pattern on the entire frame image when initialization is performed or when it is determined that the sample points are not appropriate. In the present invention, the method of arranging sample points at the time of initialization or the like is not limited to this example.

  For example, the uniform random number sequence for the horizontal axis and the uniform random number sequence for the vertical axis, which are generated in advance by the random number sequence generation unit 101 and stored in the random number sequence storage unit 102 and the like, have uniform random numbers for the number of samples. Based on the above, sample points may be randomly arranged in the entire frame image. In addition, a plurality of such uniform random number sequences for the horizontal axis (uniform random number sequences for the vertical axis) may be prepared. In this case, for example, to select one horizontal random number sequence (vertical vertical random number sequence) from among a plurality of horizontal uniform random number sequences (vertical vertical random number sequence). May be generated by the random number sequence generation unit 101 or the like and stored in the random number sequence storage unit 102. For example, when it is possible to predict where an object is present on the frame image, the sample points are arranged randomly or in a grid pattern in the region where the object is predicted to exist. May be.

  In the above embodiment, the selection random number sequence is used to select the selection random number sequence, and the horizontal axis movement random number sequence selection random number sequence is used to select the horizontal axis movement random number sequence. The example in which the random number sequence for selecting the vertical axis moving random number sequence is used for selecting the random number sequence for moving the vertical axis has been described. In the present invention, a selection random number sequence used for selection of a random number sequence may be shared. For selection of a random number sequence, a counter variable that is incremented or decremented may be used instead of the random number sequence for selection.

  In the above embodiment, a counter variable for selecting a random number from the random number sequence for selection random number sequence, a counter variable for selecting a random number from the random number sequence for horizontal axis movement, and a vertical axis movement The example in which the counter variable for selecting a random number from the random number sequence for selecting a random number sequence is shared has been described. In the present invention, this counter variable may be prepared separately for each selection random number sequence.

  In the above embodiment, an example has been described in which one temporary sample point is selected from one random number by a remainder calculation when the number of temporary sample points corresponding to the number of samples is selected from the set temporary sample points. In the present invention, when temporary sample points corresponding to the number of samples are selected from the set temporary sample points, the method of selecting one temporary sample point from one random number is not limited to this example. For example, the weight given to each sample point can be adjusted so that the total value of the weights given to each of the sample points for the number of samples becomes a constant value. According to such a configuration, by setting the upper limit value of the random number included in the selection random number sequence to the above-described constant value, provisional sample points corresponding to the number of samples are selected without using the remainder calculation.

  Further, it goes without saying that the number of elements included in various random number sequences and the number of various random number sequences are not limited to those shown in the above embodiment.

  In the above embodiment, an example in which various random number sequences are generated and stored when the power is turned on has been described. In the present invention, the timing at which various random number sequences are generated and stored is not limited to this example. For example, various random number sequences may be generated (regenerated) and stored (updated) when the processing load during the object position estimation process is light.

  In the above-described embodiment, an example in which the object position estimation device and the imaging device are different devices has been described. In the present invention, the object position estimation device may have the function of an imaging device.

  In the above-described embodiment, an example in which sample points are set for all frame images constituting a moving image has been described. In the present invention, for example, according to the processing capability of the object position estimation apparatus, sample points may be set as appropriate for frame images that are thinned out at a predetermined interval from among frame images constituting a moving image. .

  Note that the object position estimation apparatus according to the present invention can be realized by using a normal computer system without using a dedicated system. For example, a computer-readable recording medium such as a flexible disk, a CD-ROM (Compact Disc Read-Only Memory), a DVD (Digital Versatile Disc), or an MO (Magneto Optical Disc) is recorded on a computer. An object position estimation device that performs the above-described processing may be configured by storing and distributing the program in a medium and installing the program in a computer system.

  Furthermore, the program may be stored in a disk device or the like included in a server device on the Internet, and may be downloaded onto a computer by being superimposed on a carrier wave, for example.

  As mentioned above, although embodiment of this invention was described, this embodiment is only an illustration and does not limit the technical scope of this invention. The present invention can take various other embodiments, and various modifications such as omission and replacement can be made without departing from the gist of the present invention. These embodiments and modifications thereof are included in the scope of the invention described in the claims and equivalents thereof. The invention described in the scope of claims at the beginning of the present application will be appended.

(Appendix 1)
Random number sequence generating means for generating a first random number sequence having random numbers for the number of samples and a second random number sequence having random numbers for the number of samples;
Random number sequence storage means for storing the first random number sequence and the second random number sequence generated by the random number sequence generation means;
For each of the sample points for the number of samples set in the set frame image of the frame images constituting the moving image, the larger the number of sample points, the greater the weight assigned to the sample points. A temporary sample point setting means for setting a temporary sample point of the same coordinates as
Temporary sample point selection for selecting temporary sample points corresponding to the number of samples from the temporary sample points set by the temporary sample point setting unit based on the first random number sequence stored in the random number sequence storage unit Means,
Temporary sample point moving means for moving each of the temporary sample points selected by the temporary sample point selecting means based on the second random number sequence stored in the random number sequence storage means;
Sample point setting means for setting the sample points for the number of samples having the same coordinates as the temporary sample points after being moved by the temporary sample point moving means, in a setting target frame image among the frame images constituting the moving image; ,
The setting target set in the set frame image and the color at the sample point set in the setting target frame image for each of the sample points set in the setting target frame image by the sample point setting means Weight giving means for giving a higher weight as the similarity between the color at the sample point corresponding to the sample point set in the frame image is higher;
The moving image based on the coordinates of the sample points set in the setting target frame image and the weights given to the sample points set in the setting target frame image by the weight assigning unit. Object position estimating means for estimating the position of the object on the top,
An object position estimation apparatus characterized by the above.

(Appendix 2)
The first random number sequence has uniform random numbers corresponding to the number of samples,
The second random number sequence has normal random numbers corresponding to the number of samples.
The object position estimation apparatus according to Supplementary Note 1, wherein

(Appendix 3)
The random number sequence generation means includes at least two first random number sequences, at least two second random number sequences, a third random number sequence having at least two uniform random numbers, and at least two uniform numbers. Generating a fourth random number sequence having random numbers;
The random number sequence storage means stores at least two first random number sequences, at least two second random number sequences, a third random number sequence, and a fourth random number sequence generated by the random number sequence generation means,
The temporary sample point selection unit is configured to select one of at least two first random number sequences stored in the random number sequence storage unit based on the third random number sequence stored in the random number sequence storage unit. A first random number sequence is selected, and temporary sample points corresponding to the number of samples are selected from the temporary sample points set by the temporary sample point setting means based on the selected first random number sequence. ,
Based on the fourth random number sequence stored in the random number sequence storage unit, the temporary sample point moving unit is configured to select two out of at least two second random number sequences stored in the random number sequence storage unit. Selecting a second random number sequence, and moving each of the temporary sample points selected by the temporary sample point selection means based on the two selected second random number sequences,
The object position estimation apparatus according to appendix 1 or 2, characterized in that.

(Appendix 4)
The weighting unit includes a color in a specific region centered on the sample point set in the setting target frame image, and the setting for each of the sample points set in the setting target frame image by the sample point setting unit. A higher weight is assigned to a higher similarity with a color in a specific area centered on a sample point corresponding to the sample point set in the set target frame image, which is set in the completed frame image;
The object position estimation apparatus according to any one of appendices 1 to 3, wherein

(Appendix 5)
The specific area is set wider as the number of samples is smaller.
The object position estimation apparatus according to Supplementary Note 4, wherein

(Appendix 6)
A random number sequence generating step for generating a first random number sequence having random numbers for the number of samples and a second random number sequence having random numbers for the number of samples;
A random number sequence storage step for storing the first random number sequence and the second random number sequence generated by the random number sequence generation step;
For each of the sample points for the number of samples set in the set frame image of the frame images constituting the moving image, the larger the number of sample points, the greater the weight assigned to the sample points. A temporary sample point setting step for setting a temporary sample point of the same coordinates as
Temporary sample point selection step for selecting temporary sample points for the number of samples from the temporary sample points set by the temporary sample point setting step based on the first random number sequence stored by the random number sequence storage step When,
A temporary sample point moving step of moving each of the temporary sample points selected by the temporary sample point selecting step based on the second random number sequence stored by the random number sequence storing step;
A sample point setting step for setting the sample points for the number of samples having the same coordinates as the temporary sample points after being moved by the temporary sample point moving step in a setting target frame image among the frame images constituting the moving image; ,
The setting target set in the set frame image and the color at the sample point set in the setting target frame image for each of the sample points set in the setting target frame image by the sample point setting step A weighting step of giving a greater weight to the color at the sample point corresponding to the sample point set in the frame image, as the similarity is higher;
The moving image based on the coordinates of the sample points set in the setting target frame image and the weights assigned to the sample points set in the setting target frame image by the weighting step. An object position estimating step for estimating the position of the object above,
An object position estimation method characterized by the above.

(Appendix 7)
Computer
Random number sequence generation means for generating a first random number sequence having random numbers for the number of samples and a second random number sequence having random numbers for the number of samples;
Random number sequence storage means for storing the first random number sequence and the second random number sequence generated by the random number sequence generation means;
For each of the sample points for the number of samples set in the set frame image of the frame images constituting the moving image, the larger the number of sample points, the greater the weight assigned to the sample points. A temporary sample point setting means for setting a temporary sample point of the same coordinates as
Temporary sample point selection for selecting temporary sample points corresponding to the number of samples from the temporary sample points set by the temporary sample point setting unit based on the first random number sequence stored in the random number sequence storage unit means,
Temporary sample point moving means for moving each of the temporary sample points selected by the temporary sample point selecting means based on the second random number sequence stored in the random number sequence storage means;
Sample point setting means for setting the sample points for the number of samples having the same coordinates as the temporary sample points after being moved by the temporary sample point moving means, in setting frame images among the frame images constituting the moving image,
The setting target set in the set frame image and the color at the sample point set in the setting target frame image for each of the sample points set in the setting target frame image by the sample point setting means Weight giving means for giving a higher weight as the degree of similarity between the color at the sample point corresponding to the sample point set in the frame image is higher;
The moving image based on the coordinates of the sample points set in the setting target frame image and the weights given to the sample points set in the setting target frame image by the weight assigning unit. Function as object position estimation means for estimating the position of an object on
A program characterized by that.

11 ... CPU, 12 ... ROM, 13 ... RAM, 14 ... flash memory, 15 ... RTC, 16 ... touch screen, 17 ... interface, 100 ... object position Estimation apparatus 101... Random number sequence generation unit 102... Random number sequence storage unit 103... Temporary sample point setting unit 104... Temporary sample point selection unit 105. 106 ... Sample point setting unit 107 ... Weighting unit 108 ... Object position estimation unit 109 ... Moving image acquisition unit 200 ... Imaging device 600 ... Object

Claims (15)

And at least two first random number sequence having a number of samples of the random number, and at least two second random number sequence has a random number of the number of samples, and the third random number sequence having at least two uniform random numbers, A random number sequence generating means for generating
At least the two of the first random number sequence is generated, at least the two of the second random number sequence is generated, and the generated third random number sequence, a random number sequence storing means for storing in advance,
A temporary sample point setting means for setting a temporary sample point having the same coordinates as the sample point as the weight assigned to the sample point increases,
Based on the third random number sequence stored in advance, the first random number sequence is selected from at least two first random number sequences stored in advance, and the selected one random number Temporary sample point selection means for selecting temporary sample points for the number of samples from the set temporary sample points based on the first random number sequence;
Temporary sample point moving means for moving a selected temporary sample point based on the second random number sequence stored in advance;
Sample point setting means for setting a sample point having the same coordinates as the temporary sample point after movement;
Weight giving means for newly giving a larger weight as the degree of similarity between the color at the sample point after movement and the color at the sample point before movement is higher;
An object position estimating means for estimating the position of the object based on the coordinates of the set sample points and the newly assigned weight;
Comprising
An object position estimation apparatus characterized by the above. At least two first random number sequences having random numbers for the number of samples, at least two second random number sequences having random numbers for the number of samples, and a fourth random number sequence having at least two uniform random numbers; A random number sequence generating means for generating
Random number sequence storage means for storing in advance the generated at least two first random number sequences, the generated at least two second random number sequences, and the generated fourth random number sequence;
A temporary sample point setting means for setting a temporary sample point having the same coordinates as the sample point as the weight assigned to the sample point increases,
Temporary sample point selection means for selecting temporary sample points for the number of samples from the set temporary sample points based on the first random number sequence stored in advance;
Based on the fourth random number sequence stored in advance, two second random number sequences are selected from at least two second random number sequences stored in advance, and the two selected two random number sequences are selected. Temporary sample point moving means for moving the selected temporary sample point based on the second random number sequence;
Sample point setting means for setting a sample point having the same coordinates as the temporary sample point after movement;
Weight giving means for newly giving a larger weight as the degree of similarity between the color at the sample point after movement and the color at the sample point before movement is higher;
An object position estimating means for estimating the position of the object based on the coordinates of the set sample points and the newly assigned weight;
Comprising
An object position estimation apparatus characterized by the above. The provisional sample point setting means sets a larger weight for a sample point for each of the sample points for the number of samples set in a set frame image of frame images constituting a moving image. Set as many temporary sample points with the same coordinates as the sample points.
Object position estimation apparatus according to claim 1 or 2, characterized in that. The temporary sample point selection means is based on the first random number sequence stored in advance in the random number sequence storage means, out of the temporary sample points set by the temporary sample point setting means, for the number of samples. Select a temporary sample point,
The object position estimating apparatus according to claim 3 . The temporary sample point moving means moves each of the temporary sample points selected by the temporary sample point selecting means based on the second random number sequence stored in advance in the random number sequence storage means.
The object position estimating apparatus according to claim 4 . The sample point setting means sets the sample points for the number of samples having the same coordinates as the temporary sample points after being moved by the temporary sample point moving means to the setting target frame images of the frame images constituting the moving image. Set,
The object position estimating apparatus according to claim 5 . The weight assigning unit is set in each of the sample points set in the setting target frame image by the sample point setting unit and the color at the sample point set in the setting target frame image and the set frame image. A higher weight is newly given as the degree of similarity is higher with the color at the sample point corresponding to the sample point set in the setting target frame image.
The object position estimating apparatus according to claim 6 . The object position estimating means includes the coordinates of the sample points set in the setting target frame image and the newly assigned weights to the sample points set in the setting target frame image by the weight assigning means. And estimating the position of the object on the moving image based on
The object position estimating apparatus according to claim 7 . The first random number sequence has uniform random numbers corresponding to the number of samples,
The second random number sequence has normal random numbers corresponding to the number of samples.
Object position estimation apparatus according to any one of claims 1 to 8, characterized in that. The weighting unit includes a color in a specific region centered on the sample point set in the setting target frame image, and the setting for each of the sample points set in the setting target frame image by the sample point setting unit. A higher weight is newly assigned to the color in a specific area centered on a sample point corresponding to the sample point set in the set target frame image, which is set in the completed frame image,
Object position estimation apparatus according to any one of claims 6 to 8, characterized in that. The specific area is set wider as the number of samples is smaller.
The object position estimation apparatus according to claim 10 . And at least two first random number sequence having a number of samples of the random number, and at least two second random number sequence has a random number of the number of samples, and the third random number sequence having at least two uniform random numbers, A random number sequence generation step for generating
At least the two of the first random number sequence is generated, at least the two of the second random number sequence is generated, and the random number sequence storing step of storing the generated third random number sequence, previously,
A temporary sample point setting step for setting a larger number of temporary sample points having the same coordinates as the sample points as the weight given to the sample points is larger;
Based on the third random number sequence stored in advance, the first random number sequence is selected from at least two first random number sequences stored in advance, and the selected one random number A temporary sample point selection step of selecting temporary sample points corresponding to the number of samples from the set temporary sample points based on the first random number sequence;
A temporary sample point moving step of moving the selected temporary sample point based on the second random number sequence stored in advance;
A sample point setting step for setting a sample point having the same coordinates as the temporary sample point after movement;
A weighting step for newly giving a larger weight as the degree of similarity between the color at the sample point after movement and the color at the sample point before movement is higher;
An object position estimation step for estimating the position of the object based on the coordinates of the set sample points and the newly assigned weights;
including,
An object position estimation method characterized by the above. At least two first random number sequences having random numbers for the number of samples, at least two second random number sequences having random numbers for the number of samples, and a fourth random number sequence having at least two uniform random numbers; A random number sequence generation step for generating
A random number sequence storage step for storing in advance the generated at least two first random number sequences, the generated at least two second random number sequences, and the generated fourth random number sequence;
A temporary sample point setting step for setting a larger number of temporary sample points having the same coordinates as the sample points as the weight given to the sample points is larger;
A temporary sample point selecting step for selecting temporary sample points for the number of samples from the set temporary sample points based on the first random number sequence stored in advance;
Based on the fourth random number sequence stored in advance, two second random number sequences are selected from at least two second random number sequences stored in advance, and the two selected two random number sequences are selected. A temporary sample point moving step of moving the selected temporary sample point based on the second random number sequence;
A sample point setting step for setting a sample point having the same coordinates as the temporary sample point after movement;
A weighting step for newly giving a larger weight as the degree of similarity between the color at the sample point after movement and the color at the sample point before movement is higher;
An object position estimation step for estimating the position of the object based on the coordinates of the set sample points and the newly assigned weights;
including,
An object position estimation method characterized by the above. Computer
And at least two first random number sequence having a number of samples of the random number, and at least two second random number sequence has a random number of the number of samples, and the third random number sequence having at least two uniform random numbers, Random number sequence generation means for generating
At least two of the first random number sequence is generated, at least the two of the second random number sequence is generated, the generated third random number sequence and the random number sequence storing means for storing in advance,
Temporary sample point setting means for setting a larger number of temporary sample points having the same coordinates as the sample points as the weight given to the sample points is larger,
Based on the third random number sequence stored in advance, the first random number sequence is selected from at least two first random number sequences stored in advance, and the selected one random number Temporary sample point selection means for selecting temporary sample points corresponding to the number of samples from the set temporary sample points based on the first random number sequence;
Temporary sample point moving means for moving the selected temporary sample point based on the second random number sequence stored in advance;
Sample point setting means for setting a sample point having the same coordinates as the temporary sample point after movement,
Weight giving means for newly giving a larger weight as the degree of similarity between the color at the sample point after movement and the color at the sample point before movement is higher;
Object position estimating means for estimating the position of the object based on the coordinates of the set sample points and the newly assigned weights;
Function as
A program characterized by that. Computer
At least two first random number sequences having random numbers for the number of samples, at least two second random number sequences having random numbers for the number of samples, and a fourth random number sequence having at least two uniform random numbers; Random number sequence generation means for generating
Random number sequence storage means for storing in advance the generated at least two first random number sequences, the generated at least two second random number sequences, and the generated fourth random number sequence;
Temporary sample point setting means for setting a larger number of temporary sample points having the same coordinates as the sample points as the weight given to the sample points is larger,
Temporary sample point selection means for selecting temporary sample points for the number of samples from the set temporary sample points based on the first random number sequence stored in advance;
Based on the fourth random number sequence stored in advance, two second random number sequences are selected from at least two second random number sequences stored in advance, and the two selected two random number sequences are selected. Temporary sample point moving means for moving the selected temporary sample point based on the second random number sequence;
Sample point setting means for setting a sample point having the same coordinates as the temporary sample point after movement,
Weight giving means for newly giving a larger weight as the degree of similarity between the color at the sample point after movement and the color at the sample point before movement is higher;
Object position estimating means for estimating the position of the object based on the coordinates of the set sample point and the newly assigned weight;
Function as
A program characterized by that.

Images