JP4843787B2 - Subject tracking method and apparatus - Google Patents

Description

  The present invention relates to a subject tracking method and apparatus for tracking a moving subject with an imaging device.

FIG. 13 is a schematic diagram used for explaining a known subject tracking method using the template matching method. In the conventional subject tracking method, first, a subject existing area where a subject is present is extracted from shooting data in which the subject is photographed first, and the reference template 101 is created by acquiring the subject existing area data as an extraction template. (Reference template acquisition step). Then, the position of the subject at this time is stored. Next, the new shooting data 102 is scanned by the scanning line 103 with the reference template 101, and template matching is performed between the reference template 101 and the new shooting data 102. When the subject 104 on the new shooting data 102 and the reference template 101 match, the matching position is set as the subject and the position, and the moving distance, the moving speed, etc. are obtained from the difference between the position and the previous position. Then, the movement information of the moving subject is obtained and tracking is performed by the photographing apparatus. However, with this method, when the subject moves to the far side or the near side, the subject on the new shooting data and the reference template may not match, and the moving position of the subject may not be detected. Therefore, as shown in Japanese Patent Laid-Open No. 2001-243476, the reference template is enlarged or reduced at a plurality of magnifications to create a comparison template including a plurality of enlargement / reduction templates (comparison template creation step). A technique for template matching using multiple scale templates was proposed. According to this technology, even if the subject moves to the far side or near side and the image of the subject on the shooting data is enlarged or reduced, the moving position of the subject is detected by template matching using a plurality of enlargement / reduction templates. can do.
JP 2001-243476 A

  The moving subject may move in various directions and change its direction with respect to the photographing apparatus. Also, the brightness of the background of the subject may change depending on the shooting time zone and the weather. However, in the conventional subject tracking method, when the posture (orientation) of the subject with respect to the photographing apparatus changes, that is, when the traveling direction changes, or when the brightness of the background of the subject changes greatly, the comparison template includes There is no template that matches the image of the subject in the shooting data, and there is a problem that the moving position of the subject cannot be detected.

  An object of the present invention is to provide a subject tracking method capable of detecting a moving position of a subject even if the subject rotates or changes a traveling direction.

  Another object of the present invention is to provide a subject tracking method that can detect the movement position of a subject even if the brightness of the background of the subject changes greatly.

  The subject tracking method to be improved by the present invention is to capture a moving subject by detecting a moving position of the subject by a reference template creating step for creating a reference template, a comparison template creating step, and a matching step. Track by device.

  In the comparison template creation step, a plurality of comparison templates are created based on the reference template. In the matching step, template matching is performed using a plurality of comparison templates and new image data, and a position where the subject on the new image data substantially matches any one of the plurality of comparison templates is obtained. In the present invention, in the comparison template creation step, a plurality of enlargement / reduction templates and a plurality of rotation templates are created. The plurality of enlargement / reduction templates are created by enlarging or reducing the reference template at a plurality of magnifications. Here, “enlargement or reduction” includes not only enlargement / reduction in which the vertical and horizontal directions change at the same magnification, but also enlargement / reduction in which the vertical and horizontal directions change at different magnifications. The plurality of rotation templates are created by rotating the reference template and / or the plurality of enlargement / reduction templates by a predetermined rotation angle. Here, the smaller the “predetermined rotation angle”, the greater the number of templates to be created. Therefore, the rotation angle may be changed according to the operating speed of the subject. In other words, the rotation angle may be increased to some extent when the operation speed of the subject is fast, and the rotation angle may be decreased to some extent when the operation speed of the subject is low. In the present invention, a plurality of enlargement / reduction templates and a plurality of rotation templates created in this way are used as a plurality of comparison templates.

  According to the present invention, when the subject moves farther or closer to the front without changing the traveling direction and is enlarged or reduced on the shooting data, the probability that matching is successful when a plurality of enlargement / reduction templates are used. Is expensive. In addition, when the subject rotates without changing the traveling direction, the probability of successful matching is high when a plurality of rotation templates are used. Therefore, even when the subject rotates without changing the traveling direction, the moving position of the subject can be detected with a higher probability than in the past.

  Also, in the comparison template creation step, a plurality of enlargement / reduction templates and a plurality of distortion templates can be created. The plurality of distortion templates are created by changing the ratio of the vertical and horizontal axes and / or the inclination angle of the reference template, the plurality of enlargement / reduction templates, and / or the plurality of rotation templates. Here, the inclination angle is an angle between the orthogonal axis when the horizontal axis and / or the vertical axis of the image data is inclined with respect to the orthogonal axis. The plurality of enlargement / reduction templates and the plurality of distortion templates created in this way are used as a plurality of comparison templates.

  In this way, if the subject moves far away or toward the front without changing the traveling direction and is enlarged or reduced on the shooting data, the probability that matching will be successful when multiple enlargement / reduction templates are used. Is expensive. Further, when the subject changes its traveling direction, there is a high probability that matching is successful when a plurality of distortion templates are used. Therefore, even when the subject changes its traveling direction or rotates while changing its traveling direction, the moving position of the subject can be detected with a higher probability than in the past.

  Further, in the comparison template creation step, a plurality of enlargement / reduction templates, a plurality of rotation templates, and a plurality of distortion templates are created, and a plurality of enlargement / reduction templates, a plurality of rotation templates, and a plurality of distortion templates are used as a plurality of comparison templates. It can also be.

  In this way, if the subject moves far away or toward the front without changing the traveling direction and is enlarged or reduced on the shooting data, the probability that matching will be successful when multiple enlargement / reduction templates are used. Is expensive. In addition, when the subject rotates without changing the traveling direction, the probability of successful matching is high when a plurality of rotation templates are used. When the subject changes its traveling direction, the probability of successful matching is high when a plurality of distortion templates are used. Therefore, even when the subject rotates without changing the traveling direction, changes the traveling direction, or rotates while changing the traveling direction, the moving position of the subject can be detected with a higher probability than before.

  In many cases, the reference template is image data represented using an RGB color system, a Munsell color system, an XYZ color system, or the like. With such image data, in the comparison template creation step, a plurality of enlargement / reduction templates, a plurality of rotation templates, and / or a plurality of distortion templates are changed by changing arbitrary pixel values in the pixel data of each template. The pixel value change template may be further created and added to the comparison template. In this way, if the background brightness of the subject changes significantly and the color of each part of the subject changes, or if the subject is shot with light in the rain, or if the sunbeams hit the subject Even in the case where the brightness of the subject changes partly, the success rate of matching is increased by performing template matching using a plurality of pixel value change templates.

  The reference template may be composed of an extraction template obtained by extracting a subject existing area where a subject exists from photographing data obtained by a photographing device having a function of tracking a moving subject. In this way, a reference template that closely resembles the subject can be easily created.

  Further, the reference template may be composed of an extraction template obtained by extracting a subject existing area where a subject exists from shooting data collected in advance for the subject. In this way, it is possible to track a subject by finding a subject using previously collected shooting data without creating a reference template first. In addition, when the subject changes its posture extremely in a short time, if there is an image of the subject in a posture that is close to the pre-collected shooting data, the probability of successful matching increases.

  Also, a part of the reference template, a plurality of enlargement / reduction templates, a plurality of rotation templates, a plurality of distortion templates, and / or a pixel value change template may be removed using a template conversion filter prepared in advance, or a partial conversion may be performed by partially modifying Additional templates can be created and included in the comparison template. “Template conversion filter” means, for example, dividing an image into a plurality of vertical and horizontal directions, and deleting, enlarging, reducing, vertical / horizontal axis conversion or the like only in an arbitrary area, or in pixel data in an arbitrary area. This is a filter that can change an arbitrary pixel value. In addition, the “deformation” referred to here includes one in which a part is enlarged / reduced, distorted, or changed into an arbitrary shape. In this way, when a part of the subject is not visible due to an obstacle, or when the whole subject is not photographed in the same state, such as when part of the subject is in the water, etc. However, the probability of successful matching increases.

  Note that the enlargement or reduction magnification described above is preferably determined based on the magnification of the photographing apparatus when the reference template is obtained. In this way, it is not necessary to extremely increase the enlargement magnification range and the reduction magnification range, and the number of enlargement / reduction templates to be created can be reduced.

  In the comparison template creation step, it is preferable to create a plurality of enlargement / reduction templates, a plurality of rotation templates, a plurality of distortion templates, and a pixel value change template using a dedicated LSI. In this way, a template can be created at a higher speed than when a comparison template is created using a CPU. Therefore, there is no problem even if the number of templates to be created increases.

  If the present invention is specified as a subject tracking device, the subject tracking device includes an imaging device having a driving device, a template creation unit, a template storage unit, and a matching processing unit. The template creation unit creates a plurality of comparison templates based on the reference template. The template storage unit stores a plurality of extraction templates and a plurality of comparison templates. Further, the matching processing unit performs template matching between the plurality of comparison templates and the new shooting data, and obtains a position where the subject on the new shooting data substantially matches any one of the plurality of comparison templates. In particular, in the subject tracking device of the present invention, the template creation unit not only creates a plurality of enlargement / reduction templates by enlarging or reducing the reference template at a plurality of magnifications, but also creates a reference template and / or a plurality of enlargement / reduction templates. Create multiple rotation templates by rotating by a predetermined rotation angle, and create multiple distortion templates by changing the ratio of vertical and horizontal dimensions and / or tilt angle of the reference template, multiple scaling templates and / or multiple rotation templates The plurality of enlargement / reduction templates, the plurality of rotation templates, and the plurality of distortion templates are configured as a plurality of comparison templates.

  The template creation unit removes a part of a reference template, a plurality of enlargement / reduction templates, a plurality of rotation templates, a plurality of distortion templates, and / or a pixel value change template using a template conversion filter prepared in advance, or enlarges a part. A partial conversion template that has been reduced or distorted and converted may be further created and included in the comparison template.

  According to the present invention, it is possible to create a comparison template that is more appropriate than the conventional one, not only when the subject moves far away or toward the front, but also when the subject rotates or changes the traveling direction. The moving position of the subject can be detected with a higher probability than in the past, and the tracking accuracy is increased.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing a configuration of an example of an embodiment of a subject tracking device of the present invention that implements a subject tracking method of the present invention. FIG. 2 is a flowchart showing an example of an algorithm of a program used when the subject tracking method according to the embodiment of the present invention is implemented by the apparatus of FIG.

  As shown in FIG. 1, first, a photographing apparatus 1 for photographing a moving subject is prepared. The photographing apparatus 1 is a digital video camera provided with a zoom lens, and the zoom lens is controlled by a zoom control unit 2 that changes the magnification by a control signal. Here, if the photographing apparatus 1 is capable of digital zooming by incorporating a high-resolution CCD or CMOS sensor, it is not always necessary to include a zoom lens. In this case, the zoom control unit 2 selects an area for performing digital zoom. In this example, the subject can be tracked by the rotation drive mechanism that rotates the photographing device 1 within a predetermined angle range and the drive device 3 that includes a vertical movement mechanism that moves the lens up and down within the predetermined angle range. I have to. The driving device 3 operates based on position information obtained from a matching processing unit 5 described later, and the zoom control unit 2 is configured to appropriately change the magnification by an autofocus function.

  Shooting data (digital image data) of the shooting apparatus 1 is sent to the image input unit 4. The image input unit 4 converts the photographed data into data in a format necessary for subsequent processing. The photographing data is sent to the matching processing unit 5 and the subject area extracting unit 6. First, in order to determine a subject (tracking target), the subject region extraction unit 6 extracts a subject existing region from the photographing data obtained by the photographing device 1 and obtains an extraction template. In this example, this extraction template is used as a reference template (reference template creation step). Various methods can be adopted as a method for extracting the first subject existing area. In the simplest method, first, the photographing apparatus 1 is manually operated to catch a subject, and photographing data is captured at a position where the subject appears in the center of the screen with a certain size. Then, a subject region extraction unit 6 extracts a subject region located at the center of the photographed data from the captured data. The subject region extraction unit 6 extracts only subject region data from the captured data, and defines the extracted data as an extraction template (reference template).

  Next, based on the reference template, the template generation unit 7 creates a plurality of comparison templates (comparison template creation step). In the template generation unit 7, as described below, the reference template formed by the reference template extracted by the subject region extraction unit 6 and the new image data of the subject extracted from the new shooting data by the matching processing in the matching processing unit 5. Based on the above, a plurality of enlargement / reduction templates, a plurality of rotation templates, a plurality of distortion templates, and a plurality of pixel value change templates are created and stored in the template storage unit 8 together with the reference template as a comparison template. The comparison template is comparison image data used for comparison with a predetermined area in the photographic data in order to detect a subject existing area from the new photographic data. In the following description, both the first reference template obtained from the subject region extraction unit 6 and the new reference template obtained from the matching processing unit 5 will be described as the reference template T1.

Equation 1 below is a basic determinant used when the reference template is enlarged, reduced, rotated, compressed, or the like.

  In the above basic formula, A is a transformation matrix, and x is a pixel coordinate in the reference template (original image data). X ′ represents the coordinates of the pixel after conversion of magnification (image conversion), that is, after enlargement / reduction. Here, the pixel means all the pixels included in the image data, and is typically represented by a and b. K and n of k, l, m, and n in the transformation matrix A are horizontal and vertical magnification factors, and l and m are rotations used to rotate the coordinates by 90 ° and change the vertical and horizontal magnifications. It is a magnification factor. In order to obtain a simple enlargement / reduction template with a constant aspect ratio, the values of k and n in the transformation matrix A may be increased or decreased at the same rate, and the values of l and m may be set to zero.

On the other hand, as shown in FIG. 3, when creating a plurality of enlargement / reduction templates T2 obtained by enlarging or reducing the reference template T1 by changing the vertical and horizontal magnifications, a transformation matrix of the following Equation 2 is used.

  In this equation, l and m in the transformation matrix A of the basic equation are set to 0, and the values of k and n are determined so as to obtain desired vertical and horizontal magnifications. The above expression is an expression when the reference template T1 is multiplied by k times in the horizontal direction and n times in the vertical direction.

FIG. 4 shows an example of creating a rotation template T3 obtained by enlarging or reducing the vertical and horizontal dimensions of the reference template T1 and further rotating it by 90 °. In this example, the reference template T1 is m times in the horizontal direction and l times in the vertical direction, and further rotated by 90 °, and k and n in the transformation matrix A of the basic formula are set to 0, and l and m are arbitrary. The following formula 3 is used as the value of.

  The enlargement or reduction magnification is determined based on the zoom magnification of the photographing apparatus 1 when the reference template is obtained. The zoom lens magnification data can be obtained from the zoom control unit 2. You may create an enlarged / reduced template by enlarging or reducing the reference template at multiple pre-defined magnifications, but the previous photographic data except for those with very fast moving speeds, such as rockets and supersonic jets There is no significant difference between the size of the subject in the middle and the size of the subject in the next shooting data. However, if the above-mentioned magnification is fixed in advance, there may be a case where an enlarged or reduced template having a size that cannot be considered from the situation of the previous photographing data is created. Therefore, if the magnification for enlargement or reduction is determined based on the magnification of the zoom lens when the previous new photographing data is photographed, it is not necessary to create a useless comparison template, and the matching processing in the matching processing unit 5 is performed. However, since it is not necessary to unnecessarily increase the number of templates to be compared, the matching processing speed can be increased. In the present embodiment, the template generation unit 7 generates a template using a dedicated LSI instead of a CPU. Since the dedicated LSI specializes in functions, the calculation speed can be increased compared to the CPU. Therefore, a lot of comparison data can be created in a short time.

As in the example of FIG. 4, when the above Equation 3 is used, it is possible to obtain a rotated template obtained by rotating an enlarged / reduced template rotated by 90 ° and changing the aspect ratio, but as shown in FIG. In order to rotate the angle θ, the following formula 4 is used.

  In the example of FIG. 5, the reference template T1 is rotated counterclockwise by an angle of θ ° about the center point of the subject. Therefore, when performing rotation, k, l, m, and n in the basic formula may be set to k = cos θ, l = −sin θ, m = sin θ, and n = cos θ.

  If the above equation is used, a simply rotated template T4 can be obtained by rotating the enlarged / reduced template at an arbitrary angle. In addition, when the enlarged / reduced template with the aspect ratio changed as shown in FIG. 3 is rotated by 90 ° as shown in FIG. 4, after creating the enlarged / compressed template using Equation 2 instead of Equation 3, You may make it obtain a distortion template by rotating 90 degree | times using the said Numerical formula 4. FIG. FIG. 6 shows an example in which the reference template T1 is enlarged or reduced, and further rotated by a desired angle θ to create a plurality of distortion templates. In this example, the distortion template T5 is created by converting the reference template using the basic formula.

  FIG. 7 shows a basic configuration of a dedicated LSI for performing basic equation calculations. In FIG. 7, a circle indicates a multiplier and a circle indicates a + mechanism, and an adder. When such a configuration is used, the coordinates (a, b) of the pixels of the original image data can be converted into (ka + lb, ma + nb). Needless to say, an actual dedicated LSI is provided with function realizing means for realizing further necessary functions. Further, in order to arbitrarily change the rotation angle, a function for executing the above mathematical formula 4 may be added.

FIG. 8 shows that when a reference template obtained from shooting data is expressed in the RGB color system, a plurality of RGB change plates (pixel value changing templates) having different pixel values are changed by changing the pixel values of the shooting data. Thus, an example in the case of changing the luminance / saturation is schematically shown. This change can be realized by using Equation 5 below.

  In the above equation, A is a transformation matrix, y is the RGB value of the pixel of the reference template, and y ′ is the RGB value of the pixel after conversion. The coefficients α to η in the conversion matrix are conversion coefficients for converting pixel values. By changing the coefficient of α to η and changing the pixel value, it becomes possible to brighten a dark image or make a bright image a dark image, that is, change luminance. In addition, it is possible to change the color, that is, change the saturation by changing the coefficients of α to η. As a result, a pixel value changing template corresponding to a change in weather can be produced. Note that the reference template is not limited to the one represented in the RGB color system, and a template represented in another color system such as Munsell color system or XYZ color system may be used. Of course. When the arithmetic unit that executes the above formula is realized by a dedicated LSI, an LSI having a basic configuration as shown in FIG. 9 may be prepared.

  FIGS. 10A to 10D show an example in which a partial conversion template T6 is created by removing a portion (a portion in the vertical direction of the code C1) from a reference template T1 ′ using a template conversion filter prepared in advance. . In this example, as shown in FIG. 10 (B), after the screen is divided, as shown in FIG. 10 (C), selection of the domain area (in this example, the portion in the vertical direction of C1) And a partial conversion template T6 was created by removing the domain region C1.

  FIGS. 11A to 11D show an example in which a partial conversion template T7 in which a part (a portion in the vertical direction of the code C2) is deformed from the reference template T1 'using a template conversion filter is shown. In this example, a partial conversion template T7 is created by deforming the shape of the domain region C2.

  FIGS. 12A to 12D show an example in which a partial conversion template T8 in which a part (frame portion of reference C3) is deformed from the reference template T1 'using a template conversion filter is shown. In this example, the conversion area T2 is created by enlarging the domain region C2.

  If various partial conversion templates T6 are created using such a template conversion filter, when a part of the subject becomes invisible due to an obstacle, or when a part of the subject is in the water, etc. As described above, even when the entire subject is not photographed in the same state, the success probability of matching increases.

  Next, returning to FIG. 2, an example of the embodiment of the subject tracking method of the present invention using the apparatus of FIG. 1 will be described. First, as shown in step ST1 of FIG. 2, the first reference template is acquired. In step ST2, a plurality of comparison templates are created based on the reference template. The created plurality of comparison templates are sent to the template storage unit 8 and stored as data. Next, new image data is sent to the matching processing unit 5, and comparison templates sequentially selected from a plurality of comparison templates stored in the template storage unit 8 are sent to the matching processing unit 5. As described above, the matching processing unit 5 performs template matching between new image data and a comparison template. Then, it is determined whether or not the subject on the new image data, the plurality of data, and any one of the comparison templates substantially match. If any of the comparison templates matches the subject data on the shooting data, the data of the matching position is obtained (matching step). In the present invention, not only the enlargement / reduction template, but also the rotation template, distortion template, pixel value change template, and partial conversion template are created by the template generation unit 7. Therefore, when the subject moves farther or closer to the front side without changing the traveling direction and is enlarged / reduced on the photographing data, the matching is successful by the plurality of enlargement / reduction templates T2. In addition, when the subject rotates without changing the traveling direction, matching is successful by using a plurality of rotation templates T3 and T4. When the subject changes its traveling direction, matching is successful by using a plurality of distortion templates T5. Even when the brightness of the background of the subject changes greatly and the color of each part of the subject changes, matching is successful by using the plurality of pixel value change templates T6. For example, if the brightness of the background of the subject changes greatly, the color of each part of the subject changes, the subject is illuminated in the rain, or the brightness of the subject is partially due to sunbeams, etc. Even if it changes greatly, there are cases where matching is successful. Even when a part of the subject is hidden by an obstacle, matching may be successful by using a plurality of partial conversion templates T6. As described above, according to the present embodiment, various types of comparison templates that are close to the expected changed shape of the reference template are prepared, so that the matching success probability is higher than when only the enlargement / reduction template is prepared. Become.

  Next, when the moving position of the subject can be detected by matching, position information is supplied to at least the drive device 3 to track the subject, and the drive device 3 detects the position of the photographing device 1 based on the input position information. And the angle of the zoom lens is changed. Further, image data on new image data detected by matching is acquired as a new reference template (step ST5), and a new comparison template is created based on the new reference template (step ST6). ). The new extraction template and the newly created comparison template are stored in the template storage unit 8 together with the previously created comparison template. In order to prevent the stored data from becoming excessive, the stored data in the template storage unit 8 may be updated each time a comparison template is created. However, if the memory capacity is large, comparison templates for a predetermined number of times of creation may be stored, and the old comparison template and the new comparison template may be updated sequentially. The above steps are repeated as long as new image data is continuously input.

  According to the subject tracking method of this example, the rotation templates T3 and T4 and the distortion template T5 are used not only when the subject moves far away or toward the front, but also when the subject rotates or changes the traveling direction. In addition, by using the comparison template including the partial conversion template T6, it is possible to detect the moving position of the subject with a higher probability than in the past.

  In the above example, the extraction template obtained by extracting the subject existence area from the photographing data obtained by the photographing apparatus is used as the reference template. However, the extraction is performed by extracting the subject existence area where the subject exists from the photographing data collected in advance for the subject. A template may be used as a reference template.

It is a conceptual diagram used in order to explain the concept of the subject tracking method of the present invention. 3 is a flowchart for explaining a subject tracking method shown in FIG. 1. FIG. 2 is a diagram illustrating an example of creating a plurality of enlargement / reduction templates by enlarging or reducing a reference template in the subject tracking method shown in FIG. 1. It is a figure which shows the example which produces a some rotation template in the subject tracking method shown in FIG. FIG. 2 is a diagram showing an example of creating a plurality of distortion templates in the subject tracking method shown in FIG. 1. FIG. 10 is a diagram showing another example of creating a plurality of distortion templates in the subject tracking method shown in FIG. 1. It is a figure which shows the aspect which performs the conversion matrix of FIG. 6 using a dedicated LSI. It is a figure which shows an example which produces the some RGB change plate in the to-be-photographed object tracking method shown in FIG. It is a figure which shows the aspect which performs the conversion matrix of FIG. 8 using a dedicated LSI. (A)-(D) is a figure used in order to explain an example of a process which creates a partial conversion template in a subject tracking method shown in FIG. (A)-(D) is a figure used in order to explain other examples of the process of creating a partial conversion template in the subject tracking method shown in FIG. (A)-(D) are the figures used in order to demonstrate the further another example of the process which produces a partial conversion template in the photographic subject tracking method shown in FIG. It is the schematic used in order to demonstrate the well-known subject tracking method using the template matching method.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image pick-up device 2 Zoom control part 3 Drive apparatus 4 Image input part T1 Reference template T2 Enlargement / reduction template T3, T4 Rotation template T5 Distortion template T6, T7, T8 Partial conversion template

Claims (11)

A reference template creation step for creating a reference template;
A comparison template creation step of creating a plurality of comparison templates based on the reference template;
A template matching is performed between the plurality of comparison templates and new shooting data, and a matching step for obtaining a position where the subject on the new shooting data substantially matches any one of the plurality of comparison templates, An object tracking method for detecting a moving position of the object and tracking the moving object by an imaging device,
In the comparison template creation step, the reference template is enlarged or reduced at a plurality of magnifications to create a plurality of enlargement / reduction templates, and the reference template and / or the plurality of enlargement / reduction templates are rotated by a predetermined rotation angle. A plurality of rotation templates, and the plurality of enlargement / reduction templates and the plurality of rotation templates as the plurality of comparison templates ,
The reference template is image data represented using a predetermined color system such as RGB color system, Munsell color system, and XYZ color system,
In the comparison template creation step, a plurality of pixel value change templates obtained by changing arbitrary pixel values in the plurality of enlargement / reduction templates and / or the plurality of rotation templates are further created as the comparison templates. Subject tracking method. A reference template creation step for creating a reference template;
A comparison template creation step of creating a plurality of comparison templates based on the reference template;
A template matching is performed between the plurality of comparison templates and new shooting data, and a matching step for obtaining a position where the subject on the new shooting data substantially matches any one of the plurality of comparison templates, An object tracking method for detecting a moving position of the object and tracking the moving object by an imaging device,
In the comparison template creation step, the reference template is enlarged or reduced at a plurality of magnifications to create a plurality of enlargement / reduction templates, and tilt angles of the reference template and / or the plurality of enlargement / reduction templates are changed. A plurality of distortion templates, the plurality of scaling templates and the plurality of distortion templates as the plurality of comparison templates ,
The reference template is image data represented using a predetermined color system such as RGB color system, Munsell color system, and XYZ color system,
In the comparison template creation step, a plurality of pixel value change templates obtained by changing arbitrary pixel values in the plurality of enlargement / reduction templates and / or the plurality of distortion templates are further created as the comparison templates. Subject tracking method. A reference template creation step for obtaining a reference template;
A comparison template creation step of creating a plurality of comparison templates based on the reference template;
A template matching is performed between the plurality of comparison templates and new shooting data, and a matching step for obtaining a position where the subject on the new shooting data substantially matches any one of the plurality of comparison templates, An object tracking method for detecting a moving position of the object and tracking the moving object by an imaging device,
In the comparison template creation step, the reference template is enlarged or reduced at a plurality of magnifications to create a plurality of enlargement / reduction templates, and the reference template and / or the plurality of enlargement / reduction templates are rotated by a predetermined rotation angle. A plurality of rotation templates, and a plurality of distortion templates are generated by changing inclination angles of vertical and horizontal axes of the reference template, the plurality of enlargement / reduction templates and / or the plurality of rotation templates. , The plurality of rotation templates and the plurality of distortion templates as the plurality of comparison templates ,
The reference template is image data represented using a predetermined color system such as RGB color system, Munsell color system, and XYZ color system,
In the comparison template creation step, a plurality of pixel value change templates obtained by changing arbitrary pixel values in the plurality of enlargement / reduction templates, the plurality of rotation templates, and / or the plurality of distortion templates are further used as the comparison templates. A subject tracking method characterized by creating .   The said reference template consists of an extraction template which extracted the to-be-existed area | region where the said subject exists from the imaging | photography data obtained by the imaging device which has the function to track the to-be-moved subject. Subject tracking method.   The subject tracking method according to claim 1, wherein the reference template is an extraction template obtained by extracting a subject existing area where the subject exists from photographing data collected in advance with respect to the subject.   A part of the reference template, the plurality of enlargement / reduction templates, the plurality of rotation templates, the plurality of distortion templates and / or the pixel value changing template may be removed using a template conversion filter prepared in advance, or the part may be removed. The subject tracking method according to any one of claims 1 to 3, wherein a deformed partial conversion template is further created and included in the comparison template.   The subject tracking method according to claim 1, wherein the magnification is determined based on a magnification of the photographing apparatus when the reference template is obtained.   4. The comparison template creation step of creating the plurality of enlargement / reduction templates, the plurality of rotation templates, and / or the plurality of distortion templates using a dedicated LSI. Subject tracking method described in one. The subject tracking method according to claim 1, wherein in the comparison template creation step, the pixel value change template is created using a dedicated LSI. An imaging device having a driving device for tracking a moving subject;
A template creation unit for creating a plurality of comparison templates based on a reference template;
A template storage unit for storing the plurality of comparison templates;
A matching processing unit that performs template matching between the plurality of comparison templates and new shooting data, and obtains a position where the subject on the new shooting data substantially matches any one of the plurality of comparison templates; With
A subject tracking device for tracking the subject by the photographing device by detecting a moving position of the subject by the matching processing unit and outputting a detection signal to the photographing device;
The reference template is image data represented using a predetermined color system such as RGB color system, Munsell color system, and XYZ color system,
The template creation unit creates a plurality of enlargement / reduction templates by enlarging or reducing the reference template at a plurality of magnifications, and rotates the reference template and / or the plurality of enlargement / reduction templates by a predetermined rotation angle. expansion of creating a rotating template, said reference template, by changing the ratio and / or the inclination angle of the vertical and horizontal axes of said plurality of scaling a template and / or the plurality of rotary templates to create a plurality of strain templates, said plurality A plurality of pixel value change templates in which arbitrary pixel values in the reduction template, the plurality of rotation templates and / or the plurality of distortion templates are changed are created, the plurality of enlargement / reduction templates, the plurality of rotation templates , and the plurality of templates distortion template and a plurality of pixel values change of Object tracking apparatus characterized by being configured to the template and the plurality of comparison template.   The template creation unit removes a part of the reference template, the plurality of enlargement / reduction templates, the plurality of rotation templates, the plurality of distortion templates, and / or the pixel value change template using a template conversion filter prepared in advance. The subject tracking device according to claim 10, further comprising: creating a partial conversion template in which the part is enlarged / reduced or the part is distorted and converted and included in the comparison template.

Images