JP6098320B2 - Image processing apparatus, image processing method, and image processing program - Google Patents

Description

  The present invention relates to a technique related to image processing for processing a moving image captured by a video camera or the like and detecting or tracking a captured object.

  2. Description of the Related Art Conventionally, moving images captured by a video camera or the like are processed to track an imaged object (a moving object such as a person or a vehicle) (see, for example, Patent Document 1). In tracking an object using a moving image, for each frame image constituting the moving image, an object captured in the frame image is detected, and an object captured between temporally consecutive frame images (detected object) It is done by identifying.

  Further, if the tracking of the object is performed by sequential processing that repeats the detection of the object and the identification of the detected object for each frame image constituting the moving image, the processing time becomes long. Therefore, in order to shorten the processing time, the moving image is divided into processing blocks with a predetermined number of frames, and the divided processing blocks are divided into multi blocks by a CPU having a plurality of cores or a PC cluster in which a plurality of PCs are combined. It is considered to perform parallel processing using a thread method.

JP 2002-133421 A

  However, the above-described multi-thread method requires merge processing for identifying the object being tracked (integrating the processing results) between temporally continuous processing blocks. This merging process takes a longer processing time as the number of objects to be identified increases.

  Therefore, there is a demand for a technique for efficiently performing a merge process and shortening a processing time in a process of tracking an object captured in a moving image by a multi-thread method.

  An object of the present invention is to provide a technique capable of efficiently performing a merge process and sufficiently shortening a processing time in a process related to tracking an object captured in a moving image by a multi-thread method. .

  In order to achieve the above object, the image processing apparatus of the present invention is configured as follows.

  The frame image extraction unit extracts a frame image related to the input moving image at a predetermined frame extraction interval. This moving image is, for example, a moving image captured by a video camera. The frame extraction interval in the frame image extraction unit is, for example, about several frames to several tens of frames. However, it goes without saying that the frame extraction interval may be set outside the above range. The frame extraction interval may be determined according to the number of objects captured in a frame image sampled from a moving image. In this case, the frame extraction interval may be dynamically changed according to the number of objects being imaged.

  The division processing unit divides the input moving image into a plurality of processing blocks. The object tracking unit processes a plurality of processing blocks divided by the division processing unit with multi-threads, and tracks an object captured in the moving image. The merge processing unit integrates the tracking results of the objects processed by the object tracking unit using multithreads for a plurality of processing blocks divided by the division processing unit.

  The image processing apparatus further includes an object number detection unit that detects the number of objects captured in the frame image for a plurality of frame images extracted by the frame image extraction unit. Then, the division processing unit divides the input moving image into a plurality of processing blocks based on the number of objects detected by the object number detection unit for the plurality of frame images extracted by the frame image extraction unit.

  For example, the division processing unit divides a portion in the input moving image where the number of captured objects is equal to or greater than a predetermined value into processing blocks having a larger number of frames than other portions. This reduces the number of merge processes in which the number of objects to be identified is relatively large in the merge process in which the process results are integrated between temporally continuous process blocks. Accordingly, it is possible to reduce the processing time required for the merge process in the process of tracking the object captured in the moving image by the multi-thread method.

  The object number detection unit estimates a variation in the number of captured objects in the input moving image based on the number of captured objects detected for the plurality of frame images extracted by the frame image extraction unit. It is good also as a structure. In this way, it is possible to suppress the frame image in which the number of objects captured by the object number detection unit is detected, and the processing for tracking the object captured in the moving image can be performed more efficiently. In this case, the division processing unit may divide the moving image into a plurality of processing blocks based on the change in the number of captured objects in the input moving image estimated by the object number detection unit.

  The division processing unit may have a configuration in which a frame image common to temporally continuous processing blocks is present when the moving image input to the moving image input unit is divided into processing blocks.

  If comprised in this way, an object can be identified using the same frame image in the merge process between the temporally continuous process blocks. Therefore, it is possible to simplify the process for identifying the object and to suppress a decrease in the identification accuracy of the object, so that the object can be properly tracked.

  Further, an image processing apparatus according to the present invention processes, in a multi-thread manner, a division processing unit that divides an input moving image into a plurality of processing blocks, and a plurality of processing blocks divided by the division processing unit. An object tracking unit that tracks an object captured in an image, and a merge processing unit that integrates tracking results of objects processed by the object tracking unit in a multi-thread manner. May be configured such that a frame image common to temporally continuous processing blocks exists. Even in this configuration, as described above, the object can be identified using the same frame image in the merge processing between temporally continuous processing blocks. Therefore, since the process related to object identification can be simplified, the processing time required for the merge process can be shortened. Further, as described above, since it is possible to suppress a decrease in the object identification accuracy, the object can be properly tracked.

  The image processing method according to the present invention is an invention in which the function of each unit included in the above-described image processing apparatus is executed by a computer.

  An image processing program according to the present invention is an invention that causes a computer to execute the functions of the units included in the above-described image processing apparatus.

  According to the present invention, merge processing can be performed efficiently and processing time can be sufficiently shortened in processing related to tracking of an object captured in a moving image by a multi-thread method.

It is a block diagram which shows the structure of the principal part of an image processing apparatus. It is a block diagram which shows the function structure of an image processor. It is a flowchart which shows operation | movement of an image processor. It is a figure explaining a frame image extraction process. It is a figure explaining processing load amount judgment processing. It is a figure explaining processing load amount fluctuation presumption processing. It is a figure explaining a division process. It is a figure which shows an object map. It is a block diagram which shows the function structure of the image processor concerning another example. It is a flowchart which shows operation | movement of the image processor concerning another example.

  Hereinafter, an image processing apparatus according to an embodiment of the present invention will be described.

  FIG. 1 is a block diagram showing the configuration of the main part of the image processing apparatus according to this example. The image processing apparatus 1 includes an image input unit 2, a moving image storage unit 3, an image processing processor 4, and an output unit 5. The image processing apparatus 1 can be realized by a personal computer (PC) in which the image processing program according to the present invention is installed. In this example, the computer that executes this image processing program is the image processor 4. Similarly, the computer that executes the image processing method according to the present invention is also the image processor 4 in this example.

  In this example, a video camera 10 is connected to the image input unit 2. The imaging area of the video camera 10 is a monitoring area that tracks object detection and movement of the detected object. A moving image of an imaging area captured by the video camera 10 is input to the image input unit 2. The moving image that the video camera 10 inputs to the image input unit 2 is 10 to 30 frames per second. The tracking target object includes not only a moving object such as a person or a vehicle but also a left object (non-moving object). However, installation objects such as benches, fences, and trash cans located in the imaging area (in the monitoring area) are the background and not the object to be tracked.

  The device connected to the image input unit 2 is not limited to the video camera 10 described above, and may be a recording medium or the like on which a moving image is recorded.

  The moving image storage unit 3 is a recording medium such as a hard disk that records the moving image input to the image input unit 2. The moving image input to the image input unit 2 is compressed by the known MPEG2 or the like and recorded in the moving image storage unit 3.

  The image processor 4 processes a moving image stored in the moving image storage unit 3, that is, a moving image input to the image input unit 2, and performs processing related to detection and tracking of an imaged object (hereinafter referred to as “tracking”). This is called tracking processing.) The image processor 4 has a configuration corresponding to a frame image extraction unit, an object number detection unit, a division processing unit, an object tracking unit, and a merge processing unit, which are referred to in the present invention.

  The output unit 5 outputs a processing result related to object detection and tracking in the image processor 4 to the host device.

  The image processing apparatus 1 includes a main control unit (not shown) that controls the image input unit 2, the moving image storage unit 3, the image processing processor 4, the output unit 5, and the like.

  FIG. 2 is a block diagram illustrating a functional configuration of the image processing processor. The image processor 4 includes a frame image extraction unit 41, an object number detection unit 42, a division processing unit 43, an object tracking unit 44, and a merge processing unit 45.

  The frame image extraction unit 41 extracts a frame image applied to the processing target moving image input to the image input unit 2 and stored in the moving image storage unit 3 at a predetermined frame extraction interval. This frame extraction interval is several frames to several tens of frames. The time interval of the frame images extracted here is about several hundred ms to 1 s. However, it goes without saying that the frame extraction interval may be set outside the above range. Further, the frame extraction interval may be a fixed value or a variable value. The frame extraction interval may be determined according to the number of objects captured in a frame image sampled from a moving image. In this case, the frame extraction interval may be dynamically changed according to the number of objects being imaged.

  The object number detection unit 42 detects the number of objects captured in the frame image of the frame image extracted by the frame image extraction unit 41. In addition, the object number detection unit 42 estimates the variation (variation with time) of the number of objects captured in the moving image that is the processing target, based on the number of objects detected for the plurality of extracted frame images. . This estimation accuracy can be improved by reducing the above-described frame extraction interval, but the processing time becomes longer. Therefore, the above-described frame extraction interval is determined in consideration of this estimation accuracy and processing time. Here, a case where the frame extraction interval is 4 frames will be described as an example.

  The division processing unit 43 divides the processing target moving image into processing blocks including a plurality of frame images based on the variation in the processing load on the processing target moving image estimated by the object number detection unit 42. . The number of frames in the processing block is not constant. The processing block is composed of temporally continuous frame images. Furthermore, the same frame image is included in temporally continuous processing blocks.

  The object tracking unit 44 includes a plurality of cores, and performs parallel processing on the processing blocks divided by the division processing unit 43 using a multithread method. In FIG. 2, four cores (core A, core B, core C, and core D) are illustrated, but the number of cores provided in the object tracking unit 44 may be any number as long as it is two or more. The object tracking unit 44 performs tracking processing of an object in which each core (core A to core D) is imaged in units of divided processing blocks.

  The merge processing unit 45 performs a merge process for integrating the object tracking results performed in units of processing blocks obtained by dividing each core (core A to core D) of the object tracking unit 44.

  Hereinafter, the operation of the image processing apparatus 1 according to this example will be described in detail. The image processing apparatus 1 stores a moving image to be processed in the moving image storage unit 3.

  FIG. 3 is a flowchart showing the operation of the image processor. The image processor 4 performs a frame image extraction process in which the frame image extraction unit 41 extracts a frame image related to the processing target moving image stored in the moving image storage unit 3 at a predetermined frame extraction interval (s1). . FIG. 4 shows an example in which a frame image is extracted at a predetermined frame extraction interval from a moving image to be processed configured with m frames (F1 to Fm). In the example shown in FIG. 4, the frame extraction interval is 4 frames.

  When the frame extraction interval is a fixed value, if the preset frame extraction interval is x frames, the (a × x) th frame image in the moving image to be processed is extracted. However, a is a natural number. In this example, the number of frames of the frame image extracted in s1 (hereinafter referred to as an extracted frame) is m / 4 (rounded down to the decimal point).

  In this example, the frame extraction interval is a fixed value (4 frames), but may be a variable value.

  The image processor 4 performs object number detection processing for detecting the number of objects captured in the frame image for each extracted frame extracted in s1 in the object number detection unit 42 (s2). s2 is processing for detecting an object imaged by known pattern matching, background difference, or the like and obtaining the number of the imaged object. Since the process according to s2 only needs to be able to obtain the number of objects being imaged, it is not necessary to detect the attributes (features such as shapes) of the object being imaged.

  In FIG. 5, a frame image (frame Fn) in which the object being imaged is 2, a frame image (frame Fn + 4) in which the object being imaged is 5, and a frame image (3 in which the object being imaged is 3) Frame Fn + 8) is illustrated.

  The image processor 4 uses the object number detection unit 42 to estimate the variation in the number of objects imaged in the moving image to be processed based on the number of objects imaged in each extraction frame detected in s2. Number fluctuation estimation processing is performed (s3). In s3, as shown in FIG. 6, it is the process which estimates the fluctuation | variation of the number of the objects currently imaged by the frame image over the whole moving image of a process target. That is, it is a process of estimating the number of objects being imaged for frame images other than the extracted frame based on the detection result of the number of objects being imaged with respect to the extracted frame.

  The estimation accuracy of the object number variation estimation process relating to s3 can be improved by reducing the above-described frame extraction interval, but the processing time (s1 to s3 (particularly, the time required for the process of s2)) is increased. Therefore, the above-described frame extraction interval may be determined in consideration of the processing time for s1 to s3 and the estimation accuracy of the object number variation estimation processing for s3.

  The image processor 4 performs a dividing process of dividing the moving image into processing blocks based on the variation in the processing load amount in the moving image to be processed estimated in s3 in the dividing processing unit 43 (s4). . In the division process according to s4, first, a division point is determined for the moving image to be processed.

  An example of a method for determining this division point is shown below.

  (1) The division processing unit 43 provisionally determines a division point for each predetermined A frame from the first frame of the moving image to be processed. For example, a division point is provisionally determined every two frames. The division processing unit 43 determines, for each provisionally determined division point, the number of objects (the number detected in s2 or the estimated value in s3) captured in the immediately subsequent frame image (for example, five). If the number is less than (), the division point is finally determined. On the contrary, if the number is a predetermined number (for example, five) or more, the provisional determination of the division point is canceled. Thereby, a division point is not set immediately before a frame image in which the number of captured objects is relatively large (in this example, five or more).

  (2) In the example of (1) above, the difference in the number of frames of frame images located between adjacent division points may increase. As will be described later, since the moving image is divided into processing blocks based on the division points determined here, the difference in processing load between the divided processing blocks becomes large. For this reason, it is preferable to add a restriction that the tentatively determined division point cannot be canceled. For example, for a temporarily determined dividing point, when the immediately preceding dividing point has been canceled, it is preferable to provide a restriction for determining this dividing point. Thereby, it is possible to prevent a difference in the number of frames of frame images located between adjacent division points from increasing.

  Note that the restriction of the division points where the provisional decision cannot be canceled is an example in which the number of frame images located between adjacent division points is reduced to a maximum of two times, but the restriction is limited to three times or four times. It may be.

  (3) Further, based on the processing result of s3, a division point is determined immediately before the frame image in which the number of objects being imaged is less than a predetermined number. In this case, a minimum value (for example, 2 frames (however, the minimum value is 2 frames or more)) or a maximum value (for example, 4 frames) of a frame image located between adjacent division points is set. It is preferable to determine the dividing point based on this restriction.

  This determination of the dividing point is more than when the total number of dividing points in which the number of objects captured in the frame image immediately after the dividing point is equal to or greater than the predetermined number is determined for each predetermined A frame. As long as it is a technique that reduces the amount, a method other than the methods shown in the above (1) to (3) may be used.

  FIG. 7 exemplifies the dividing points determined by the method (2). In this figure, the division points provisionally determined between Fn + 3 and Fn + 4 are cancelled.

  When the division processing unit 43 determines the division point shown in FIG. 7 for the moving image to be processed, for each division point, from the frame immediately after that (division start frame) to the frame immediately after the next division point ( The division is performed up to the next division start frame) as one processing block. Therefore, in this example, the processing block is composed of 3 frames or 5 frames. Also, as shown in FIG. 7, in two temporally continuous processing blocks, one final frame and the other top frame are the same frame image.

  Although FIG. 7 shows an example in which a frame included in common in two temporally continuous processing blocks is a frame immediately after the division point (division start frame), it may be a frame immediately before the division point. In this case, for each division point, the division from the immediately preceding frame to the frame immediately before the next dividing point may be performed as one processing block.

  The image processor 4 performs an object tracking process in the object tracking unit 44 (s5). The object tracking unit 44 processes the processing blocks divided by the division processing unit 43 in parallel using a multi-thread method. Specifically, four cores (core A, core B, core C, and core D) perform tracking processing of the object being imaged in units of divided processing blocks.

  In this tracking process, each core detects an object captured for each frame image included in the processing block, and acquires a position on the frame image for each detected object. Further, the detected object is associated between the frame images to create an object map. In the object map created here, a temporary ID is assigned to each detected object. For example, the core A gives temporary IDs such as A1, A2,. The core B assigns temporary IDs such as B1, B2,... To the detected object. The core C gives temporary IDs such as C1, C2,... To the detected object. The core D assigns temporary IDs such as D1, D2,... To the detected object.

  FIG. 8A is a diagram showing an object map created by the core A in the tracking process for the processing blocks from Fn to Fn + 2 shown in FIG. FIG. 8B is a diagram showing an object map created by the core B in the tracking process for the processing blocks from Fn + 2 to Fn + 6 shown in FIG. As shown in FIG. 8, the object map is obtained by registering a frame image and a position (coordinates) on the frame image in association with each provisional ID assigned to the detected object. In the object map shown in FIG. 8, an object whose position is not shown is an object that has not been captured in the frame image. That is, in FIG. 8, the object whose temporary ID is A4 is not captured in the frame image having the frame number Fn. In addition, the object whose temporary ID is A5 is not captured in the frame image of the frame number Fn and the frame number Fn + 1. Further, the object whose temporary ID is B3 is not captured in the frame images of the frame numbers Fn + 5 and Fn + 6. In addition, the object whose temporary ID is B5 is not captured in the frame image having the frame number Fn + 6.

  As described above, the object tracking unit 44 performs the tracking processing of the object imaged in the processing block divided by multi-threads by a plurality of cores, and creates the object map shown in FIG.

  In the merge processor 45, the image processor 4 performs a merge process for integrating the object maps created by the cores (s6). In s6, it is the process which identifies the object to which temporary ID was provided between the object maps concerning the process block which continues in time. Specifically, a common frame image is included in the object map relating to the processing blocks that are temporally continuous. In the example shown in FIG. 8, the frame number Fn + 4 is a common frame image. In this frame image, objects having the same position of the detected object are identified as the same object. For example, in the example shown in FIG. 8, the object whose temporary ID is A1 and the object whose temporary ID is B5 are identified as the same object. The object whose temporary ID is A2 and the object whose temporary ID is B3 are identified as the same object. The object whose temporary ID is A3 and the object whose temporary ID is B2 are identified as the same object. The object whose temporary ID is A4 and the object whose temporary ID is B4 are identified as the same object. The object whose temporary ID is A5 and the object whose temporary ID is B1 are identified as the same object.

  As described above, when the moving image to be processed is divided into processing blocks, the same frame image is included in temporally continuous processing blocks, so that the accuracy of the merge processing related to s6 can be sufficiently ensured. In addition, since this merging process can be performed by a relatively simple process (a process for matching detection positions), the processing time required for the merging process can be further reduced.

  Further, the image processor 4 can construct an object map in which each captured object is distinguished over the entire moving image to be processed by assigning the real ID to the object identified by the merge process. That is, it is possible to obtain an object map indicating the tracking result of the imaged object over the entire moving image to be processed.

  Thus, the image processing apparatus 1 according to this example can suppress the number of merge processes in which the number of objects to be identified is relatively large. Therefore, merge processing can be performed efficiently and processing time can be sufficiently shortened in processing related to tracking of an object captured in a moving image by the multi-thread method.

  Further, since the same frame image is included in temporally continuous processing blocks, it is possible to suppress a decrease in object identification accuracy in the merge processing. In addition, since the merge process can be performed by a relatively simple process, the processing time required for the merge process can be further reduced.

  In addition, the process concerning s1-s4 and s6 in the example mentioned above may be executed by any of the cores A to D shown in FIG. 2, or may be executed by another core not shown. Good.

  Further, as illustrated in FIG. 9, the image processing processor 4 may be configured not to include the frame image extraction unit 41 and the object number detection unit 42 described above.

  In this case, the image processor 4 may perform the process shown in FIG. In the division processing unit 43, the image processor 4 determines a division point for each predetermined A frame from the first frame of the moving image to be processed (s11). The image processor 4 performs an object tracking process in the object tracking unit 44 (s12). In the merge processor 45, the image processor 4 performs a merge process for integrating the object maps created by the cores (s13).

  s12 is the same process as s5 described above, and s13 is the same process as s6 described above.

  In this example, since the division points are determined equally for the moving image to be processed, the number of merge processes in which the number of objects to be identified is relatively large cannot be suppressed as in the above example. However, when the processing target moving image is divided into processing blocks, since the same frame image is included in temporally continuous processing blocks, as described above, the accuracy of the merge processing related to s13 is sufficiently ensured. it can. In addition, since the merge process can be performed by a relatively simple process, the processing time required for the merge process can be further reduced.

  In the above example, the image processor 4 having a plurality of cores is configured to process the processing blocks divided by the division processing unit 43 in parallel in a multi-thread method, but in a PC cluster in which a plurality of PCs are combined, A configuration may be adopted in which the processing blocks divided by the division processing unit 43 are processed in parallel by a multithread method.

DESCRIPTION OF SYMBOLS 1 ... Image processing apparatus 2 ... Image input part 3 ... Moving image memory | storage part 4 ... Image processor 5 ... Output part 41 ... Frame image extraction part 42 ... Object number detection part 43 ... Division processing part 44 ... Object tracking part 45 ... Merge Processing part

Claims (8)

A frame image extraction unit for extracting a frame image related to the input moving image at a predetermined frame extraction interval;
An object number detection unit for detecting the number of captured images for a plurality of frame images extracted by the frame image extraction unit;
A division processing unit that divides an input moving image into a plurality of processing blocks based on the number of objects detected by the object number detection unit for a plurality of frame images extracted by the frame image extraction unit;
An object tracking unit that processes a plurality of processing blocks divided by the division processing unit in a multi-thread manner and tracks an object captured in the moving image;
A plurality of processing blocks divided by the division processing unit, a merge processing unit that integrates tracking results of objects processed by the object tracking unit in a multi-thread, and
The division processing unit divides a portion in the input moving image where the number of captured objects is equal to or greater than a predetermined value into processing blocks having a larger number of frames than other portions. apparatus. A frame image extraction unit for extracting a frame image related to the input moving image at a predetermined frame extraction interval;
An object number detection unit for detecting the number of captured images for a plurality of frame images extracted by the frame image extraction unit;
A division processing unit that divides an input moving image into a plurality of processing blocks based on the number of objects detected by the object number detection unit for a plurality of frame images extracted by the frame image extraction unit;
An object tracking unit that processes a plurality of processing blocks divided by the division processing unit in a multi-thread manner and tracks an object captured in the moving image;
A plurality of processing blocks divided by the division processing unit, a merge processing unit that integrates tracking results of objects processed by the object tracking unit in a multi-thread, and
The object number detection unit estimates a variation in the number of captured objects in an input moving image based on the number of captured objects detected for a plurality of frame images extracted by the frame image extraction unit. ,
The division processing unit is an image processing device that divides the moving image into a plurality of processing blocks based on a change in the number of objects captured in the input moving image estimated by the object number detection unit . The object number detection unit estimates a variation in the number of captured objects in an input moving image based on the number of captured objects detected for a plurality of frame images extracted by the frame image extraction unit. ,
The division processing section has been estimated the object speed detector, based on a variation in the number of objects being imaged in the input moving image, and divides the moving image into a plurality of processing blocks, according to claim 1 Image processing apparatus.   The division processing unit causes a frame image common to temporally continuous processing blocks to exist when the moving image input to the moving image input unit is divided into processing blocks. The image processing apparatus described. Computer
A frame image extraction step for extracting a frame image related to the input moving image at a predetermined frame extraction interval;
An object number detecting step for detecting the number of objects being imaged for a plurality of frame images extracted in the frame image extracting step;
A division processing step for dividing the input moving image into a plurality of processing blocks based on the number of objects detected in the object number detection step for the plurality of frame images extracted in the frame image extraction step;
An object tracking step of processing a plurality of processing blocks divided in the division processing step in a multi-thread manner and tracking an object captured in the moving image;
A merge processing step of integrating a tracking result of an object processed in multi-thread in the object tracking step with respect to a plurality of processing blocks divided in the division processing step ;
The division processing step is a step of dividing a portion where the number of captured objects is a predetermined value or more in the input moving image into processing blocks having a larger number of frames than other portions. , Image processing method. Computer
A frame image extraction step for extracting a frame image related to the input moving image at a predetermined frame extraction interval;
An object number detecting step for detecting the number of objects being imaged for a plurality of frame images extracted in the frame image extracting step;
A division processing step for dividing the input moving image into a plurality of processing blocks based on the number of objects detected in the object number detection step for the plurality of frame images extracted in the frame image extraction step;
An object tracking step of processing a plurality of processing blocks divided in the division processing step in a multi-thread manner and tracking an object captured in the moving image;
A merge processing step of integrating a tracking result of an object processed in multi-thread in the object tracking step with respect to a plurality of processing blocks divided in the division processing step;
The object number detection step estimates a variation in the number of captured objects in the input moving image based on the number of captured objects detected for the plurality of frame images extracted in the frame image extraction step. Step,
The division processing step is a step of dividing the moving image into a plurality of processing blocks based on a change in the number of objects captured in the input moving image estimated in the object number detection step. Method. On the computer,
A frame image extraction step for extracting a frame image related to the input moving image at a predetermined frame extraction interval;
An object number detecting step for detecting the number of objects being imaged for a plurality of frame images extracted in the frame image extracting step;
A division processing step for dividing the input moving image into a plurality of processing blocks based on the number of objects detected in the object number detection step for the plurality of frame images extracted in the frame image extraction step;
An object tracking step of processing a plurality of processing blocks divided in the division processing step in a multi-thread manner and tracking an object captured in the moving image;
A merge processing step of integrating a tracking result of an object processed in multi-thread in the object tracking step with respect to a plurality of processing blocks divided in the division processing step ,
The division processing step is a step of dividing a portion where the number of captured objects is a predetermined value or more in the input moving image into processing blocks having a larger number of frames than other portions. , Image processing program. On the computer,
A frame image extraction step for extracting a frame image related to the input moving image at a predetermined frame extraction interval;
An object number detecting step for detecting the number of objects being imaged for a plurality of frame images extracted in the frame image extracting step;
A division processing step for dividing the input moving image into a plurality of processing blocks based on the number of objects detected in the object number detection step for the plurality of frame images extracted in the frame image extraction step;
An object tracking step of processing a plurality of processing blocks divided in the division processing step in a multi-thread manner and tracking an object captured in the moving image;
A merge processing step of integrating a tracking result of an object processed in multi-thread in the object tracking step with respect to a plurality of processing blocks divided in the division processing step,
The object number detection step estimates a variation in the number of captured objects in the input moving image based on the number of captured objects detected for the plurality of frame images extracted in the frame image extraction step. Step,
The division processing step is a step of dividing the moving image into a plurality of processing blocks based on a change in the number of objects captured in the input moving image estimated in the object number detection step. program.

Images