JP4934809B2 - Image processing apparatus and image processing method - Google Patents

Description

  The present invention relates to an image processing apparatus and an image processing method.

  In recent years, for example, a real-time high-speed vision system as described in Non-Patent Document 1 has been known. In such an imaging apparatus, imaging at a high speed such as 1000 FPS and imaging at a high resolution such as 1024 × 1024 pixels are possible.

In order to perform processing such as image recognition on high-speed and high-resolution image data acquired from such an imaging apparatus, the same number (label) is applied to a group of pixels selected according to any criterion and connected to each other. So-called labeling is generally performed.
TM Bernard, BY Zavidovique, FJ Devos, “A programmable artificial retina”, IEEE J. Solid-State Circuits, Vol.28, No.7, pp.789-797, 1993.

  In the conventional labeling method, two scans are performed on one frame of image data acquired from the imaging device. That is, after a temporary label is attached in the first scan, it is attached again in the second scan to obtain image data with a true label. As a result, image data with a correct label can be obtained, but a long processing time is required to perform two scans. Furthermore, the image data with a temporary label must be temporarily stored until the second scanning, which requires a large memory area. These problems become more serious as the size of the image data is larger.

  In the conventional labeling technique, the number of usable label numbers may be limited to a certain number depending on the amount of usable memory, for example. In such a case, for example, if labels are added even for noise, there is a possibility that the labels to be attached to necessary data may be insufficient. This becomes a more serious problem, for example, the more so-called sesame salt noise and the smaller the amount of memory provided in the mounted hardware.

  Therefore, the present invention has been made in view of the above, and enables high-speed processing when labeling captured image data, and further reduces the amount of memory required for the processing. An object is to provide a possible image processing apparatus and an image processing method.

  In order to solve the above problems, an image processing apparatus according to the present invention is an image processing apparatus that inputs an image frame captured by an imaging apparatus and labels each element constituting the image frame, and has a predetermined order. A label holding unit that holds a plurality of labels, an input unit that inputs an image frame captured by the imaging device, and each element in the image frame that is input by the input unit are specified in order, and are subjected to labeling. An attention element selection means for selecting an attention element; and a labeling means for attaching a label held by the label holding means to the attention element selected by the attention element selection means. If there are other attention elements that were selected before selecting the current attention element and are adjacent to the current attention element on the image frame, The previous label, which is a label attached to the element, is attached to the current attention element, and when there are a plurality of previous labels, the current attention element is given the label that is the earliest in the order among the plurality of previous labels, The element that is the first in the order among the plurality of previous labels is reattached to the element of interest.

  The image processing method of the present invention is an image processing method for inputting an image frame captured by an imaging device and labeling each element constituting the image frame, wherein the input means captures the image frame captured by the imaging device. An input step for inputting, a target element selection means for selecting a target element to be labeled by sequentially specifying each element in the image frame input by the input means, and a label A labeling step of attaching a label held by a label holding unit that holds a plurality of labels having a predetermined order to a target element selected by the target element selecting unit, and in the labeling step, The labeling means is selected before the attention element selection means selects the current attention element, and the current attention element and image are selected. If there are other elements of interest that are adjacent on the frame, attach the previous label, which is the label attached to the other elements of interest, to the current element of interest. The label that is the first in the order among the plurality of previous labels is attached, and the label that is the first in the order among the plurality of previous labels is reattached to the other elements of interest.

  According to such an image processing apparatus and image processing method of the present invention, an element composed of one or more pixels in an input image frame is a current attention element (hereinafter also referred to as “current attention element”). When selected, the labeling means attaches the label held by the label holding means to the selected current attention element. At this time, if the labeling means is selected before the present time and there is another attention element adjacent to the current attention element on the image frame (hereinafter also referred to as “previous point adjacent attention element”), the labeling means The same label number as the label (destination label) attached to the previous attention element adjacent to the previous time is attached to the current attention element. It is assumed that a plurality of attention elements are adjacent to each other, in many cases, the plurality of attention elements represent the same object to be imaged, and the adjacent previous point-of-interest attention element and the current attention element are the same. The label number is attached.

  In addition, when there are a plurality of destination labels, the labeling means attaches the label that comes first in a predetermined order to the current attention element among the plurality of destination labels. Further, the labeling means reattaches the earliest label to the immediately preceding adjacent attention element. The case where there are a plurality of destination labels is a case where there are a plurality of previous time point adjacent attention elements. At this time, if the plurality of previous point-of-interest attention elements are not adjacent to each other, the plurality of previous point-of-interest adjacent attention elements are adjacent (connected) via the current point-of-interest element. In the labeling before the present time, the plurality of previous time point adjacent attention elements are given different label numbers. On the other hand, the labeling means re-attaches the most predecessor label to the adjacent element of interest at the previous time point, so that a plurality of label numbers that are different from each other despite being connected through the current attention element. It is possible to attach the same label number to the previous point of interest adjacent element and the current point of interest element.

  As described above, in order to attach the same label number to all connected elements of interest, it is not necessary to perform, for example, two scans on the image frame, and correct labeling can be performed with only one scan. . Therefore, it is possible to reduce the processing time for labeling.

  Further, since the labeling is completed in the first scan for the image frame, it is not necessary to temporarily store, for example, image data with a temporary label until the second scan. For this reason, it is possible to reduce the amount of memory necessary for completing the labeling.

  In addition, when the labeling unit reattaches the label that is the first in the order among the plurality of previous labels to the other target element, the image processing apparatus reuses the label that has already been attached to the other target element. A reusable label holding means that holds the reusable label, and the labeling means attaches a reusable label held by the reusable label holding means when labeling the target element selected by the target element selecting means. Is preferred.

  In addition, when the labeling unit reattaches the label that precedes the first in the order among the plurality of previous labels to the other target element, the image processing method uses the reuse label as the label that has already been attached to the other target element. In the labeling step, the reusable label held by the reusable label holding means when the labeling means labels the attention element selected by the attention element selection means. It is also suitable to attach.

  According to the present invention, the reuse label holding means holds a label (a label already attached) to be deleted when the label is reattached as a reuse label. The reuse label is used again when labeling is performed by the labeling means. This makes it possible to effectively use the amount of memory (label number limit amount) that can be used by the entire apparatus.

  In the image processing apparatus, each element constituting the image frame is composed of a plurality of pixels, and the element-of-interest selecting means selects an element having a predetermined number or more of pixels exceeding a certain criterion as a labeling target. It is also preferable to select it as an attention element.

  Also, in the image processing method, each element constituting the image frame is composed of a plurality of pixels, and in the target element selection step, an element having a predetermined number or more of pixels exceeding a certain criterion is subject to labeling. It is also suitable to be selected as an attention element.

  According to this invention, the process by the element-of-interest selection means is performed in units of “elements” composed of a plurality of pixels. At this time, the element-of-interest selection means selects, as the element of interest, an element having a predetermined number or more of pixels exceeding a certain criterion. On the other hand, for example, when a small number of noises are included in an element, even if the number of pixels exceeds a certain criterion, if the number does not reach the predetermined number X, the selected cell is not selected. As a result, the compatibility (noise resistance) against noise pixels can be improved. In addition, for example, an element that is not selected as a target element can be prevented from being labeled, and as a result, it is possible to effectively utilize the amount of memory that can be used by the entire apparatus (the limit amount of the label number). Become.

  In the image processing apparatus, the determination criterion may be pixel luminance.

  In the image processing method, the determination criterion may be a luminance of a pixel.

  According to the present invention, the luminance of the pixel is used as the determination criterion. This is because the luminance of a pixel is a suitable reference for determining whether or not the pixel is noise.

  The image processing apparatus further includes a block dividing unit that divides one image frame input by the input unit into a plurality of blocks including a plurality of elements, and the attention element selection unit selects a target element to be labeled. It is also preferable that the processing to be performed is performed in parallel in a plurality of blocks, and the labeling means performs the processing to label the target element selected by the target element selection unit in parallel in the plurality of blocks.

  The image processing method further includes a block dividing step in which the block dividing unit divides one image frame input by the input unit into a plurality of blocks made up of a plurality of elements. The process of selecting the target element to be performed is performed in parallel in a plurality of blocks, and in the labeling step, the process of labeling the target element selected by the target element selection unit is performed in parallel in the plurality of blocks. Is also suitable.

  According to this invention, the block dividing means divides one image frame into a plurality of blocks. At this time, one block includes a plurality of the above elements. Each process performed by the element-of-interest selecting unit and the labeling unit is performed in parallel for each of the plurality of blocks. As a result, the time required for each process can be reduced, and real-time processing can be performed.

  Further, the image processing apparatus includes a centroid position calculating unit that calculates a centroid position in a group of one or more elements having the same label, a centroid position of an element group in an arbitrary block, and an element group in another block. When the distance calculation means for calculating the distance between the center of gravity position and the distance value calculated by the distance calculation means is equal to or less than a predetermined threshold, the element group in an arbitrary block and the element group in another block, It is also preferable to further comprise integration means for re-labeling.

  Further, the image processing method includes a centroid position calculating step in which the centroid position calculating means calculates a centroid position in a group of one or more elements with the same label, and a distance calculating means in which the element group in an arbitrary block A distance calculating step for calculating a distance between the center of gravity position and the center of gravity position of the element group in another block; and the integration unit, when the distance value calculated by the distance calculating unit is equal to or less than a predetermined threshold, It is also preferable to further comprise an integration step of re-labeling the element group in the block and the element group in the other block.

  According to this invention, when the center-of-gravity position calculating means calculates the center-of-gravity position in the group of one or more elements having the same label, the distance calculating means calculates the center-of-gravity position of the element group in an arbitrary block, The distance between the center of gravity of the element group in the block is calculated. Then, when the distance value is equal to or less than a predetermined threshold, the integration unit re-labels the plurality of element groups related to the distance.

  As described above, when the processes performed by the element of interest selection unit and the labeling unit are performed in parallel for each block, for example, in the case of an imaging target that exists across multiple blocks, for each block Different labels may be attached. On the other hand, in the present invention, the integration unit represents the same imaging object (distance is equal to or less than a predetermined threshold), but the same group of elements with different labels are attached. Merge into labels (re-same labels). As a result, the same label can be attached to the image data representing the same imaging object.

  According to the present invention, when labeling imaged image data, high-speed processing can be performed, and further, the amount of memory required for the processing can be reduced.

  DESCRIPTION OF EMBODIMENTS Hereinafter, preferred embodiments of an image processing apparatus and an image processing method according to the present invention will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same elements are denoted by the same reference numerals, and redundant description is omitted. Further, the dimensional ratios in the drawings do not necessarily match those described.

  First, the overall configuration of the image processing system 1 according to an embodiment of the present invention will be described. FIG. 1 is a schematic configuration diagram of an image processing system 1. As illustrated in FIG. 1, the image processing system 1 includes a labeling device 100 (image processing device) and an imaging device 200. In the image processing system 1, the labeling device 100 performs labeling on image data captured by the imaging device 200 in real time. Hereinafter, each component of the image processing system 1 will be described in detail.

The imaging device 200 captures an imaging target by converting an incident optical signal into an electrical signal. In the present embodiment, the imaging device 200 is a video camera capable of continuously capturing moving images at high speed and high resolution. . The imaging apparatus 200 has a pixel area of 2 ^N × 2 ^N [pixels], performs video imaging with Y [FPS], and the number of gradations per pixel is M [bits]. Hereinafter, in the present embodiment, for example, N = 10 is set, Y = 1000 is set, and M = 10 is set.

  The imaging apparatus 200 uses, for example, a plurality of white markers that move against a black paper as an imaging target. FIG. 2 is a diagram illustrating one image frame imaged by the imaging apparatus 200. In one frame of 1024 × 1024 pixels, markers 1 to 3 are imaged as imaging objects. In the image frame, each pixel can be expressed using xy coordinates with the upper left pixel as a reference point (0, 0). That is, the upper right pixel corresponds to (1023, 0) on the xy coordinate, the lower left pixel corresponds to (0, 1023) on the xy coordinate, and the lower right pixel corresponds to (1023) on the xy coordinate. , 1023). This is the same in the following. The imaging apparatus 200 outputs captured image data to the labeling apparatus 100 in units of frames, for example, by well-known TMDS (Transition Minimized Differential Signaling) serial communication.

  Returning to FIG. 1, the labeling apparatus 100 receives an image frame captured by the imaging apparatus 200 and labels parts corresponding to the respective markers that are imaging objects in real time. As illustrated in FIG. 1, the labeling apparatus 100 includes a first labeling unit 110 and a second labeling unit 120.

  The first labeling unit 110 performs labeling on hardware, and can be physically incorporated on, for example, a normal FPGA (Field Programmable Gate Array) board. The FPGA board can be provided in a normal computer system including a CPU, a memory, a communication interface, a display interface, and the like. Functionally, the first labeling unit 110 includes an input unit 111 (input unit), a block dividing unit 112 (block dividing unit), a target cell selecting unit 113 (target element selecting unit), and a labeling unit 114 (labeling unit). ) And a label holding unit 115 (label holding means, reused label holding means). Hereinafter, each component of the 1st labeling part 110 is demonstrated in detail.

  The input unit 111 inputs an image frame (FIG. 2) captured by the imaging device 200 by, for example, TMDS serial communication. The input unit 111 performs, for example, fixed pattern noise removal and shading correction processing on the input image frame, and then outputs the image frame to the block dividing unit 112.

  The block dividing unit 112 receives an image frame from the input unit 111 and divides the single image frame into a plurality of blocks. FIG. 3 is a diagram for explaining the block dividing operation of the block dividing unit 112. As shown in FIG. 3, in this embodiment, the block dividing unit 112 divides one image frame of 1024 × 1024 pixels into 16 blocks of 64 × 1024 pixels. Thus, parallel processing for each block can be performed in subsequent processing described later. The block dividing unit 112 outputs the divided image frames to the target cell selecting unit 113.

  Returning to FIG. 1, the target cell selection unit 113 receives the image frame divided into blocks from the block division unit 112, specifies each cell constituting the image frame in order, and the specified cell is a label. It is determined whether the cell is a “target cell” to be attached or a “background cell” not to be labeled (hereinafter also referred to as “target cell selection process”). The cell-of-interest selection unit 113 performs the cell-of-interest selection process in parallel for each of the 16 blocks divided into blocks. Here, the “cell” is an element constituting an image frame and refers to a set of a plurality of pixels. In the present embodiment, one cell is, for example, 8 × 8 pixels. That is, one block (64 × 1024 pixels) consists of 8 × 128 cells, and one image frame (16 × 64 × 1024 pixels) consists of 16 × 8 × 128 cells.

  Hereinafter, the process performed by the attention cell selection unit 113 will be described in detail. The cell-of-interest selection unit 113 first refers to one cell (hereinafter, “current cell”) that is currently considered in a block to be processed (one block among 16 blocks that are processed in parallel). (Hereinafter, also referred to as “current cell specifying process”). FIG. 4 is a diagram for explaining the current cell specifying process by the target cell selecting unit 113. As shown in FIG. 4, the attention cell selection unit 113 specifies, for example, the current cell in order for all the cells constituting the block, for example, from the upper left cell to the lower right cell. Up to the cell, the current cell is specified in order while proceeding to the right in each cell in the same row and downward in each row.

Next, the target cell selection unit 113 counts how many pixels exceed a certain criterion in the current cell. In the present embodiment, the determination criterion is a luminance value of each pixel constituting the cell. Specifically, the cell-of-interest selecting unit 113 determines how many pixels having a luminance value exceeding I [cd / m ² ], which is the threshold value of the luminance value, among 8 × 8 pixel pixels in one cell. Count.

Then, the target cell selection unit 113 selects the current cell as the “target cell” when the number of pixels having a luminance value exceeding I [cd / m ² ] is equal to or more than X in the current cell. To do. On the other hand, the target cell selection unit 113 selects the current cell as the “background cell” when the number of pixels having a luminance value exceeding I [cd / m ² ] is not equal to or greater than X in the current cell. . In the present embodiment, a white marker that moves against a black paper background is used as the imaging target. Therefore, the white marker portion in the image data has a high luminance value and is easily selected as the “target cell”. Also, the black paper portion has a low luminance value and is easily selected as a “background cell”. In the present embodiment, for example, I = 120 and X = 10 are set.

  When the current cell specifying unit 113 finishes the current cell specifying process and the target cell selecting process for one cell, the target cell selecting unit 113 identifies the current cell and whether the current cell is a target cell or a background cell. Is output to the labeling unit 114. This output processing is performed in parallel for each of the 16 blocks divided into blocks.

  Returning to FIG. 1, the labeling unit 114 receives information specifying the current cell and information indicating whether the current cell is the target cell from the target cell selecting unit 113, and the current cell is the target cell. In this case, the label held by the label holding unit 115 is attached to the target cell, that is, labeling is performed. On the other hand, the labeling unit 114 does not perform labeling when the current cell is a background cell. The labeling unit 114 performs the labeling in parallel for each of the 16 divided blocks.

  The labels used for labeling are held in the label holding unit 115, and each label has a predetermined order for use. The label holding unit 115 holds a plurality of labels according to the above order, and includes a main queue (label holding unit) and a reuse queue (reuse label holding unit). The main queue holds preferentially used labels according to the above order, and the reuse queue holds labels that are used after all the labels held in the main queue are used. Is.

  FIG. 5 is a diagram illustrating a state in which the label is held in the label holding unit 115. 5A and 5B illustrate the main cue of the label holding unit 115, and the labels used for labeling are stored in the main cue of the label holding unit 115 in order from left to right. . In this embodiment, as shown in FIG. 5A, a total of 1024 label numbers are set to be usable, and the order of each label is determined by the size of the number. That is, label number 1 precedes label number 2 in order, and label number 1 is stored to the left of label number 2 in the main queue of FIG. When the labels are used, they are used in order from the left to the right of the main queue. For example, in FIG. 5A, the labels are used in the order of label number 1 and label number 2. FIG. 5B shows labels left in the main queue (labels that can be used thereafter) after label number 1 and label number 2 are used in order.

  Hereinafter, the labeling performed by the labeling unit 114 will be specifically described with further reference to FIGS. FIGS. 6 to 9 show four cells (cell a, cell b, cell c, out of 8 × 128 cells constituting a block to be processed (one block among 16 blocks processed in parallel)). Only a part of the cell d) is imaged. In each figure, for convenience of explanation, each cell is given a symbol (a, b, c, d), and the symbol is displayed on the upper left of the cell. Further, as described above, the attention cell selection unit 113 sequentially proceeds from the upper left cell to the lower right cell of the block in the right direction in each cell in the same row and in the lower direction in each row. The current cell is specified. Therefore, in FIGS. 6 to 9, each cell is specified as the current cell in the order of cell a, cell b, cell c, and cell d. Hereinafter, a cell specified as the current cell prior to the current cell at an arbitrary time is also referred to as a “previous time cell”. That is, for example, when the cell d is specified as the current cell, the cell a, the cell b, and the cell c are the previous time cells. In the following description, the cell d is a current cell and a target cell.

  The labeling unit 114 labels the target cell d according to the case 1 to be described later depending on the state of a label (hereinafter also referred to as “destination label”) attached to a previous time point cell adjacent to the target cell d. 4 is divided into four methods. In the following description, “adjacent” means that the target cell is in contact with one surface for convenience of description. That is, in the cell of interest d, the cell a is a previous point cell but is not adjacent. On the other hand, in the attention cell d, the cell b and the cell c are the previous time cells and are adjacent to each other. Hereinafter, the labeling method of the labeling unit 114 will be described for each case.

[Case 1]
The situation in case 1 is illustrated by FIG. That is, as shown in FIG. 6A, the cells b and c which are adjacent previous time cells are not labeled. From this, it can be seen that the cell b and the cell c were background cells before the present time. In this case, the label attaching unit 114 assigns the label number that precedes in order among the labels held in the main queue of the label holding unit 115 to the attention cell d. For example, in FIG. 5A, the label number that precedes in the order is the label number 1 stored on the leftmost side of the main queue. This is illustrated in FIG. 6 (B), and the attention cell d is labeled 1.

[Case 2]
The situation in case 2 is illustrated by FIG. That is, as shown in FIG. 7A, the cell b which is the adjacent previous time cell is not labeled, and the cell c which is the adjacent previous time cell has a label (destination label). attached. Thus, it can be seen that the cell b is the background cell and the cell c is the target cell before the present time. Further, since the cell c and the cell d are adjacent to each other, it can be seen that the cell c and the cell d correspond to different parts of the same imaging target (marker), respectively. In this case, the labeling unit 114 attaches the same label number as the previous label to the attention cell d. In FIG. 7B, the label number 1 of the cell c which is the previous label is attached to the attention cell d.

[Case 3]
The situation in case 3 is illustrated by FIG. That is, as shown in FIG. 8A, the cell b that is an adjacent previous point-in-time cell has a label (destination label), and the cell c that is an adjacent previous point-in-time cell has a label. Not. From this, it can be seen that the cell b was the target cell and the cell c was the background cell before the present time. In addition, since the cell b and the cell d are adjacent to each other, it can be seen that the cell b and the cell d respectively correspond to different parts of the same imaging target (marker). In this case, the labeling unit 114 attaches the same label number as the previous label to the attention cell d. In FIG. 8B, the label number 1 of the cell b which is the previous label is attached to the attention cell d.

[Case 4]
The situation in case 4 is illustrated by FIG. That is, as shown in FIG. 9A, labels (different destination labels) are attached to both the cells b and c, which are the previous time cells. Thus, it can be seen that the cell b and the cell c were the target cells before the present time. In addition, since cell b, cell c, and cell d are adjacent (connected through cell d), cell b, cell c, and cell d correspond to different parts of the same object to be imaged. It is expected that In this case, the labeling unit 114 attaches the first label in the order among the previous labels to the attention cell d. This is shown in FIG. 9B, in which not the label number 2 of the cell c, which is the previous label, but the label number 1 of the cell b, which is the preceding label in order, is attached to the target cell d. ing. Although not shown, for example, when the same label number is assigned to the cell b and the cell c, which are the previous time cells, the labeling unit 114 attaches the number to the cell of interest d.

  After that, the labeling unit 114 reattaches the first label that is not the first in the order to the first label that is the first in the order. That is, in the example of FIG. 9B, the label number 2 attached to the cell c is re-assigned to the label number 1 of the cell b which is a preceding label in order. FIG. 9C shows this, and the label number assigned to the cell c is changed from the label number 2 to the label number 1, and as a result, the cell b, the cell c, and the cell The same label is attached to d.

  The label number deleted by reassigning the label number is stored in the reuse queue of the label holding unit 115. FIG. 5C shows a state in which the label number 2 deleted in the case of FIG. 9 is stored in the reuse queue of the label holding unit 115. The label number stored in the reuse queue of the label holding unit 115 is used after all the label numbers stored in the main queue of the label holding unit 115 are used. That is, in the example of FIG. 5, the labeling unit 114 uses the label number 1024 that is the last in the order among the label numbers stored in the main queue, and then the label number that is the first in the order in the reuse queue. 2 is used again.

ただし、ｍ_０[Ｎ]はラベル番号Ｎの０次モーメントであり、ｍ_ｘ１[Ｎ]はラベル番号Ｎのｘ軸に関しての１次モーメントであり、ｍ_ｙ１[Ｎ]はラベル番号Ｎのｙ軸に関しての１次モーメントである。また、Ｋ（ｘ，ｙ，Ｎ）は、座標（ｘ，ｙ）のピクセルがラベル番号Ｎが貼られたセルに含まれる場合には「１」、含まれない場合には「０」となる関数である。 The labeling method by the labeling unit 114 has been described above. Along with this labeling, the labeling unit 114 calculates a zeroth-order moment and a first-order moment at each label. In the present embodiment, each moment is defined as follows, for example.

Where m ₀ [N] is the zeroth moment of label number N, m _x1 [N] is the first moment with respect to the x axis of label number N, and m _y1 [N] is the y axis of label number N. Is the first moment. K (x, y, N) is “1” when the pixel at the coordinate (x, y) is included in the cell to which the label number N is pasted, and “0” otherwise. It is a function.

Such moment calculation processing is performed while labeling each cell. That is, in the case 1, the label number 1 is used for labeling for the first time, and after the labeling, the zero-order moment m ₀ [1] of the label number 1 and the label are expressed by the above formulas (1) to (3). A first moment m _x1 [1] with respect to the x-axis of number 1 and a first moment m _y1 [1] with respect to the y-axis of label number 1 are calculated. Moreover, in the said cases 2-4, the label number 1 is used in multiple times for the labeling to an attention cell, and whenever the label number 1 is used, said Formula (1)-(3) , M ₀ [1], m _x1 [1], m _y1 [1] are updated. In the case 4, the label attached to the cell c is changed from the label number 2 to the label number 1, and in this case, for the label number 1, the above formulas (1) to (3) are used. m ₀ [1], m _x1 [1], m _y1 [1] are updated, and m ₀ [2], m _x1 [2], m _y1 [2] that have already been obtained for label number 2 are Deleted. Then, for example, later label number 2 is when it is reused, _{m 0} [2] for the label number _{2, m x1 [2],} m y1 [2] is calculated again.

  The labeling unit 114 performs the labeling and moment calculation processing as described above for all cells in one block. In addition, as described above, the labeling unit 114 performs labeling and moment calculation processing in parallel for each of the 16 blocks divided into blocks. Therefore, when the labeling and moment calculation processing ends in one block, The labeling and moment calculation processing for the entire one image frame input from the imaging apparatus 200 ends.

  FIG. 10 shows an example of a labeled image frame when the labeling as described above is performed on the image frame shown in FIG. 2, for example. In FIG. 10, one image frame is divided into 16 blocks, and labeling is performed for each block. Details regarding FIG. 10 will be described later. The labeling unit 114 outputs the labeled image frame as shown in FIG. 10 and the moment calculation value at each label number used in the labeled image frame to the second labeling unit 120.

  Returning to FIG. 1, the second labeling unit 120 corrects the result of the labeling performed by the first labeling unit 110 on software. In the result of the labeling performed by the first labeling unit 110, as can be seen from an example of the marker 2 in FIG. 10, for example, a set of a plurality of pixel data (cell group 2 and cell group 3) different for each block is one. When representing the imaging object (marker 2), different label numbers (label number 2 and label number 3) are attached to each set of pixel data for each block. This is because the first labeling unit 110 performs labeling independently for each block. The second labeling unit 120 performs correction for this on software.

  As shown in FIG. 1, the second labeling unit 120 can be physically provided in a normal computer system including a CPU, a memory, a communication interface, a display interface, and the like. The second labeling unit 120 and the first labeling unit 110 are connected by, for example, a PCI (Peripheral Component Interconnect) bus, and are provided so as to be able to transmit and receive labeled image data and the like. Functionally, the second labeling unit 120 includes an input unit 121, a centroid position calculation unit 122 (centroid position calculation unit), a distance calculation unit 123 (distance calculation unit), an integration unit 124 (integration unit), and an output unit 125. It is configured with. Hereinafter, each component of the second labeling unit 120 will be described in detail.

  The input unit 121 inputs a labeled image frame and a moment calculation value at each label number used in the labeled image frame from the first labeling unit 110. The input unit 121 outputs the input labeled image frame to the integration unit 124, and outputs a moment calculation value at each label number used in the labeled image frame to the barycentric position calculation unit 122.

ただし、ｘ[Ｎ]はラベル番号Ｎが付けられたセル群の重心位置のｘ座標であり、ｙ[Ｎ]はラベル番号Ｎが付けられたセル群の重心位置のｙ座標である。 The center-of-gravity position calculation unit 122 receives the calculated moment value of each label number from the input unit 121 and calculates the center-of-gravity position in the image frame for each label number. That is, the center-of-gravity position calculation unit 122 calculates the center-of-gravity position in the image frame with respect to the cells having the same label. Specifically, the center-of-gravity position calculation unit 122 calculates the center-of-gravity position (x [N], y [N]) of the cell group to which the label number N is attached based on, for example, the following mathematical formula.

However, x [N] is the x coordinate of the barycentric position of the cell group with the label number N, and y [N] is the y coordinate of the barycentric position of the cell group with the label number N.

  In the example as shown in FIG. 10, the centroid position calculation unit 122 uses the centroid position (x [] in the cell group (a set of pixel data representing the marker 1, the cell group 1) labeled 1 in the block 1. 1], y [1]) are calculated, and the center of gravity position (x [2], y [2] in the cell group (a set of pixel data representing the marker 2, cell group 2) labeled 2 in block 10 is calculated. ]), And the block 11 calculates the barycentric positions (x [3], y [3]) in the cell group labeled with the label number 3 (a set of pixel data representing the marker 2, cell group 3), In block 16, the barycentric position (x [4], y [4]) in the cell group (label data group representing marker 3, cell group 4) labeled 4 is calculated. In FIG. 10, the barycentric position calculated by the barycentric position calculating unit 122 is shown as a point in a circle representing each marker. The centroid position calculation unit 122 outputs information representing the centroid position of each cell group calculated in this way to the distance calculation unit 123.

  The distance calculation unit 123 receives information representing the centroid position in each cell group from the centroid position calculation unit 122, and the centroid position of a certain cell group in an arbitrary block and the centroid position of a certain cell group in another block. The distance between is calculated. Specifically, the distance calculation unit 123 first identifies a cell group having a centroid position within a certain range from a boundary between two blocks (hereinafter also referred to as “block boundary”), and identifies the cell group. And information indicating the center of gravity position of the cell group are stored for each block boundary. Then, when there are a plurality of cell groups stored in association with an arbitrary block boundary, the distance calculation unit 123 obtains the distance between the centroid positions between the cell groups.

Specifically, in the example illustrated in FIG. 10, the distance calculation unit 123 first determines a constant from each block boundary (B ₁ , B ₂ ,..., B ₉ , B ₁₀ , B ₁₁ ,..., B ₁₅ ). A cell group having a centroid position within the range is specified. In the present embodiment, the certain range is set to, for example, upper and lower 64 pixels (one upper and lower block). In this case, for example, the distance calculation unit 123 specifies the cell group 1 for the block boundary B ₁ , specifies the cell group 2 and the cell group 3 for the block boundary B ₁₀ , and sets the block boundary B ₁₅ On the other hand, the cell group 4 is specified.

Next, the distance calculation unit 123 stores information representing the centroid position of each of the identified cell groups for each block boundary. In the present embodiment, the distance calculation unit 123 prepares, for example, boundary [B] [xy], which is a two-dimensional array, and associates the centroid position xy of the cell group with the label number N associated with the block boundary B _i. Information representing _N is stored in boundary [B _i ] [xy _N ]. Note that, based on the above formulas (4) and (5), the centroid position xy _N = (x [N], y [N]). That is, in the example shown in FIG. 10, the distance calculating unit 123, for example, in association with block boundary B _1, the center of gravity of the cell group 1 (x [1], y [1]) the information indicating the boundary [B ₁ ] Store in [xy ₁ ]. Further, the distance calculation unit 123 stores information representing the centroid position (x [2], y [2]) of the cell group 2 in boundary [B ₁₀ ] [xy ₂ ] in association with the block boundary B ₁₀ . Information representing the barycentric position (x [3], y [3]) of the cell group 3 is stored in boundary [B ₁₀ ] [xy ₃ ]. In addition, the distance calculation unit 123 stores information representing the barycentric position (x [4], y [4]) of the cell group 4 in boundary [B ₁₅ ] [xy ₄ ] in association with the block boundary B ₁₅ .

Next, the distance calculation unit 123 calculates the distance between the centroid positions when there are a plurality of pieces of information stored in association with an arbitrary block boundary. That is, in the example illustrated in FIG. 10, the distance calculation unit 123 first scans each row of boundary [B] [xy], which is a two-dimensional array, and finds a row in which a plurality of values are stored. In the example shown in FIG. 10, the line ₁₀ B, the center of gravity of the cell group 2 (x [2], y [2]) and, the center of gravity of the cell group 3 (x [3], y [3]) and Is found to be stored. Next, the distance calculation unit 123 obtains a distance d between the centroid position (x [2], y [2]) and the centroid position (x [3], y [3]). Although not illustrated in FIG. 10, the distance calculation unit 123 does not perform processing for obtaining the distance between the centroid positions between a plurality of cell groups existing in the same block. Next, the distance calculation unit 123 integrates the obtained distance d and information for specifying a plurality of cell groups related to the distance d (information for specifying the cell group 2 and information for specifying the cell group 3). It outputs to 124.

  The integration unit 124 receives the distance obtained by the distance calculation unit 123 and information for specifying a plurality of cell groups related to the distance. Further, the integration unit 124 receives the labeled image frame as illustrated in FIG. 10 from the input unit 121. Then, when the distance input by the distance calculation unit 123 is equal to or less than the predetermined threshold D, the integration unit 124 includes a plurality of cell groups related to the distance in the labeled image frame input by the input unit 121. Re-label the.

  That is, in the example illustrated in FIG. 10, the integration unit 124 determines that the distance d between the centroid position (x [2], y [2]) and the centroid position (x [3], y [3]) is predetermined. When the threshold value D is equal to or less than the threshold value D, different label numbers (label number 2 and label number 3) assigned to the cell group 2 and the cell group 3 are integrated into the label number 2, for example. That is, the label number 2 is reassigned to the cell group 3. As a result, the same label number is assigned to the cell group 2 and the cell group 3 which are image data representing the same imaging object (marker 2). On the other hand, when the distance d is not equal to or smaller than the predetermined threshold D, the integration unit 124 keeps the label numbers attached to the cell group 2 and the cell group 3 as they are. In this case, for example, the cell group 2 and the cell group 3 represent different imaging objects.

  The integration unit 124 outputs the image frame that has been relabeled as described above to the output unit 125 (FIG. 1). The output unit 125 can output the image frame input from the integration unit 124 to another device (not shown) such as a position tracking device. Thus, for example, a position tracking device or the like can be used when, for example, the label attached to the image frame recognizes and tracks the position of each marker of the imaging target.

  Next, an operation (image processing method) performed by the image processing system 1 will be described with reference to FIG. FIG. 11 is a sequence diagram showing the overall operation of the image processing system 1.

  First, the imaging apparatus 200 images, for example, a plurality of imaging objects (step S101), and outputs the captured image data to the labeling apparatus 100 in units of frames (see FIG. 2 above) (step S102).

  Next, the input unit 111 of the first labeling unit 110 of the labeling apparatus 100 inputs the image frame output from the imaging apparatus 200 in step S102, and performs, for example, fixed pattern noise removal and shading correction processing. Thereafter, the input unit 111 outputs the image frame to the block dividing unit 112 (step S103).

  Next, the block dividing unit 112 divides the one image frame input from the input unit 111 in step S103 into a plurality of 16 blocks (see FIG. 3 above). The block dividing unit 112 outputs the image frame divided into the blocks to the target cell selecting unit 113 (step S104).

  Next, the cell-of-interest selection unit 113 inputs the image frame divided by the block dividing unit 112 in step S104, and performs cell-of-interest selection processing in parallel for each of the 16 divided blocks. This is performed (step S105).

  FIG. 12 is a flowchart for explaining the cell-of-interest selection process in step S105. As shown in FIG. 12, the cell-of-interest selection unit 113 first identifies one current cell in a block to be processed (one block out of 16 blocks processed in parallel). At this time, the target cell selection unit 113 uses the current cell specifying method described above with reference to FIG. 4, for example (step S <b> 201).

Next, the cell-of-interest selection unit 113 has pixels having a luminance value exceeding the luminance value threshold value I [cd / m ² ] among, for example, 8 × 8 pixels in the current cell identified in step S201. Is counted (step S202).

  Next, the cell-of-interest selection unit 113 determines whether or not the number of pixels counted in step S202 is equal to or greater than a predetermined threshold value X (step S203). When the number of pixels is equal to or greater than the predetermined threshold value X (step S203: YES), the target cell selection unit 113 selects the current cell as the “target cell” (step S204). On the other hand, when the number of pixels is not equal to or greater than the predetermined threshold value X (step S203: NO), the target cell selection unit 113 selects the current cell as the “background cell” (step S205).

  When the attention cell selection processing is completed as described above, the attention cell selection unit 113 labels information for specifying the current cell and information indicating whether the current cell is the attention cell or the background cell. It outputs to the part 114 (step S105 of FIG. 11).

  Next, the labeling unit 114 inputs the information specifying the current cell output from the target cell selecting unit 113 in step S105 and the information indicating whether or not the current cell is the target cell. Labeling is performed in parallel for each of the 16 divided blocks (step S106).

  FIG. 13 is a flowchart for explaining the labeling method in step S106. As shown in FIG. 13, the labeling unit 114 first determines whether or not the current cell is a target cell (step S301).

  In step S301, when the current cell is not the target cell but a background cell (step S301: NO), the labeling unit 114 ends the processing in step S106 without performing labeling described later.

  On the other hand, if the current cell is the target cell in step S301 (step S301: YES), the labeling unit 114 checks whether there is a destination label (step S302).

  If there is no previous label in step S302 (step S302: NO), the labeling unit 114 sets the label number that precedes in the order among the labels held in the main queue of the label holding unit 115 as the target cell. wear. In addition, for example, when all the labels held in the main queue of the label holding unit 115 are already used, the labeling unit 114 includes the labels held in the reuse queue of the label holding unit 115. The label number that precedes in the order is assigned to the target cell. Note that the above case corresponds to case 1 described above (step S303).

  If there is a destination label in step S302 (step S302: YES), the labeling unit 114 determines whether there are a plurality of destination labels (step S304).

  In step S304, when there is only one destination label (step S304: NO), the labeling unit 114 attaches the same label number as the only one destination label to the target cell. Note that this case corresponds to the cases 2 and 3 described above (step S305).

  In step S304, when there are a plurality of destination labels (step S304: YES), the labeling unit 114 attaches the destination label that precedes in the order to the target cell (step S306). Although not illustrated in the drawings, when all of the plurality of destination labels are the same, the labeling unit 114 attaches the same destination label to the target cell.

  Next, the labeling unit 114 reattaches the first label that is not the first in the order among the plurality of first labels to the first label that is the first in the order. The label number deleted by re-assigning the label number is stored in the reuse queue of the label holding unit 115 (step S307). Note that the cases in step S306 and step S307 correspond to case 4 described above.

  Along with the labeling described above, the labeling unit 114 calculates or updates the zeroth-order moment and the first-order moment in each label based on the above-described equations (1) to (3) (step S107 in FIG. 11).

  Next, in all the cells of the block to be processed (one block out of 16 blocks to be processed in parallel), it is determined whether or not the processing in steps S105 to S107 has been completed (step S108). ). In the present embodiment, the target cell selection unit 113 specifies the current cell using the current cell specification method described above with reference to FIG. For this reason, for example, the time point when the process of step S105 to step S107 is completed for the rightmost cell of the process target block is the time point when the process is completed for all the cells in the process target block. Judgment can be made.

  If it is determined in step S108 that the processing in steps S105 to S107 has not been completed for all cells in the processing target block (step S108: NO), the processing flow proceeds to step S105.

  On the other hand, in step S108, when the processing in steps S105 to S107 is completed for all the cells in one block to be processed (step S108: YES), one image input in step S103. This means that the processing from step S105 to step S107 has been completed for the entire frame. Then, the labeling unit 114 outputs, for example, a labeled image frame as illustrated in FIG. 10 and a moment calculation value at each label number used in the labeled image frame to the second labeling unit 120 ( Step S109).

  Next, the input unit 121 of the second labeling unit 120 receives the input relating to step S109. The input unit 121 outputs the input labeled image frame to the integration unit 124, and outputs the moment calculation value at each label number used in the labeled image frame to the barycentric position calculation unit 122 (step S110).

  Next, based on the moment calculation value input from the input unit 121 in step S110, the center-of-gravity position calculation unit 122 is assigned the same label, for example, based on the above-described mathematical formulas (4) and (5). The center-of-gravity position in the image frame for each cell group (for each label number) is calculated. The centroid position calculation unit 122 outputs information representing the calculated centroid position for each cell group to the distance calculation unit 123 (step S111).

  Next, the distance calculation unit 123 receives information representing the centroid position of each cell group calculated by the centroid position calculation unit 122 in step S111, and the centroid position is within a certain range from an arbitrary block boundary. Thus, the distance between the centroid position of a certain cell group in an arbitrary block and the centroid position of a certain cell group in another block is calculated. As described above, this distance calculation processing can be performed based on, for example, a two-dimensional array. Next, the distance calculation unit 123 outputs the obtained distance and information for specifying a plurality of cell groups related to the distance to the integration unit 124 (step S112).

  Next, the integration unit 124 inputs the distance obtained by the distance calculation unit 123 in step S112 and information for specifying a plurality of cell groups related to the distance. Then, when the distance calculated in step S112 is equal to or less than the predetermined threshold D, the integration unit 124, in the labeled image frame input by the input unit 121 in step S110, has a plurality of distances related to the distance. Re-label the cells. Then, the integration unit 124 outputs the re-labeled image frame to the output unit 125 (step S113).

  Finally, the output unit 125 outputs the image frame input from the integration unit 124 in step S113 to another device such as a position tracking device (step S114).

  Next, operations and effects of the image processing system 1 according to the present embodiment will be described. According to the image processing system 1 of the present embodiment, when any cell in the input image frame is selected as the current cell of interest (hereinafter also referred to as “currently cell of interest”), the labeling unit 114. Attaches the label held by the label holding unit 115 to the selected current attention cell. At this time, the labeling unit 114 is adjacent to the current cell of interest on the image frame, and the previous time point cell (hereinafter, also referred to as “previous point adjacent cell of interest”) that was the cell of interest before the current time point. In some cases, the same label number as the label (destination label) attached to the target cell adjacent to the previous time point is attached to the current target cell. The fact that a plurality of cells of interest are adjacent to each other is assumed that the plurality of cells of interest often represent the same imaging object, and the plurality of cells of interest are assigned the same label number.

  In addition, when there are a plurality of destination labels, the labeling unit 114 attaches the label that precedes in the predetermined order among the plurality of destination labels to the current attention cell. Further, the labeling unit 114 reattaches the earliest label to the previous point of interest adjacent cell. The case where there are a plurality of destination labels is a case where there are a plurality of previous point-of-interest cells of interest. At this time, when the plurality of previous point-of-interest adjacent cells are not adjacent to each other, the plurality of previous point-of-interest adjacent cells are adjacent (connected) via the current point-of-interest cell. In the labeling before the present time, different label numbers are assigned to the plurality of previous time point adjacent attention cells. On the other hand, the labeling unit 114 adds a label number different from each other even though the labeling unit 114 is connected via the current target cell by reattaching the earliest label to the previous target cell. It is possible to attach the same label number to the previous point-of-interest attention cell and the current point-of-interest cell.

  As described above, in order to attach the same label number to all connected cells of interest, it is not necessary to perform, for example, two scans on the image frame, and correct labeling can be performed with only one scan. . Therefore, it is possible to reduce the processing time for labeling.

  Further, since the labeling is completed in the first scan for the image frame, it is not necessary to temporarily store, for example, image data with a temporary label until the second scan. For this reason, it is possible to reduce the amount of memory necessary for completing the labeling.

  Further, in the present embodiment, a label that is deleted when a label is reattached (a label that has already been attached) is held in the reuse queue of the label holding unit 115 as a reuse label. The reuse label is used again when labeling by the labeling unit 114 is performed. This makes it possible to effectively use the amount of memory (label number limit amount) that can be used by the entire apparatus.

  In the present embodiment, the process by the target cell selection unit 113 is performed in units of cells including a plurality of pixels. At this time, the cell-of-interest selection unit 113 selects a cell having a predetermined number X or more of pixels exceeding a certain determination criterion as the cell of interest. On the other hand, for example, when a small number of noises are included in a cell, even if the number of pixels exceeds a certain criterion, if the number does not reach the predetermined number X, the pixel is not selected as a target cell. As a result, the compatibility (noise resistance) against noise pixels can be improved. In addition, for example, a cell that is not selected as a target cell can be prevented from being labeled, and as a result, it is possible to effectively use the amount of memory (label number limit) that can be used in the entire apparatus. Become.

  In the present embodiment, the luminance of the pixel is used as the determination criterion. This is because the luminance of a pixel is a suitable reference for determining whether or not the pixel is noise.

  In the present embodiment, the block dividing unit 112 divides one image frame into a plurality of blocks. At this time, one block consists of a plurality of cells. Then, each process performed by the attention cell selection unit 113 and the labeling unit 114 is performed in parallel for each of the plurality of blocks. As a result, the time required for each process can be reduced, and real-time processing can be performed.

  In the present embodiment, when the centroid position calculation unit 122 calculates the centroid position in a group of one or more cells with the same label, the distance calculation unit 123 determines the centroid position of the cell group in an arbitrary block. And the distance between the center of gravity of the cell group in the other block is calculated. When the distance is equal to or less than the predetermined threshold D, the integration unit 124 re-labels the plurality of cell groups related to the distance.

  In the present embodiment, each process performed by the cell-of-interest selection unit 113 and the labeling unit 114 is performed in parallel for each block as described above. For this reason, for example, in the case of an imaging target that exists across a plurality of blocks, a different label may be attached to each block. On the other hand, in the present embodiment, the integration unit 124 integrates cell groups that are attached with different labels into the same label even though they represent the same imaging object. As a result, the same label can be attached to the image data representing the same imaging object. In addition, since this integration process can be performed based on an array (for example, the above-described two-dimensional array boundary [B] [xy]) instead of image data, a long processing time is not required.

  As mentioned above, although preferred embodiment of this invention was described, it cannot be overemphasized that this invention is not limited to the said embodiment.

  For example, in the above-described embodiment, “adjacent” has been described as one surface being in contact with any cell (so-called 4-neighbor labeling). However, the present invention is not limited to this, and so-called 8-neighbor labeling is also possible. The concept of the present invention can be applied.

  Further, in the above-described embodiment, for example, three markers have been described as the number of markers that are imaging objects, but the concept of the present invention can be applied to more imaging objects.

  Further, for example, in the above embodiment, the background cell is not labeled. However, the present invention is not limited to this, and the background cell may be labeled to indicate that the cell is a background cell. .

1 is a schematic configuration diagram of an image processing system. It is the figure which imaged one image frame imaged with the imaging device. It is a figure for demonstrating block division operation | movement. It is a figure for demonstrating the present cell specific process. It is a figure which shows a mode that the label is hold | maintained. It is a figure for demonstrating the labeling method. It is a figure for demonstrating the labeling method. It is a figure for demonstrating the labeling method. It is a figure for demonstrating the labeling method. It is a figure which shows an example of an image frame with a label. It is a sequence diagram which shows the whole operation | movement of an image processing system. It is a flowchart for demonstrating attention cell selection processing. It is a flowchart for demonstrating the method of labeling.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Image processing system, 100 ... Labeling apparatus, 110 ... 1st labeling part, 111 ... Input part, 112 ... Block division part, 113 ... Target cell selection part, 114 ... Label attaching part, 115 ... Label holding part, 120 ... 2nd labeling part, 121 ... input part, 122 ... gravity center position calculation part, 123 ... distance calculation part, 124 ... integration part, 125 ... output part, 200 ... imaging device.

Claims (10)

An image processing device for inputting an image frame captured by an imaging device and labeling each element constituting the image frame,
Label holding means for holding a plurality of labels having a predetermined order;
Input means for inputting an image frame captured by the imaging device;
By designating each element in the image frame input by the input means in order, information specifying the current element and information indicating whether or not the current element is a target element to be labeled are output. The element of interest selection means,
Each time the attention element selection means outputs information, the attention element selected by the attention element selection means based on information specifying the current element and information indicating whether or not the current element is the attention element Labeling means for attaching a label held by the label holding means,
With
Each element constituting the image frame is composed of a plurality of pixels,
The element of interest selection means is
Select an element that has a predetermined number of pixels exceeding a certain criterion as a target element to be labeled,
The labeling means includes
When the attention element selection means is selected before selecting the current attention element, and there is another attention element adjacent to the current attention element on the image frame, it is attached to the other attention element. A prior label, which is a selected label, is attached to the current attention element,
When there are a plurality of the previous labels, the current attention element is given a label that is the earliest in the order among the plurality of previous labels, and the other attention element is included in the order among the plurality of previous labels. An image processing apparatus characterized by reattaching a label that precedes the most. When the labeling unit reattaches a label that is the first in the order among the plurality of previous labels to the other target element, the label already attached to the other target element is retained as a reuse label. A reusable label holding means;
2. The image according to claim 1, wherein the labeling means attaches a reuse label held by the reuse label holding means when a label is attached to the attention element selected by the attention element selection means. Processing equipment.   The image processing apparatus according to claim 1, wherein the determination criterion is a luminance of a pixel. A block dividing unit that divides one image frame input by the input unit into a plurality of blocks including the plurality of elements;
The element of interest selection means performs processing for selecting an element of interest to be labeled in parallel in the plurality of blocks,
The image according to any one of claims 1 to 3, wherein the labeling unit performs a process of labeling the target element selected by the target element selecting unit in parallel in the plurality of blocks. Processing equipment. Centroid position calculating means for calculating a centroid position in a group of one or more elements having the same label;
A distance calculating means for calculating a distance between the centroid position of the element group in an arbitrary block and the centroid position of the element group in another block;
When the distance value calculated by the distance calculation means is a predetermined threshold value or less, the integration means for re-labeling the element group in the arbitrary block and the element group in the other block;
The image processing apparatus according to claim 4, further comprising: An image processing method for inputting an image frame captured by an imaging device and labeling each element constituting the image frame,
An input step in which the input means inputs an image frame captured by the imaging device;
The element of interest selection means designates each element in the image frame input by the input means in order, thereby identifying the current element and whether or not the current element is the element of interest to be labeled. An element selection step for outputting information representing
Labeling means, based on information specifying the current element and information indicating whether or not the current element is the attention element, every time the attention element selection means outputs information, A labeling step of attaching a label held by a label holding means for holding a plurality of labels having a predetermined order to the element of interest selected by
With
Each element constituting the image frame is composed of a plurality of pixels,
In the attention element selection step,
An element having a predetermined number of pixels exceeding a certain criterion is selected as a target element to be labeled,
In the labeling step, the labeling means includes:
When the attention element selection means is selected before selecting the current attention element, and there is another attention element adjacent to the current attention element on the image frame, it is attached to the other attention element. A prior label, which is a selected label, is attached to the current attention element,
When there are a plurality of the previous labels, the current attention element is given a label that is the earliest in the order among the plurality of previous labels, and the other attention element is included in the order among the plurality of previous labels. An image processing method characterized by reattaching a label that precedes the most. When the labeling unit reattaches a label that is the earliest in the order among the plurality of previous labels to the other target element, the label already attached to the other target element is reused as a reuse label. Held in the label holding means,
In the labeling step, the labeling means attaches a reuse label held in the reuse label holding means when labeling the attention element selected by the attention element selection means. The image processing method according to claim 6.   The image processing method according to claim 6, wherein the determination criterion is a luminance of a pixel. The block dividing means further comprises a block dividing step of dividing one image frame input by the input means into a plurality of blocks composed of a plurality of the elements,
In the attention element selection step, the process of selecting the attention element to be labeled is performed in parallel in the plurality of blocks,
9. The labeling step according to claim 6, wherein in the labeling step, a process of labeling the target element selected by the target element selecting unit is performed in parallel in the plurality of blocks. Image processing method. A center-of-gravity position calculating unit that calculates a center-of-gravity position in a group of one or more elements with the same label;
A distance calculation step in which the distance calculation means calculates a distance between the centroid position of the element group in an arbitrary block and the centroid position of the element group in another block;
The integration unit reintegrates the same label to the element group in the arbitrary block and the element group in the other block when the distance value calculated by the distance calculation unit is a predetermined threshold value or less. Steps,
The image processing method according to claim 9, further comprising:

Images