JP5953166B2 - Image processing apparatus and program - Google Patents

Description

  The present invention relates to an image processing apparatus and program for processing an image obtained by imaging an animal with a tail.

  Analysis of recognition behavior of a target object placed in a space using animals such as mice and rats has been widely performed. In the analysis of such behavior, the observer visually determines whether or not the rat or the like touches the target object in the space photographed by the video camera, and records that it is touched / not touched. Currently.

Takeshi Koide, “Basic knowledge of mouse experiments”, Ohmsha, Japan, March 2009

  However, in the conventional method described above, since the observer looks at the time taken, it takes time until the result is obtained. In addition, there are cases where the evaluation criteria for whether or not the object is touched is different for each observer, and there is also a problem in comparing the results.

  The present invention has been made in view of the above circumstances, and an object thereof is to provide an image processing apparatus and a program that can perform behavior analysis of an animal with a certain evaluation criterion relatively quickly. .

  An invention for solving the problems of the conventional example is an image processing apparatus, and a receiving means for receiving a series of image information in which a target animal having a tail and a main body and a target object are imaged. A setting unit that identifies a target object from the image information, sets a range of interest around a contour line of the identified target object, and recognizes the main body of the target animal from the image information, and the main body recognized in the image information Recognizing means for recognizing a line connecting the recognized main body and the detected point as a tail existence range by finding a point having a color different from the background in the vicinity, the center of gravity of the recognized main body, and the contour line of the main body Temporarily determining means for tentatively determining the position farthest from the center of gravity as the head of the target animal, and the temporary determining means, a vector from the recognized body centroid to a line segment that is the recognized tail existence range. Provisional And correction processing means for subjecting the modification process of the recognized position of the boss was head, and means for outputting the recognized position of the head after the correction processing, in which it was decided to include a.

  Here, it may further include marker output means for outputting a predetermined marker when the output recognition position of the head is included in the attention range.

  A program according to an aspect of the present invention includes a computer, a target animal including a tail and a main body, a receiving unit that receives a series of image information in which a target object is imaged, and the target object from the image information. And setting means for setting an attention range around the outline of the identified target object, and recognizing the main body of the target animal from the image information, and the background is colored in the vicinity of the main body recognized in the image information. Recognizing means for recognizing a line connecting the recognized main body and the detected point as a tail existence range by finding a different point, the center of gravity of the recognized main body, and the position farthest from the center of gravity on the contour line of the main body Tentatively determining means for tentatively determining the head of the target animal, and the head from which the temporary determining means has temporarily determined a vector from the recognized main body centroid to a line segment that is the recognized tail existence range. And correction processing means for subjecting the modification process of the recognized position of the means for outputting the recognized position of the head after the correction processing, in which a function as an.

  According to the present invention, the recognition position of the head of the target animal is output by relatively simple image processing. Therefore, the recognition position of the animal can be compared with a certain evaluation criterion relatively quickly and using the information on the recognition position. Behavioral analysis can be performed.

It is a block diagram showing an example of an image processing apparatus according to the present embodiment. It is explanatory drawing showing an example of the image information made into the object of the process by the image processing apparatus which concerns on this Embodiment. It is a functional block diagram showing the example of the image processing apparatus which concerns on this Embodiment. It is explanatory drawing showing the example of the process by the image processing apparatus which concerns on this Embodiment. It is explanatory drawing showing the example of the process which identifies the direction of the head by the image processing apparatus which concerns on this Embodiment. It is explanatory drawing showing the example of the information output by the image processing apparatus which concerns on this Embodiment. It is explanatory drawing showing another example of the information output by the image processing apparatus which concerns on this Embodiment.

  Embodiments of the present invention will be described with reference to the drawings. As illustrated in FIG. 1, the image processing apparatus 1 according to the embodiment of the present invention is connected to a camera 2 and includes a control unit 11, a storage unit 12, an operation unit 13, a display unit 14, and an interface unit 15. Composed.

  Here, the control unit 11 is a program control device such as a CPU, and operates according to a program stored in the storage unit 12. In the present embodiment, the control unit 11 accepts a series of image information (moving images) captured by the camera 2 and stores it in the storage unit 12. The image information stored in the storage unit 12 is a processing target. The control unit 11 receives this image information from the storage unit 12 and performs the next process. In this image information, a target animal having a tail and a main body and a target object are imaged.

  The control unit 11 identifies the target object from the image information, and sets an attention range around the contour line of the identified target object. Further, the control unit 11 recognizes the main body of the target animal from the image information, finds a point having a color different from the background near the main body recognized in the image information, and connects a line connecting the recognized main body and the found point. Recognize as the existence range of the tail. Further, the control unit 11 preliminarily determines the recognized center of gravity of the main body and the position farthest from the center of gravity on the outline of the main body as the head of the target animal, and the tail existence range recognized from the recognized main body center of gravity. Using the vector to a certain line segment, the correction processing of the recognition position of the head temporarily determined by the temporary determination means is executed. And the control part 11 outputs the recognition position of the head after this correction process. Details of the processing by the control unit 11 will be described later.

  The storage unit 12 holds a program executed by the control unit 11. The program may be provided by being stored in a computer-readable storage medium such as a DVD-ROM and copied to the storage unit 12. The storage unit 12 also operates as a work memory for the control unit 11. In the present embodiment, the storage unit 12 stores image information to be processed by the control unit 11.

  The operation unit 13 is, for example, a mouse or a keyboard that is a pointing device. The operation unit 13 accepts a user operation and outputs information representing the content of the operation to the control unit 11. The display unit 14 is a display or the like, and displays and outputs information according to an instruction input from the control unit 11. The interface unit 15 is an interface such as a USB (Universal Serial Bus), for example, and outputs information input from an external device such as the camera 2 to the control unit 11.

  The camera 2 captures an image of a box with the target object arranged on a plane on which the target animal including the tail and the main body can move at predetermined timings (for example, every 1/45 seconds) from the direction of planar view. The camera 2 outputs image information obtained by the imaging to the image processing apparatus 1. Image information obtained by imaging by the camera 2 is, for example, as illustrated in FIG. FIG. 2 shows a state in which a mouse (a target animal with a tail) and target objects α and β are arranged in a box. The mouse has a tail portion T and a main body portion B (a portion other than the tail). A plane floor portion (surface on which the target animal moves) as a background to be photographed is provided with a sheet (sand color) of a color (background color) different from the color of the mouse.

  Here, the content of the process by the control part 11 is demonstrated. As illustrated in FIG. 3, the control unit 11 according to the present embodiment executes a program stored in the storage unit 12 to functionally receive the reception unit 21, the setting unit 22, the recognition unit 23, The provisional determination unit 24, the correction processing unit 25, and the output unit 26 are included.

  The receiving unit 21 receives a series of image information captured by the camera 2. The setting unit 22 displays one of the image information received by the receiving unit 21 on the display unit 14 and receives an input of the position of the target object from the user. Here, it is assumed that the user designates at least a part of the pixels in the range where each target object is photographed as the position of the target object for each target object. The setting unit 22 obtains the pixel value of the designated pixel. The setting unit 22 predetermines an approximate color within a predetermined range from the color represented by the obtained pixel value (centering on the color represented by the pixel value of the designated pixel on the coordinates of the predetermined color space). It may be determined that it is within a specified Euclidean distance). Then, the setting unit 22 selects an approximate color pixel from adjacent pixels starting from the pixel designated by the user, and then performs a process of selecting the approximate color pixel adjacent to the selected pixel. A pixel group selected repeatedly (a continuous range of approximate colors, hereinafter referred to as a pixel block) is selected as a pixel block in a range where the target object is imaged.

  In a specific example of the present embodiment, the setting unit 22 uses a grab cut (GrabCut) to include a foreground (a target object is a target object) including pixels designated from image information received by the receiving unit 21. A process of selecting a pixel in the range being imaged is performed. Since processing such as grabbing and cutting is widely known, detailed description thereof will be omitted here. When the pixel block in the range in which the target object is imaged is selected in this way, the setting unit 22 sets the pixel group at the peripheral portion of the pixel block as the pixel group in the target range. Specifically, a pixel group that belongs to the pixel block and is adjacent to a pixel that does not belong to the pixel block may be set as the pixel group in the target range. Alternatively, a pixel that is adjacent to a pixel that does not belong to the pixel block and a pixel that is adjacent to a pixel that does not belong to the pixel block, and a predetermined distance (the distance between the pixels is between the pixel coordinates). It is also possible to set a pixel group consisting of pixels within a certain range as a pixel group in the target range. If the position of the camera 2 and the position of the target object are fixed, the pixel group of the attention range is the same in any image information. Therefore, if the pixel group of the attention range is defined in one image information, In the other image information, the same pixel group (in the corresponding coordinates) becomes the pixel group in the target range.

  The recognition unit 23 sequentially selects each piece of image information received by the receiving unit 21 as a processing target. Then, the main body of the target animal is recognized from the selected image information of the processing target. There are various recognition methods. For example, if the body part of the target animal is recognized by using a CAMShift (Continuous Adaptive Mean Shift) method or the like that identifies pixels of a moving object from a series of image information. Good. Specifically, the recognizing unit 23 applies the CAMShift method to a series of image information, and specifies a region in which the main body portion of the target animal is included in each image information. The recognition unit 23 generates a histogram of pixel values in an area specified as an area including the main body portion of the target animal from the image information selected as the processing target, and finds the largest pixel value in the histogram. (Most pixel value). The recognizing unit 23 uses a predetermined Euclidean color centered on the color represented by the most recent pixel value on the coordinates of the predetermined color space in a predetermined range from the color represented by the most pixel value. Within the area found by CAMShift, select the pixel of the approximate color from the neighboring pixels starting from the pixel with the lowest pixel value within the area found by CAMShift. , A pixel group selected by repeating the process of selecting pixels of the approximate color adjacent to the selected pixel (continuous range of approximate colors; hereinafter, the pixel group selected here is referred to as a pixel block), A pixel representing the body of the target animal is used.

  The recognizing unit 23 performs processing for determining the direction of the tail. In this process, as shown in FIG. 4, the recognizing unit 23 obtains the barycentric position W of the pixel group selected as the pixel representing the main body of the target animal from the image information to be processed (S1). Then, an imaginary circle having a plurality of radii r1, r2,... Rn is generated around the center of gravity (S2). The recognizing unit 23 selects a pixel that is an approximate color of the previous lowest pixel value on each generated virtual circle (P1, P2,...), And each pixel (Pi) selected in the process S2 and the process S1. A virtual line segment that connects the center of gravity obtained in step 1 is generated. The recognizing unit 23 obtains an angle range Q in which the generated line segment is most present among the predetermined angle ranges centered on the position of the center of gravity obtained in the process S1 (S3).

  The provisional determination unit 24 is a position i-th farthest from the center of gravity obtained in the process S1 of FIG. 4 among the pixels representing the main body of the target animal selected by the recognition unit 23 in the image information processed by the recognition unit 23. .. Are selected as candidates for the position of the head, and it is temporarily determined that there is a head of the target animal (FIG. 5). The determined pixel positions (N) are stored in the storage unit 12 in order of increasing distance from the barycentric position.

As illustrated in FIG. 5, the correction processing unit 25 uses the direction of the angle range obtained by the recognition unit 23 in step S <b> 5 of FIG. 4 as the tail direction (DT) in the image information processed by the recognition unit 23. Similarly, the head direction (DH) is obtained with the angle range on the opposite side of the center of gravity W obtained in step S1 of FIG. 4 as the head direction (DH). Then, the correction processing unit 25 sequentially starts from i = 1 in the direction of the head (DH) from the center of gravity W to the position of the i-th pixel stored in the storage unit 12 (the head of the target animal). Check if there is a position). If the correction processing unit 25 has the position (head position) of the i-th pixel stored in the storage unit 12 by the temporary determination unit 24 in the head direction (DH) obtained here, the temporary determination unit 25 24, assuming that the position of the i-th pixel stored in the storage unit 12 is the position of the head of the target animal, information indicating the position of the pixel (coordinate information of image information may be output) is output.
With this process, the tail direction is determined in real time without the need for enormous calculations corresponding to the way of bending and the angle of the tail as in the case of using the template matching method in determining the direction of the tail. The process can be executed.

  Further, the correction processing unit 25 increments i by 1 if there is no position (head position) of the i-th pixel stored in the storage unit 12 by the temporary determination unit 24 in the calculated head direction (DH). Thus, the process is repeated from the determination of whether or not the calculated i-th pixel position (head position) is stored in the storage unit 12 by the temporary determination unit 24 in the head direction (DH).

  The output unit 26 displays a mark (for example, a red circle) indicating that the head of the target animal is present at the position of the pixel represented by the information output from the correction processing unit 25. The output unit 26 outputs the pixel position represented by the information output from the correction processing unit 25 in the pixel group in the range of interest set by the setting unit 22, that is, output from the correction processing unit 25. When the position of the head specified by the information is included in the attention range, a predetermined marker is output. For example, the marker may be placed as a red square figure or the like displayed on a predetermined position (for example, the upper left corner) of the image information so as to be superimposed on the image information.

  Further, in the description so far of the present embodiment, the correction processing unit 25 is based on the information on the position of the head temporarily determined for the image information to be processed by the recognition unit 23 and the same image information that has been processed. It has been investigated whether or not the head direction estimated based on the obtained tail position matches, but the present embodiment is not limited to this.

  For example, in order to obtain the head direction to be compared with the information of the head position temporarily determined for the image information Pj that is recognized as the j-th processing target by the recognition unit 23 in the series of image information, The head direction estimated based on the image information Pj-1 that is the j-1th processing target, the head direction estimated based on the jth image information Pj image information Pj that is the processing target , And the head direction estimated based on the image information Pj + 1 that is the first processing target after j + 1 may be averaged (the arithmetic average of the angle values from the center of gravity may be calculated). do it). In this case, the correction processing unit 25 is a pixel in the direction of the calculated average head among the head position candidates tentatively determined for the image information Pj that is recognized as the j-th processing target by the recognition unit 23. Then, assuming that the pixel farthest from the center of gravity is the position of the head of the target animal, information indicating the position of the pixel is output.

  Thus, the image information before and after may be taken into consideration. In this example, one piece of image information is used before and after, but the number of pieces of image information estimated before and after the image information to be processed is determined experimentally. You can also.

  The image processing apparatus 1 according to the present embodiment has the above configuration and operates as follows. In the present embodiment, the image processing apparatus 1 that has received a series of image information images a target object by segmentation using points input by a user's instruction operation from the series of image information. Find out the range that is.

  Further, the image processing apparatus 1 specifies a range where the main body of the target animal exists in each image information based on a series of image information. The image processing apparatus 1 finds out the range of pixel values that are the pixel values of colors that approximate the pixel value that is included in the range in which the main body exists in each image information (continuous pixel block portion), and The range is recognized as the range where the main body of the target animal is imaged.

  The image processing apparatus 1 calculates the center of gravity of the range in which the main body is imaged, and tentatively determines a pixel farthest from the center of gravity as a pixel candidate whose head is imaged. The image processing apparatus 1 further specifies, as the tail direction, around the main body of the target animal, the center of the main body of the target animal, and the direction in which most pixels of a color similar to the pixels included in the main body of the target animal are found. Then, with respect to the direction of the tail, the opposite side across the center of gravity is recognized as the direction of the head.

  The image processing apparatus 1 checks whether or not the candidate of the pixel in which the head that has been previously determined is captured is included in the direction of the recognized head. The determination is made, and the process of determining whether or not the candidate pixel for which the temporarily determined head is imaged is included in the recognized head direction is continued.

  The image processing apparatus 1, when the recognized head direction is included in the recognized head direction, and the head is imaged at the position of the temporarily determined pixel candidate. to decide. The image processing apparatus 1 displays the marker M when the position where it is determined that the head has been imaged is at the peripheral edge of the target object found earlier, and the mouse is interested in the target object. It is informed that the head is directed to the target object (FIG. 6A). In addition, since the mouse has the property of depending on the wall surface, the mouse may lean on the target object (FIG. 6B). In this case, the marker is not displayed because the head of the target animal is not located at the periphery of the target object.

  In another aspect of the present embodiment, the position of the center of gravity of the target animal detected from each image information may be plotted, and an image representing the movement locus of the center of gravity of the target animal may be generated and output (FIG. 7 ( a)). In addition, on the movement locus, an image in which the range of the locus when the marker is displayed and the range where the marker is not distinguishable (for example, by changing the color) may be generated and output (FIG. 7B). )).

  According to the present embodiment, it is possible to reduce a processing load such as processing for determining the direction of the tail, and it is possible to perform a relatively quick process with a certain accuracy. For example, analysis in real time and intervention in animal behavior are possible. In addition, since no artificial judgment is required, it is possible to analyze the behavior of animals based on certain evaluation criteria.

DESCRIPTION OF SYMBOLS 1 Image processing apparatus, 2 Cameras, 11 Control part, 12 Storage part, 13 Operation part, 14 Display part, 15 Interface part, 21 Reception part, 22 Setting part, 23 Recognition part, 24 Temporary determination part, 25 Correction process part, 26 Output unit.

Claims (3)

A receiving means for receiving a series of image information in which a target animal having a tail and a main body and a target object are imaged;
Setting means for identifying a target object from the image information and setting a range of interest around the contour line of the identified target object;
The main body of the target animal is recognized from the image information, a point having a color different from the background is found in the vicinity of the main body recognized in the image information, and a line connecting the recognized main body and the found point is defined as a tail existence range. Recognition means to recognize as
Tentatively determining means for temporarily determining the position of the recognized body as the head of the target animal, and the position farthest from the center of gravity on the contour line of the body,
Correction processing means for providing a vector from the recognized center of gravity of the body to a line segment that is the recognized existence range of the tail for correction processing of the recognition position of the head temporarily determined by the temporary determination means;
Means for outputting the recognition position of the head after the correction processing;
An image processing apparatus. The image processing apparatus according to claim 1,
An image processing apparatus further comprising marker output means for outputting a predetermined marker when the output recognition position of the head is included in the attention range. Computer
A receiving means for receiving a series of image information in which a target animal having a tail and a main body and a target object are imaged;
Setting means for identifying a target object from the image information and setting a range of interest around the contour line of the identified target object;
The main body of the target animal is recognized from the image information, a point having a color different from the background is found in the vicinity of the main body recognized in the image information, and a line connecting the recognized main body and the found point is defined as a tail existence range. Recognition means to recognize as
Tentatively determining means for temporarily determining the position of the recognized body as the head of the target animal, and the position farthest from the center of gravity on the contour line of the body,
Correction processing means for providing a vector from the recognized center of gravity of the body to a line segment that is the recognized existence range of the tail for correction processing of the recognition position of the head temporarily determined by the temporary determination means;
Means for outputting the recognition position of the head after the correction processing;
Program to function as.

Images