JP6700769B2 - Image processing device, image processing method, and program - Google Patents

Description

  The present invention relates to an image processing device, an image processing method, and a program.

  There is known an object detection device that detects a human body or the like from an image. The object detection device performs detection in various situations. For example, the object detection device is used to measure the number of spectators in a stadium or the number of visitors to an amusement park. When measuring the number of people from an image, the entire image may be measured, but only a part of the image may be measured. In that case, it is necessary to set the detection area of the image. For example, Patent Document 1 discloses a method of setting a detection area that enables detection when the height is set to detect a person without considering the zoom ratio and the optical characteristics of the camera. ing.

JP2012-59196A

  However, the above-mentioned Patent Document 1 does not disclose a method of selecting/extracting an image suitable for setting a detection region among images of a plurality of frames in a moving image.

  An object of the present invention is to provide an image processing device, an image processing method and a program capable of extracting an image suitable for setting a detection area from an image file.

The image processing device of the present invention is a display control unit that displays a setting screen on which a file can be selected on a display unit, and a still image file in which the image file selected by the user for the setting screen includes only one image. In some cases, the image is extracted as an extracted image, and when the image file is a moving image file including a plurality of images, an extraction unit that extracts one image of the plurality of images as an extracted image, and the extraction unit In the extracted image thus extracted , a setting unit that sets a detection area in which an object detection process is executed according to a user operation; and, if the image file is the still image file, the detection set in the extracted image. When the detection process is performed on a region and the image file is the moving image file, the detection region set in the extracted image and the extracted image among the plurality of images included in the moving image file A region corresponding to the detection region in another different image, and an object detection unit that performs the detection process for each , the display control unit, when the detection region is set by the setting unit, It is characterized in that a numerical value indicating at least one of a coordinate, a width and a height of the detection area in the extracted image is displayed on the setting screen .

  According to the present invention, an image suitable for setting a detection area can be extracted from an image file.

It is a figure which shows the structural example of the image processing apparatus by this embodiment. It is a figure which shows the hardware structural example of the image processing apparatus by this embodiment. It is a figure which shows an example of the main window of an image processing apparatus. It is a figure which shows an example of a detection area setting window. 6 is a flowchart showing an image processing method according to the present embodiment.

  FIG. 1 is a diagram showing an example of the configuration of an image processing apparatus according to an embodiment of the present invention. The image processing apparatus 101 includes an object detection unit 102, a file input unit 103, a file output unit 104, a display unit 105, a detection area setting unit 106, and an operation input unit 109. The detection area setting unit 106 includes an image extraction unit 107 and a setting unit 108. The file input unit 103 inputs the image file 111.

  FIG. 2 is a diagram illustrating an example of the hardware configuration of the image processing apparatus 101. The image processing apparatus 101 includes a CPU 201, a ROM 202, a RAM 203, a hard disk drive (HDD) 204, a disk drive 205, a network interface 206, an input device 207, and a display device 208. The network interface 206 is connected to the network 211. The CPU 201 is a control device that integrally controls the image processing device 101. The ROM 202 stores a program for the CPU 201 to control the image processing apparatus 101. A secondary storage device may be provided instead of the ROM 202. The RAM 203 is a memory for the CPU 201 to expand the program read from the ROM 202 and execute processing. The RAM 203 is also used as a temporary storage memory and as a storage area for temporarily storing data to be processed. The HDD 204 stores the image file 111. The CPU 201 inputs the image file 111 from the HDD 204. When the CPU 201 inputs the image file 111 from the disk drive 205 or the network interface 206, the HDD 204 is not always necessary. The disk drive 205 reads the image file 111 in the CD, DVD, Blu-ray disk, or flexible disk, and the CPU 201 inputs the image file 111 via the disk drive 205. When the CPU 201 inputs the image file 111 from the HDD 204 or the network interface 206, the disk drive 205 is not always necessary. The network interface 206 is a circuit that performs communication via the network 211. The CPU 201 inputs the image file 111 via the network 211. When the CPU 201 inputs the image file 111 from the HDD 204 or the disk drive 205, the network interface 206 is not always necessary. The display device 208 is a device such as a display that displays images and area frames. The input device 207 includes a keyboard for inputting numerical values, a pointing device for instructing the display position of the display area of the display device 208, a mouse, a touch panel, and the like.

  As described above, the hardware configuration of the image processing apparatus 101 has the same hardware configuration elements as those installed in a general PC (personal computer). Therefore, various functions realized by the image processing apparatus 101 can be implemented as software that operates on a PC. The image processing apparatus 101 can realize the functions of FIG. 1 and the flowchart of FIG. 5 described later by the CPU 201 executing the program.

  FIG. 3 is a diagram showing an example of the main window 301 displayed on the display device 208 by the display unit 105. The main window 301 has an input file selection button 302, a detection area setting window start button 303, a detection area display area 304, and a detection start button 305.

  FIG. 4 is a diagram showing an example of the detection area setting window 401 displayed on the display device 208 by the display unit 105. The detection area setting window 401 has an extracted image 402, a detection area setting frame 403, an OK button 404, and a cancel button 405.

  FIG. 5 is a flowchart showing an example of the image processing method of the image processing apparatus 101. When the image processing apparatus 101 is activated, the display unit 105 displays the main window 301 of FIG. 3 on the display device 208. In step S1001, when the selection of the input file selection button 302 by the user is detected, the display unit 105 displays the file selection dialog for the selection by the user. The file input unit 103 inputs the image file 111 selected by the user via the HDD 204, the disk drive 205 or the network interface 206. The type of the image file 111 may be either a still image file or a moving image file. The input file selection button 302 and the file selection dialog are examples of the file input unit 103.

  Next, in step S1002, the display unit 105 displays the detection area setting window 401 of FIG. 4 according to the selection of the detection area setting window activation button 303 by the user. The detection area setting window 401 is a window having the function of the detection area setting unit 106.

  Next, in step S1003, the image extraction unit 107 extracts the extracted image 402 from the image file 111 selected by the user in step S1001. The method of extracting the extracted image 402 will be described below. When the image file 111 is a still image file including only one image, the image extracting unit 107 extracts the one image from the image file 111 as the extracted image 402. When the image file 111 is a moving image file composed of images of a plurality of frames, the image extracting unit 107 extracts the image of the first frame of the image file (moving image file) 111 as the extracted image 402.

  Next, in step S1004, the display unit 105 displays the extracted image 402 in the detection area setting window 401 as shown in FIG. If the extracted image 402 is displayed in the same size as the image, the extracted image 402 may not fit in the detection area setting window 401. For example, when the image display area of the detection area setting window 401 has a width of 1600 and a height of 900, and the image size of the extracted image 402 is a width of 3200 and a height of 1800, the display unit 105 divides the extracted image 402 into two minutes. Resize to 1 and display. The display unit 105 displays an image having a predetermined magnification with respect to the extracted image 402.

  Next, in step S1005, the setting unit 108 sets the detection area setting frame 403 corresponding to the detection area according to the user's operation of the frame on the extracted image 402. The operation input unit 109 inputs the operation information of the input device 207 by the user, and the setting unit 108 sets the detection area setting frame 403 according to the operation information. A method of setting the detection area setting frame 403 will be described below. The extracted image 402 in FIG. 4 is rectangular. A rectangular detection area setting frame 403 is displayed on the extracted image 402. To draw the detection area setting frame 403, the user drags the mouse on the extracted image 402. The point where the user has moved the mouse down on the extracted image 402 becomes one vertex, the point which the user moved up on the extracted image 402 becomes another vertex, and both vertices are opposite vertices. By extending a line parallel to each side of the extracted image 402 from both vertices, two intersections are created. The rectangle connecting the above four points becomes the detection area setting frame 403. The display unit 105 displays the detection area setting frame 403 on the extracted image 402 having a predetermined magnification according to the user's operation.

  When the selection of the OK button 404 by the user is detected, the setting unit 108 closes the detection area setting window 401 and sets the XY coordinates, width, and height of the upper left apex of the detection area setting frame 403 as the detection area. The display unit 105 displays the XY coordinates, width, and height of the detection area display area 304. In this way, the setting unit 108 sets the detection area corresponding to the detection area setting frame 403 in the extracted image 402. However, when the extracted image 402 is resized and displayed in step S1004, the setting unit 108 sets the detection area by multiplying the XY coordinates, the width, and the height by the reciprocal of a predetermined magnification of the resize. The display unit 105 displays the set XY coordinates, width and height in the detection area display area 304. For example, in the case of resizing to half as in the above example, the setting unit 108 sets the detection area by multiplying the XY coordinates, the width, and the height by the reciprocal 2. The size of the detection area is the size of the reciprocal of a predetermined magnification with respect to the size of the detection area setting frame 403.

  When detecting the selection of the cancel button 405 by the user, the setting unit 108 closes the detection area setting window 401 and does not set a new detection area such as XY coordinates. The method of setting the detection area setting frame 403 is not limited to the above method. For example, any of various methods such as a method of bringing in a frame prepared in advance and a method of moving the sides of the frame in parallel may be used. Further, the shape of the detection area setting frame 403 may be a shape other than a rectangle, as long as it is a closed space shape that can be detected by the object detection unit 102.

  Next, in step S1006, the object detection unit 102 starts detection of a person (object) according to the selection of the detection start button 305 by the user. The area detected by the object detection unit 102 is inside the rectangle of the detection area setting frame 403 defined by the X coordinate, Y coordinate, width, and height stored in the setting unit 108. The images to be detected by the object detection unit 102 are all the images in the image file 111. The object detection unit 102 detects a person (object) in the detection area corresponding to the detection area setting frame 403 in one or a plurality of images in the image file 111. In the present embodiment, the object to be detected by the object detection unit 102 is all the images in the image file 111, but not all the images need to be detected. For example, the object to be detected by the object detection unit 102 is any one of the image file 111 such as the first image, the last image, a predetermined image, an image with a predetermined frame interval, an image with a predetermined reproduction time interval in the image file 111. Any image can be used as long as it is an image.

  Next, in step S1007, the file output unit 104 saves the result file of the person detection by the object detection unit 102 in a predetermined output folder. The above result file has an image in which a detected person in the image file 111 is marked, and a list in which the detected number of persons for each image to be detected is described as metadata. In step S1006, the object detection unit 102 generates this result file by detecting a person. Thus, the image processing apparatus 101 ends the process.

  The process of step S1003 is not limited to the above process. For example, in step S1003, when the image file 111 is a moving image file, the image extracting unit 107 may extract the image of the last frame of the image file (moving image file) 111 as the extracted image 402.

  Further, in step S1003, when the image file 111 is a moving image file, the image extracting unit 107 extracts, as the extracted image 402, an image of a frame after a predetermined reproduction time has elapsed from the first frame of the image file (moving image file) 111. You may.

  Further, in step S1003, when the image file 111 is a moving image file, the image extracting unit 107 may extract an image of a predetermined number of frames from the first frame of the image file (moving image file) 111 as the extracted image 402.

  Further, in step S1003, when the image file 111 is a moving image file, the image extracting unit 107 extracts an image of a frame after a predetermined percentage of the entire reproduction time has elapsed from the first frame of the image file (moving image file) 111. May be extracted as

  Further, in step S1003, when the image file 111 is a moving image file, the image extracting unit 107 sets, as the extracted image 402, an image of a frame number that is a predetermined ratio of the total number of frames from the first frame of the image file (moving image file) 111. You may extract.

  Further, in step S1003, when the image file 111 is a moving image file, the image extracting unit 107 extracts the image of the first frame of the plurality of frames associated with the event in the image file (moving image file) 111. You may extract as 402. Here, the event is, for example, an event indicating that a moving body has been detected. By selecting the frame in which the moving body is detected, the position of the moving body in the image can be intuitively grasped, which facilitates the setting of the detection area. The image extraction unit 107 can extract the image of the frame associated with this event as the extracted image 402.

  Further, in step S1003, when the image file 111 is a moving image file, the image extracting unit 107 extracts the image of the final frame of the plurality of frames associated with the event in the image file (moving image file) 111. You may extract as 402.

  Further, in step S1003, the image extraction unit 107 extracts an image of a predetermined number of frames from the first frame of the plurality of frames associated with the event in the image file (moving image file) 111 in the case of a moving image file. You may extract as 402.

  According to the present embodiment, it is possible to extract the extracted image 402 having a condition suitable for setting the detection area setting frame 403 from the image file 111 to be detected. When the image file 111 is a moving image file, the image processing apparatus 101 extracts an extracted image 402 suitable for setting the detection area setting frame 403 from images of a plurality of frames in the moving image of the image file 111. You can

  The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program. It can also be realized by the processing. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

  It should be noted that each of the above-described embodiments is merely an example of an embodiment for carrying out the present invention, and the technical scope of the present invention should not be limitedly interpreted by these. That is, the present invention can be implemented in various forms without departing from the technical idea or the main features thereof.

101 image processing device, 102 object detection unit, 103 file input unit, 104 file output unit, 105 display unit, 106 detection region setting unit, 107 image extraction unit, 108 setting unit, 111 image file

Claims (19)

A display control unit that displays a setting screen on which an image file can be selected on the display unit ,
When the image file selected by the user on the setting screen is a still image file including only one image, the image is extracted as an extracted image, and when the image file is a moving image file including a plurality of images, An extraction unit that extracts one of the plurality of images as an extracted image ,
In the extracted image extracted by the extraction unit , according to a user operation, a setting unit that sets a detection region in which the object detection process is executed ,
When the image file is the still image file, the detection processing is executed on the detection area set in the extracted image, and when the image file is the moving image file, the detection set in the extracted image is performed. detection area, and, chromatic and object detecting unit area corresponding to the detection regions in different another image with the extracted image of the plurality of images included in the moving image file, for each of performing the detection process Then
When the detection region is set by the setting unit, the display control unit causes the setting screen to display a numerical value indicating at least one of the coordinate, the width, and the height of the detection region in the extracted image. An image processing device characterized by. The extraction section, the case where the image file is a moving picture file, an image processing apparatus according to claim 1, wherein extracting the image of the first frame of the moving image file. The extraction section, the case where the image file is a moving picture file, an image processing apparatus according to claim 1, wherein extracting the image of the last frame of the moving image file. The extraction section, the case where the image file is a moving picture file, an image processing apparatus according to claim 1, wherein extracting the image of the frame from the first frame after a lapse of a predetermined reproduction time of the moving image file. The extraction section, the case where the image file is a moving picture file, an image processing apparatus according to claim 1, wherein extracting the image of a predetermined number of frames from the beginning frame of the moving image file. The extraction section, the case where the image file is a moving picture file, an image processing according to claim 1, wherein extracting the image of a frame after a predetermined ratio lapse of total playing time from the head frame of the moving image file apparatus. The extraction section, the case where the image file is a moving picture file, an image processing apparatus according to claim 1, wherein extracting the image of a number of frames th predetermined ratio of the entire number of frames from the first frame of the moving image file . The extraction unit, when the image file is a moving picture file, an image processing apparatus according to claim 1, wherein extracting the image of a frame event in the moving image file is associated. The extraction unit, when the image file is a moving picture file, an image of claim 1, wherein extracting the image of the first frame in the plurality of frames events in the moving image file is associated with Processing equipment. The extraction unit, when the image file is a moving picture file, an image of claim 1, wherein extracting the image of the last frame in the plurality of frames events in the moving image file is associated with Processing equipment. When the image file is the moving image file , the extracting unit extracts an image of a predetermined number of frames from a first frame of a plurality of frames associated with an event in the moving image file. 1. The image processing device according to 1 . The object detection unit, when the image file is the moving image file, the detection region set in the extracted image, and all images other than the extracted image among the plurality of images included in the moving image file. The image processing apparatus according to any one of claims 1 to 11, wherein the detection processing is performed on an area corresponding to the detection area in each of the above. The display control unit causes the display unit to display the extracted image extracted by the extraction unit,
The setting unit, based on the operation to the extracted image displayed on the display unit to specify the detection area, any one of claims 1 to 12, characterized in that to set the detection area in the extracted image The image processing apparatus according to item 1. A first display control step of displaying a setting screen capable of selecting an image file on the display section;
When the image file selected by the user on the setting screen is a still image file including only one image, the image is extracted as an extracted image, and when the image file is a moving image file including a plurality of images, An extraction step of extracting one of the plurality of images as an extracted image ,
In the extracted image extracted in the extracting step, a setting step of setting a detection area in which a detection process of an object is executed according to a user operation ,
When the detection area is set in the setting step, a second display control step of causing the setting screen to display a numerical value indicating at least one of coordinates, width, and height in the extracted image of the detection area,
When the image file is the still image file, the detection processing is executed on the detection area set in the extracted image, and when the image file is the moving image file, the detection set in the extracted image is performed. An object detection step of performing the detection process on each of a detection region and a region corresponding to the detection region in another image different from the extracted image among the plurality of images included in the moving image file. An image processing method characterized by the above. 15. The image processing method according to claim 14, wherein in the extracting step, when the image file is the moving image file, the image of the first frame of the moving image file is extracted. 15. The image processing method according to claim 14, wherein in the extracting step, when the image file is the moving image file, an image of a frame associated with an event in the moving image file is extracted. The image according to claim 14, wherein, in the extracting step, when the image file is the moving image file, an image of a first frame among a plurality of frames associated with an event in the moving image file is extracted. Processing method. Further comprising a third display control step of displaying the extracted image extracted in the extracting step on the display unit,
18. The detection area is set in the extracted image based on an operation of designating the detection area in the extracted image displayed on the display unit in the setting step. The image processing method according to item 1. A program for causing a computer to function as each unit of the image processing apparatus according to claim 1.

Images