JP7459151B2 - Information processing device, information processing system, information processing method, and program - Google Patents

Description

The present invention relates to a technique for detecting an object from an image.

In recent years, image analysis that uses images captured by imaging devices such as surveillance cameras to detect and track objects, estimate attributes, etc., and estimation of the number of objects using the results of such image analysis has become popular in various scenes. It is being carried out in In object detection, for example, the position and size of the object to be detected, the attributes of the object, the reliability of the object, etc. are output. In object detection, multiple detection results may occur for one object. This poses a problem in that it leads to lower reliability of detection results and lower reliability of statistical data. To address such issues, Patent Document 1 discloses that based on the position and size of the detection frame output for each detection processing frame, a detection frame whose degree of overlap with other detection frames is equal to or greater than a threshold value is A technique has been disclosed that integrates detection frames according to their properties and outputs them as a final object detection frame.

Japanese Patent Application Publication No. 2018-180945

In the technology disclosed in Patent Document 1, even if the reliability of the detection result is low, the detection results are used to integrate the detection frames and output as the final object detection frame. Reliability may be reduced. The present invention has been made in view of the above circumstances, and an object of the present invention is to finally output a more appropriate detection result in detecting an object using an image.

An information processing apparatus according to the present invention includes a detection means for detecting, from a first image, an object corresponding to a first attribute and a second attribute different from the first attribute, and the detection means A detection area detected from the first image by a first detection area in which an object corresponding to the first attribute is detected, and a second detection area in which an object corresponding to the second attribute is detected. a determination means for determining whether the first detection region and the second detection region overlap, and a determining means for determining whether the first detection region and the second detection region overlap, As a result of the detection means performing object detection on a second image obtained by enlarging an area including an area where the first detection area and the second detection area overlap, the second image When an object corresponding to the first attribute or the second attribute is detected, the object detection by the detection means for the first image is performed based on the detection result of the object detection for the second image. The present invention is characterized by comprising a correction means for correcting the detection result.

According to the present invention, in detecting an object using an image, it is possible to finally output a more appropriate detection result.

FIG. 1 is a diagram illustrating a configuration example of an information processing device according to a first embodiment. 1 is a diagram illustrating an example of a functional configuration of an information processing device according to a first embodiment; FIG. 7 is a flowchart illustrating object detection processing according to the first embodiment. FIG. 3 is a diagram illustrating object detection processing according to the first embodiment. FIG. 3 is a diagram illustrating a configuration example of an information processing device according to a second embodiment. FIG. 7 is a diagram illustrating an example of a functional configuration of an information processing device according to a second embodiment. It is a flow chart explaining object detection processing by a 2nd embodiment. 13A to 13C are diagrams illustrating an object detection process according to the second embodiment. FIG. 7 is a diagram illustrating an example of a functional configuration of an information processing device according to a third embodiment. It is a flow chart explaining object detection processing by a 3rd embodiment. 13A to 13C are diagrams illustrating an object detection process according to a third embodiment. FIG. 13 is a diagram illustrating an example of a functional configuration of an information processing device according to a fourth embodiment. It is a flow chart explaining object detection processing by a 4th embodiment. It is a figure explaining object detection processing by a 4th embodiment.

Embodiments of the present invention will be described below based on the drawings.

[First embodiment]
FIG. 1 is a block diagram showing a configuration example of an information processing apparatus 100 according to this embodiment. The information processing device 100 in this embodiment has an object detection function that detects an object to be detected from an image captured by an imaging device such as a surveillance camera. In the following, a case will be described in which a person's face is detected as an example, but the present invention is not limited to this, and can be applied to any system that analyzes an image and detects a predetermined object.

The information processing device 100 according to this embodiment includes a CPU 101, a memory 102, a communication interface (I/F) section 103, a display section 104, an input section 105, and a storage section 106. The CPU 101, memory 102, communication I/F section 103, display section 104, input section 105, and storage section 106 are communicably connected via a system bus. Note that the information processing device 100 according to the present embodiment may further include configurations other than this.

The CPU (Central Processing Unit) 101 is responsible for the overall control of the information processing device 100. The CPU 101 controls the operation of each functional unit connected via, for example, a system bus. The memory 102 stores data, programs, etc. that the CPU 101 uses for processing. The memory 102 also functions as the main memory, work area, etc. of the CPU 101. The CPU 101 executes processing based on the programs stored in the memory 102, thereby realizing the functional configuration of the information processing device 100 shown in FIG. 2, which will be described later, and the processing of the flowchart shown in FIG. 3, which will be described later.

The communication I/F unit 103 is an interface that connects the information processing device 100 to a network. The display unit 104 includes a display member such as a liquid crystal display, and displays the results of processing by the CPU 101 and the like. The input unit 105 has an operation member such as a mouse or a button, and inputs a user's operation to the information processing apparatus 100. The storage unit 106 stores, for example, various data necessary when the CPU 101 performs processing related to a program. Further, the storage unit 106 stores, for example, various data obtained by the CPU 101 performing processing related to a program. Note that data, programs, etc. used by the CPU 101 for processing may be stored in the storage unit 106.

FIG. 2 is a block diagram showing an example of the functional configuration of the information processing device 100. The information processing device 100 includes an image acquisition section 201 , an object detection section 202 , an overlap determination section 203 , an image extraction section 204 , a result modification section 205 , a result output section 206 , and a storage section 207 .

The image acquisition unit 201 acquires an image to be subjected to object detection. In this embodiment, an image to be subjected to object detection is acquired from the outside through the communication I/F unit 103. Hereinafter, the data of the image that is the object of object detection, which is acquired by the image acquisition unit 201, will also be simply referred to as an "input image." In the following description, the input image is, for example, a 1080×720 pixel RGB image with a horizontal direction (horizontal direction) width of 1080 pixels and a vertical direction (vertical direction) height of 720 pixels. Note that the input image is not limited to an RGB image of 1080 x 720 pixels, and any image can be used as the input image, and for example, the width in the horizontal direction and the height in the vertical direction may be different. .

The object detection unit 202 performs object detection based on a plurality of attributes (classes) from an image. In this embodiment, the object detection unit 202 detects a person's face from the image acquired by the image acquisition unit 201. Further, the object detection unit 202 outputs a detection result using a machine learning model that has been trained to be able to detect a "face wearing glasses" and a "face not wearing glasses" included in an image. Detection of "faces wearing glasses" and "faces not wearing glasses" can be realized by applying the technology described in Document 1 below, for example.
(Reference 1) J. Redmon, A. Farhadi, “YOLO9000:Better
Faster Stronger”, Computer Vision and Pa
ttern Recognition (CVPR) 2016.

Here, it is assumed that the detection results output by the object detection unit 202 are the position and size of the detected face, the attribute (class) of the face, and the reliability of detection. The position and size of the face are output, for example, as coordinates defining a rectangular frame surrounding the face (for example, upper left coordinates (x1, y1) and lower right coordinates (x2, y2) of the rectangle). Further, the attribute of the face indicates whether the face is wearing glasses or not. Furthermore, the detection reliability is output as a real number between 0 and 1, with 0 being the lowest reliability and 1 being the highest reliability, for example. Below, the rectangular frame surrounding the face, the attributes of the face, and the detection reliability are also simply referred to as "detection frame," "attribute," and "reliability." Note that the method for outputting the detection results is not limited to the example described above, and it is sufficient that the position and size of the detected face, the attributes of the face, and the reliability of detection can be recognized.

The overlap determination unit 203 determines whether or not the detection results obtained by the object detection unit 202 overlap with the detection results of different attributes. The overlap determination unit 203 sets any two detection frames among all the detection results obtained by the object detection unit 202, and calculates the overlap rate of the detection frames for each set. The overlap determination unit 203 determines that there is an overlap if the calculated overlap rate is equal to or greater than a threshold, that is, if there is a set of detection frames whose detection frame areas overlap by a predetermined ratio or more, and outputs the set of detection results. In this embodiment, the overlap rate is calculated using IoU (Intersection over Union), and the threshold is set to 0.5 as an example. In other words, if the quotient when the common part of the areas of the two detection frames is divided by the union of the areas is 0.5 or more, the overlap determination unit 203 determines that there is an overlap. If there is no set of detection frames that overlap by the threshold or more, the overlap determination unit 203 determines that there is no overlap.

Based on the judgment result by the overlap judgment unit 203, the image extraction unit 204 extracts a specified area from the input image using the input image and a set of detection results judged to have overlap by the overlap judgment unit 203. The image extraction unit 204 outputs an image of the extracted area (hereinafter also referred to as the "extracted image") and the upper left coordinates of the area to be extracted relative to the input image. In this embodiment, the upper left coordinates of the area to be extracted are (x1-((x2-x1)/10), y1-((y2-y1)/10)), the lower right coordinates are (x2+((x2-x1)/10), y2+((y2-y1)/10)), and a margin is taken for the union of the detection frames to extract. The image extraction unit 204 outputs the extracted image and the upper left coordinates of the area to be extracted (x1-((x2-x1)/10), y1-((y2-y1)/10)). Here, (x1, y1) is the upper left coordinate of the rectangular area that contains the union of the two detection frames, and (x2, y2) is the lower right coordinate of the rectangular area that contains the union of the two detection frames. Note that any area to be extracted that exceeds the range of the input image will be filled in with white, for example.

The result modification unit 205 reflects the detection result for the extracted image by the object detection unit 202 on the detection result for the input image according to the number of detection results for the extracted image. The result modification unit 205 first deletes the detection result used when calculating the extracted region from among the detection results of the input image. Next, if the number of detection results for the extracted image is one, the result modification unit 205 replaces it with the detection result obtained for the extracted image. In addition, when the number of detection results for the extracted image is two or more, the result correction unit 205 determines that the detection frame is a rectangular area that includes the union of the two detection frames, and the face attribute is "Unknown for wearing glasses." ”, a detection result with a detection reliability of 1 is generated and replaced.

The result output unit 206 outputs an image obtained by superimposing the detection result on the input image. In this embodiment, the result output unit 206 outputs, for example, an image in which a detection frame according to the attribute is superimposed on the input image based on the detection result. The storage unit 207 stores data used in processing in each of the functional units 201 to 206 of the information processing device 100, data obtained as a processing result, and the like.

Next, processing performed by the information processing apparatus 100 will be described with reference to FIGS. 3 and 4. FIG. 3 is a flowchart illustrating object detection processing according to the first embodiment. FIG. 4 is a diagram illustrating object detection processing according to the first embodiment.

In step S301, the image acquisition unit 201 acquires an input image (an image to be subjected to object detection). An example of the input image 410 is shown in FIG. 4(a). In this embodiment, the input image 410 is assumed to be a 1080×720 pixel image as described above.

In step S302, the object detection unit 202 performs face detection processing to detect the face of the person to be detected using the input image, and distinguishes a "face wearing glasses" and a "face without glasses" from the input image. To detect. FIG. 4B shows an example of the detection result of the face detection process on the input image, and FIG. 4C shows an example of an image obtained by superimposing the detection result on the input image. In the example shown in FIG. 4(b), four detection results A to D are obtained, and each detects the upper left coordinates (x1, y1) and lower right coordinates (x2, y2) of the detection frame, and the attribute (" "wearing glasses" or "not wearing glasses") and the reliability are output. Further, in the example shown in FIG. 4(c), detection frames 411 to 414 corresponding to detection results A to D are displayed superimposed on the input image 410. In this example, detection frames 411 and 414 corresponding to detection results A and D detected as faces wearing glasses are displayed as rectangular frames with broken lines, and detection frames 411 and 414 corresponding to detection results B and C detected as faces not wearing glasses are displayed. Detection frames 412 and 413 are displayed as solid rectangular frames.

In step S303, the overlap determination unit 203 calculates the overlap rate of detection frames for each combination of detection results for the input image, using any two detection results among the detection results for the input image as a set. In this example, the upper left coordinates of the detection frame of detection result A are (20, 200), and the lower right coordinates are (320, 500). Further, the upper left coordinates of the detection frame of detection result B are (40, 210), and the lower right coordinates are (340, 510). Therefore, the overlap rate of the detection frames of detection result A and detection result B is
IoU (A, B) = ((320-40) x (500-210)) ÷ (300 x 300 + 300 x 300 - ((320-40) x (500-210))) ≒ 0.82
becomes. In other combinations, the detection frame overlap rate is 0.

In step S304, the overlap determination unit 203 determines whether there is a combination of detection results for which the overlap rate calculated in step S303 is equal to or greater than a threshold value. If the overlap determination unit 203 determines that there is a combination of detection results in which the overlap rate of the detection frames is equal to or greater than the threshold (YES in step S304), the overlap determination unit 203 outputs the combination of detection results in which the overlap ratio is equal to or greater than the threshold; The process moves to step S305. On the other hand, if the overlap determination unit 203 determines that there is no combination of detection results in which the overlap rate of the detection frames is equal to or greater than the threshold (NO in step S304), the process proceeds to step S309. In the example of this embodiment, the threshold value of the overlap rate is assumed to be 0.5. The overlap determining unit 203 moves to step S305 if there is a set of detection results for which the overlap rate calculated in step S303 is 0.5 or more, and moves to step S309 if there is not. In this example, since the overlap rate of the detection frames of detection result A and detection result B is 0.5 or more, the overlap determination unit 203 outputs the combination (A, B) with an overlap rate of 0.5 or more. Then, the process moves to step S305.

In step S305, the image extraction unit 204 extracts the specified region from the input image using the set of the input image and the detection result output in step S304, and extracts the specified region from the input image and the upper left corner of the region to be extracted with respect to the input image. Output the coordinates of. Specifically, first, the image extraction unit 204 calculates a circumscribing rectangle for the union of two detection frames from a set of detection results. In this example, a circumscribed rectangle is calculated for the combination (A, B). As shown in FIG. 4(b), the upper left coordinates of the detection frame of detection result A are (20, 200), the lower right coordinates are (320, 500), and the upper left coordinates of the detection frame of detection result B are ( 40,210), and the lower right coordinates are (340,510). Therefore, in the circumscribed rectangle of the combination (A, B), the upper left coordinates are (20, 200) and the lower right coordinates are (340, 510). Next, the image extraction unit 204 calculates the upper left coordinate and lower right coordinate of the area to be extracted based on the calculated circumscribed rectangle. In this example, the circumscribed rectangle of the combination (A, B) has upper left coordinates (20,200) and lower right coordinates (340,510). Therefore, the upper left coordinates of the area to be extracted are (20-((340-20)/10), (200-((510-200)/10) = (-12,169). Also, the area to be extracted is The lower right coordinate of is (340+((340-20)/10),510+((510-200)/10))=(372,541).The upper left coordinate of the area to be extracted is (-12,169 ), and the lower right coordinates are (372, 541), so the extracted image has a horizontal width of 384 pixels and a vertical height of 372 pixels. Based on this, the extracted image 420 An example of this is shown in FIG. 4(d).

In step S306, the object detection unit 202 performs face detection processing using the extracted image extracted in step S305, and detects a "face wearing glasses" and a "face not wearing glasses" from the extracted image. FIG. 4E shows an example of the detection result of the face detection process on the extracted image, and FIG. 4F shows an example of an image in which the detection result is superimposed on the extracted image. In the example shown in FIG. 4E, a detection result E is obtained, and the upper left coordinates (x1, y1) and lower right coordinates (x2, y2) of the detection frame, attributes, and reliability are output. In the example shown in FIG. 4F, a detection frame 421 corresponding to the detection result E detected as a face wearing glasses is displayed as a broken rectangular frame superimposed on the extracted image 420.

In step S307, the result modification unit 205 reflects the detection result for the extracted image obtained in step S306 on the detection result of the input image. In this example, since there is one detection result (here, detection result E) for the extracted image, the result correction unit 205 deletes the detection result A and detection result B associated with the extracted image, and Replace with result E. At this time, the coordinates of the detection frame of the detection result E are converted into coordinates in the input image with the upper left coordinates (-12, 169) of the area to be extracted in the input image as a reference. Since the upper left coordinates of the detection frame of the detection result E in the extracted image are (42, 36) and the lower right coordinates are (342, 336), the upper left coordinates of the detection frame of the detection result E in the input image are (30, 205), and the lower right coordinates are (330, 505). An example of the final result of the input image including the converted result is shown in FIG. 4(g).

In step S308, the result output unit 206 determines whether processing has been completed for all combinations of detection results in which the detection frame overlap rate is equal to or greater than a threshold value. If the result output unit 206 determines that processing has been completed for all combinations of detection results for which the overlap rate is equal to or greater than the threshold (YES in step S308), the process proceeds to step S309. On the other hand, if the result output unit 206 determines that there is an unprocessed combination among the combinations of detection results for which the overlap rate is equal to or higher than the threshold (NO in step S308), the process proceeds to step S305, and steps are performed for the unprocessed combinations. Processing from S305 onwards is executed.

In step S309, the result output unit 206 outputs an image in which the final detection result for the input image as shown in FIG. 4(g) is superimposed on the input image, ends the process, and starts processing the next input image. Transition. An example of an image in which the final detection result for the input image is superimposed on the input image is shown in FIG. 4(h). In the example shown in FIG. 4(h), detection frames 413, 414, and 421 corresponding to detection results C, D, and E are displayed superimposed on the input image 410. In this example, detection frames 414 and 421 corresponding to detection results D and E detected as faces wearing glasses are displayed as rectangular frames with broken lines, and detection frames 414 and 421 corresponding to detection results C detected as faces not wearing glasses are displayed. A frame 413 is displayed as a solid rectangular frame.

According to the first embodiment, in object detection for an input image, when detection results of different attributes overlap, object detection is performed for the extracted image in which the overlapped area is extracted, and The detection results of the input image are corrected using the detection results. As a result, it is possible to finally output a more appropriate detection result as the object detection result for the input image.

(Modified example)
Note that in the object detection unit 202, the type of object to be detected and the size of the image on which object detection is performed may be arbitrary. Moreover, the object detection unit 202 is not limited to the technique disclosed in Document 1, and can apply various techniques as long as the technique can detect the object to be detected. Furthermore, in the case where there are three or more types of detection targets in the overlap determination unit 203, the above-described overlap determination method can be applied to a combination of any two types of detection results.

When there are two or more detection results for the extracted image, the result correction unit 205 is not limited to the above-mentioned processing, and may perform other processing such as the following. For example, the face attributes of the detection result with the highest detection reliability may be used for the face attributes, or the detection result itself (detection frame, attributes, and reliability) may be replaced with the detection result with the highest detection reliability. In this case, the detection result with the highest reliability may be selected from the detection results for the input image and the detection results for the extracted image.

Furthermore, the region extracted by the image extraction unit 204 may be any rectangular region that includes the union of the detection frames. For example, the margin for the circumscribing rectangle for the union of the detection frames may be set to 0, or a predetermined margin may be set based on the specifications of the object detection unit 202. For example, if the ratio of the width of the object that maximizes the detection accuracy of the object detection unit 202 to the width of the input image is 50%, the margin may be set so that the width of the rectangular region that includes the union of the detection frames is 50% of the width of the input image. Furthermore, the image extraction unit 204 may not only extract a specified region from the input image, but may also flip the image from left to right with respect to the extracted image, shift the image in the x or y direction, or change pixel values such as brightness and hue.

Furthermore, when an overlap occurs in the detection results of the input images, the result correction unit 205 directly generates the detection result without performing the second detection (that is, skips the processing of steps S305 and S306), and the overlap occurs. It may be replaced with the detected result. For example, the new detection frame is a rectangular area that includes the union of the overlapping detection frames, the face attribute is "unknown glasses worn" indicating that the result is unknown, and a detection result is generated in which the detection reliability is set to 1. Then, the detection result may be replaced with a detection result in which an overlap occurs. Furthermore, if the detection result for the extracted image is empty, the result modification unit 205 may similarly generate a detection result directly to replace the detection result with overlap.

[Second embodiment]
In the first embodiment described above, the object detection process is performed by one information processing apparatus, but in the second embodiment, a case will be described in which the object detection process is performed by a plurality of information processing apparatuses. In the following description of the second embodiment, descriptions of points similar to those of the first embodiment will be omitted.

FIG. 5 is a block diagram showing a configuration example of an information processing system including the information processing device 100 and the information processing device 500 according to this embodiment. In FIG. 5, components having the same functions as those shown in FIG. The information processing device 100 and the information processing device 500 in this embodiment have an object detection function that detects an object to be detected from an image captured by an imaging device such as a surveillance camera. In the following, a case will be described in which a person's face is detected as an example, but the present invention is not limited to this, and can be applied to any system that analyzes an image and detects a predetermined object.

Information processing device 100 and information processing device 500 are connected through network 510. The information processing device 500 includes a CPU 501, a memory 502, a communication I/F section 503, and a storage section 504. The CPU 501, memory 502, communication I/F section 503, and storage section 504 are communicably connected via a system bus. Note that the information processing device 500 according to the present embodiment may further include configurations other than this. The information processing device 100 is an example of a first information processing device, and the information processing device 500 is an example of a second information processing device. Furthermore, the communication I/F section 103 is an example of a first communication means, and the communication I/F section 503 is an example of a second communication means.

The CPU 501 is responsible for the overall control of the information processing device 500. The CPU 501 controls the operation of each functional unit connected via, for example, a system bus. The memory 502 stores data, programs, etc. that the CPU 501 uses for processing. The memory 502 also functions as the main memory, work area, etc. of the CPU 501. The CPU 501 executes processing based on the programs stored in the memory 502, thereby realizing a part of the functional configuration of the information processing device 500 shown in FIG. 6, which will be described later, and the processing of the flowchart shown in FIG. 7, which will be described later.

Communication I/F unit 503 is an interface that connects information processing device 500 to network 510. The storage unit 504 stores, for example, various data necessary when the CPU 501 performs processing related to a program. Further, the storage unit 504 stores, for example, various data obtained by the CPU 501 performing processing related to a program. Note that data, programs, and the like used by the CPU 501 for processing may be stored in the storage unit 504.

FIG. 6 is a block diagram showing an example of the functional configuration of the information processing device 500. The information processing device 500 includes an image acquisition section 601, an object detection section 602, an image processing section 603, a result output section 604, and a storage section 605.

The image acquisition unit 601 acquires an image to be subjected to object detection. In this embodiment, the image on which object detection is performed in the information processing device 500 is an extracted image, and the image acquisition unit 601 acquires the extracted image from the information processing device 100 via the communication I/F unit 603 via the network 510. do.

The object detection unit 602 detects objects based on a plurality of attributes (classes) from an image. Similar to the object detection unit 202, the object detection unit 602 detects a “face wearing glasses” and a “face not wearing glasses” included in the extracted image. In this embodiment, the object detection unit 602 detects, for example, an RGB image of 448 x 448 pixels, in which the width in the horizontal direction (horizontal direction) and the height in the vertical direction (vertical direction) are both 448 pixels. do. The rest is the same as the object detection unit 202.

The image processing unit 603 transforms the extracted image into a specified size, and outputs the transformed image and the vertical and horizontal transformation magnifications. Note that an arbitrary method such as a generally known bicubic method can be used as an algorithm for transforming an image. Furthermore, super-resolution technology may be used when enlarging an image. In the example of this embodiment, the image processing unit 603 transforms the extracted image into an image of 448×448 pixels. In this case, if the width of the extracted image is w pixels and the height is h pixels, the vertical transformation magnification is (448/w) and the horizontal transformation magnification is (448/h).

The result output unit 604 uses the transformation magnification output from the image processing unit 603 to correct the detection frame of the detection result output from the object detection unit 602 to the coordinates in the extracted image before transformation, and outputs the result. In the example of this embodiment, when the vertical and horizontal deformation magnifications are wm and hm, respectively, for the detection frame with the upper left coordinates (x1, y1) and the lower right coordinates (x2, y2), the corrected upper left coordinates are (x1/wm, y1/hm), and the lower right coordinates are (x2/wm, y2/hm). The storage unit 605 stores data used in processing in each of the functional units 601 to 604 of the information processing device 500, data obtained as a processing result, and the like.

Next, processing performed by the information processing apparatuses 100 and 500 will be described with reference to FIGS. 3, 7, and 8. FIG. 7 is a flowchart illustrating object detection processing according to the second embodiment. FIG. 8 is a diagram illustrating the processing of the flowchart shown in FIG. 7.

In the object detection process according to the second embodiment, the processes of steps S301 to S306 and steps S308 to S309 shown in FIG. 3 are similar to the object detection process according to the first embodiment. The object detection process according to the second embodiment differs from the object detection process according to the first embodiment in the process of step S307 shown in FIG. 3. A detailed process flow in the second embodiment of the process corresponding to the process of step S307 shown in FIG. 3 will be described with reference to FIG. 7.

In step S701, the result correction unit 205 of the information processing apparatus 100 determines whether the number of detection results for the extracted image obtained in step S306 is two or more. If the result correction unit 205 determines that the number of detection results for the extracted image is two or more (YES in step S701), the process proceeds to step S702, and if it determines that there are not two or more (NO in step S701), the result correction unit 205 proceeds to step S702. ), the process moves to step S706.

In step S702, the image acquisition unit 601 of the information processing device 500 receives the extracted image from the information processing device 100. An example of the received extracted image 710 is shown in FIG. 8(a). Here, the extracted image 710 in this example is similar to the extracted image 420 in the first embodiment shown in FIG. 4(d), and has a horizontal width of 384 pixels and a vertical height of 372 pixels. Suppose that the image is .

In step S703, the image processing unit 603 of the information processing device 500 transforms the extracted image received in step S702 to a predetermined size and outputs the transformed image and the vertical and horizontal transformation magnifications. In this example, the image processing unit 603 transforms the 384 x 372 pixel extracted image received in step S702 into an image of 448 x 448 pixels. Therefore, the vertical transformation magnification is (448/384) and the horizontal transformation magnification is (448/372).

In step S704, the object detection unit 602 of the information processing device 500 performs face detection processing using the extracted image after deformation, and distinguishes a “face wearing glasses” and a “face without glasses” from the extracted image after deformation. Detect. FIG. 8B shows an example of the detection result of the face detection process on the extracted image after deformation, and FIG. 8C shows an example of an image obtained by superimposing the detection result on the extracted image after deformation. In the example shown in FIG. 8(b), one detection result F is obtained, and the upper left coordinates (x1, y1) and lower right coordinates (x2, y2) of the detection frame, attributes, and reliability are output. Ru. Note that the coordinates of the detection frame in FIG. 8(b) are the coordinates in the extracted image after deformation. Furthermore, in the example shown in FIG. 8(c), a detection frame 721 corresponding to detection result F detected as a face wearing glasses is displayed as a rectangular frame with a broken line superimposed on the extracted image 720 after deformation. .

In step S705, the result output unit 606 of the information processing device 500 corrects the coordinates of the detection frame of the detection result output in step S704 using the vertical and horizontal deformation magnification, and outputs the corrected coordinates in the extracted image before deformation. . That is, the result output unit 606 converts the coordinates of the detection frame in the 448 x 448 pixel image after transformation into the coordinates of the 384 x 372 pixel image before transformation, using vertical and horizontal transformation magnifications of 448/384 and 448/372. and output to the information processing device 100. An example of the converted detection result is shown in FIG. 8(d), and an example of an image obtained by superimposing the detection result on the extracted image of 384×372 pixels is shown in FIG. 8(e). In the example shown in FIG. 8(d), the upper left coordinates (x1, y1) and lower right coordinates (x2, y2) of the detection frame of the detection result F are respectively transformed according to the deformation magnification. Furthermore, in the example shown in FIG. 8(e), a detection frame 711 corresponding to detection result F detected as a face wearing glasses is displayed as a broken rectangular frame superimposed on the extracted image 710 before transformation. .

In step S706, the result correction unit 205 of the information processing apparatus 100 reflects the detection result for the extracted image outputted from the result output unit 604 of the information processing apparatus 500 in step S705 to the detection result of the input image. In this example, since there is one detection result (here, detection result F) for the extracted image, the result correction unit 205 deletes detection result A and detection result B associated with the extracted image, and Replace with result F.

According to the second embodiment, when overlap occurs between detection results of different attributes in object detection, object detection is performed on the image of the overlapping area, and the detection result is used to correct the detection result of the input image. This makes it possible to finally output a more appropriate detection result as the detection result of object detection for the input image.

(Modified example)
Note that in this embodiment, two information processing apparatuses 100 and 500 are connected through a network 510. However, the present invention is not limited thereto, and for example, the information processing apparatus 100 may be an edge device such as a camera, and the information processing apparatus 500 may be a device connected to an external terminal (such as a USB) of the edge device. Further, the information processing device 100 may be a PC (personal computer), and the information processing device 500 may exist on a cloud.

In addition, the object detection unit 602 is configured to detect a “face wearing glasses” and a “face not wearing glasses” similarly to the object detection unit 202, but for example, a “face wearing glasses” and a “face not wearing glasses” are detected. The detector may detect only one of the "faces worn". For example, if the detector detects only "faces wearing glasses", if a "face wearing glasses" is detected, the detection result will be "face wearing glasses", and if a "face wearing glasses" is not detected, then the detection result will be "faces wearing glasses". The detection result can be a "face without glasses." Alternatively, the detector may be divided into two types: a detector for detecting a face and a discriminator for determining whether or not glasses are worn.

[Third embodiment]
In the embodiments described above, processing was performed using a single input image, but in the third embodiment, a case will be described in which processing is performed using two input images captured at different timings. In the following description of the third embodiment, description of the same points as in the first embodiment will be omitted. The configuration of the information processing apparatus 100 according to this embodiment is similar to the configuration example in the first embodiment shown in FIG.

FIG. 9 is a block diagram showing an example of the functional configuration of the information processing device 100. In FIG. 9, components having the same functions as those shown in FIG. 2 are denoted by the same reference numerals, and redundant explanations will be omitted. The information processing device 100 includes an image acquisition unit 201, an object detection unit 202, an overlap determination unit 203, an image extraction unit 904, a result modification unit 905, a result output unit 206, a storage unit 207, and a correlation unit 908.

Similar to the image extraction unit 204, the image extraction unit 904 uses the input image and a set of detection results determined to be overlapping by the overlap determination unit 203 to extract information from the input image based on the determination result by the overlap determination unit 203. Extract the specified area. The image extraction unit 904 outputs the extracted image and the upper left coordinates of the region to be extracted with respect to the input image. Furthermore, if the number of detection results for the extracted image is two or more, the image extraction unit 904 outputs a detection result with the union of the detection frames as a new detection frame. At this time, it is assumed that the face attribute is "unknown to wear glasses" and the detection reliability is 1. Note that the coordinates of the detection frame are converted into coordinates in the input image using the same method as in the first embodiment.

The matching unit 908 matches the past detection results with the current detection results. The matching unit 908 matches the detection results (hereinafter also referred to as "current detection results" or "current detection frames") with the tracking information stored therein, updates the tracking information, and updates the attributes of the detection results based on the tracking information. As the tracking information, a tracking ID, coordinates of the detection frame corresponding to the tracking ID (upper left coordinates and lower right coordinates), attributes, and attribute statistical information are stored. Here, the attribute statistical information refers to the history (number of appearances) of the attribute for a preset number of times in the past. In the example of this embodiment, the preset number is set to 2, and for example, if the previous attribute was wearing glasses and the current attribute was not wearing glasses, the attribute statistical information is set to 1 for wearing glasses and 1 for not wearing glasses. If the attribute is unknown as to whether or not wearing glasses, it is not added to the attribute statistical information.

Furthermore, the association unit 908 updates the attributes of the current (current) detection results based on information regarding the attributes of past detection results. Specifically, the association unit 908 updates the attribute that appears most frequently (has a large number of appearances) at that time as the attribute of the detection result, based on the attribute statistical information. If the appearance frequencies are the same, the attribute in the current detection result takes precedence. The correspondence between the current detection result and the retained tracking information is performed based on, for example, the coordinates of the detection frame corresponding to each tracking ID and the coordinates of the detection frame of the current detection result. In this embodiment, the distance between the center coordinates of the detection frame corresponding to each tracking ID and the center coordinates of any current detection frame is calculated for all combinations, and the distance is less than or equal to a threshold (100 in this embodiment). The tracking ID with the shortest distance is associated with the combination of the current detection result. Additionally, the coordinates corresponding to the tracking ID are updated to the center coordinates of the current detection frame. Tracking IDs that do not correspond to the current detection result are deleted. If there is a current detection result that does not correspond to the tracking ID, a new tracking ID is generated and added to the tracking information.

The result correction unit 905 reflects the detection result for the extracted image on the detection result for the input image. The result modification unit 905 deletes the detection result used when calculating the extracted area from among the detection results of the input image, and replaces it with the detection result obtained from the extracted image.

Next, object detection processing performed by the information processing apparatus 100 will be described with reference to FIGS. 3, 10, and 11. FIG. 10 is a flowchart illustrating object detection processing according to the third embodiment. FIG. 11 is a diagram illustrating the processing of the flowchart shown in FIG. 10.

In the object detection process according to the third embodiment, the processes of steps S301 to S306 and steps S308 to S309 shown in FIG. 3 are similar to the object detection process according to the first embodiment. The object detection process according to the third embodiment differs from the object detection process according to the first embodiment in the process of step S307 shown in FIG. 3. A detailed process flow in the third embodiment of the process corresponding to the process of step S307 shown in FIG. 3 will be described with reference to FIG. 10.

In step S1001, the result correction unit 905 determines whether the number of detection results for the extracted image obtained in step S306 is two or more. If the result correction unit 905 determines that the number of detection results for the extracted image is two or more (YES in step S1001), the process proceeds to step S1002, and if it determines that there are not two or more (NO in step S1001), the result correction unit 905 moves to step S1002. ), the process moves to step S1005. In this example, the detection results for the input image are as shown in FIG. 4(b), and the detection results for the extracted image are two detection results (detection result G and detection result G) as shown in FIG. 11(a). Assume that H) exists. Note that in FIG. 11A, the coordinates of the detection frame are the coordinates in the coordinate system of the extracted image.

In step S1002, the image extraction unit 904 calculates a circumscribing rectangle for the union of two detection frames from the set of detection results output in step S304, and detects the calculated circumscribed rectangle, in a similar manner to the process in step S305. Output the detection results as a frame. In the example of this embodiment, the image extraction unit 904 calculates a circumscribed rectangle for the union of two detection frames from the set of detection results G and H, and outputs a detection result I with the calculated circumscribed rectangle as a detection frame. . From the detection results shown in FIG. 11(a), in the coordinate system of the extracted image, the circumscribed rectangle for the union of the detection frames of the two detection results has the upper left coordinates (32, 31) and the lower right coordinates (352, 341). By converting from the coordinate system of the extracted image to the coordinate system of the input image, the circumscribed rectangle for the union of the detection frames of the two detection results in the coordinate system of the input image has the upper left coordinate as shown in FIG. 11(b). (20,200), and the lower right coordinates are (340,510).

In step S1003, the association unit 908 associates the detection result output from the image extraction unit 904 in step S1002 with the retained tracking information. In the example of this embodiment, the association unit 908 associates the detection result I output in step S1002 with the retained tracking information. Assuming that tracking information as shown in FIG. 11(c) is held, the association with detection result I will be explained as an example. The center coordinates of the detection frame for detection result I are (180, 355), and the center coordinates of the detection frame for tracking ID 1 are (170, 350). Therefore, the distance L between the center coordinates of the detection frame of the detection result I and the tracking ID 1 is:
L=((180-170)^2+(355-350)^2)^(1/2)≒11.2
Since it is smaller than the threshold value of 100, it becomes a candidate for association. Furthermore, if the detection result I, tracking ID2, and tracking ID3 are similarly calculated, the distance between the center coordinates will be greater than 100, and therefore they will not be candidates for association. As a result, the detection result I and the tracking ID1 are associated.

In step S1004, the association unit 907 updates the attribute of the detection result based on the tracking information. Detection result I is associated with tracking ID 1, and the statistical information on the attribute of the tracking ID (FIG. 11(c)) shows that the number of times glasses are worn is greater than the number of times glasses are not worn, so the attribute of detection result I is glasses worn.

In step S1005, the result modification unit 905 reflects the detection result for the extracted image on the detection result for the input image, similarly to the first embodiment. Here, the result modification unit 205 deletes detection result A and detection result B, and replaces them with detection result I.

According to the third embodiment, even if detection results of different attributes overlap in object detection using images, a more appropriate detection result can be finally output.

(Modified example)
Note that as a method for associating the detection results of past input images with the detection results of the current input image in the associating unit 908, a method generally used in object tracking processing can be applied. Furthermore, in this embodiment, attributes for two times are held as statistical information, but even if attributes for more than two times are held as statistical information, processing can be performed in the same manner as in this embodiment. be. Further, in this embodiment, the attribute statistical information is the attribute history (the number of appearances) for a predetermined number of times, but it may be an accumulation of reliability instead of the number of appearances.

[Fourth embodiment]
In the first embodiment described above, the extracted image is generated from the image used for the first face detection, but in the fourth embodiment, the extracted image is generated from the original image of the image used for the first face detection. The case of generation will be explained. In the following description of the fourth embodiment, descriptions of points similar to those of the first embodiment will be omitted. The configuration of the information processing apparatus 100 according to this embodiment is similar to the configuration example in the first embodiment shown in FIG.

Fig. 12 is a block diagram showing an example of the functional configuration of the information processing device 100. In Fig. 12, components having the same functions as those shown in Fig. 2 are given the same reference numerals, and duplicated explanations are omitted. The information processing device 100 has an image acquisition unit 1201, an object detection unit 1202, an overlap determination unit 203, an image extraction unit 204, a result correction unit 205, a result output unit 206, a memory unit 207, and an image processing unit 1208.

The image acquisition unit 1201 acquires an image (input image) to be subjected to object detection. Furthermore, the image acquisition unit 1201 extracts a predetermined range from the acquired image. In this embodiment, an image (input image) to be subjected to object detection is acquired from the outside through the communication I/F unit 103, and is, for example, an RGB image of 1080×720 pixels. Furthermore, it is assumed that the image acquisition unit 1201 extracts, for example, a range of upper left coordinates (30, 0) and lower right coordinates (750, 720) from the input image.

The object detection unit 1202 performs object detection based on a plurality of attributes (classes) from an image, similar to the object detection unit 202 in the first embodiment. In this embodiment, the object detection unit 1202 performs detection from an RGB image of 224 x 224 pixels, for example, where the width in the horizontal direction (horizontal direction) and the height in the vertical direction (vertical direction) are both 224 pixels. shall be. The rest is the same as the object detection unit 202 in the first embodiment.

The image processing unit 1208 transforms the image to a specified size and outputs the transformed image and the vertical and horizontal transformation magnifications. Any algorithm, such as the commonly known bicubic algorithm, can be used for transforming the image. Super-resolution technology may be used to enlarge the image. In this embodiment, the image processing unit 1208 transforms the image to an image of 224 x 224 pixels. In this case, if the width of the image before transformation is w pixels and the height is h pixels, the vertical transformation magnification is (224/w) and the horizontal transformation magnification is (224/h).

The result modification unit 205 reflects the detection result for the extracted image by the object detection unit 1202 on the detection result for the input image according to the number of detection results for the extracted image. The rest is the same as the result correction unit 205 in the first embodiment.

Next, processing performed by the information processing apparatus 100 will be described with reference to FIGS. 13 and 14. FIG. 13 is a flowchart illustrating object detection processing according to the fourth embodiment. FIG. 14 is a diagram illustrating object detection processing according to the fourth embodiment.

In step S1301, the image acquisition unit 1201 acquires an input image (an image to be subjected to object detection), and extracts a range of upper left coordinates (30,0) and lower right coordinates (750,720) from the acquired input image. Extract to obtain a 720x720 pixel image. An example of an input image 1401 is shown in FIG. 14(a), and an example of an image 1402 of 720×720 pixels extracted from the input image 1401 is shown in FIG. 14(b).

In step S1302, the image processing unit 1208 transforms the 720×720 pixel image acquired in step S1301 into a 224×224 pixel image (hereinafter referred to as a first processing target image). At this time, both the vertical deformation magnification and the horizontal deformation magnification are (224/720). An example of the first processing target image 1403 is shown in FIG. 14(c).

In step S1303, the object detection unit 1202 performs face detection processing to detect human faces using the first processing target image, similar to step S302 shown in FIG. 3, and detects "faces with glasses" and "faces without glasses" from the first processing target image. An example of the detection results of the face detection processing for the first processing target image is shown in FIG. 14(d). In the example shown in FIG. 14(d), three detection results A to C are obtained, and the upper left coordinates (x1, y1) and lower right coordinates (x2, y2) of the detection frame, attributes, and reliability are output for each. Note that the coordinates of the detection frame in FIG. 14(d) are coordinates in the first processing target image.

In step S1304, the overlap determination unit 203 sets any two of the detection results for the first processing target image as a pair, and calculates the overlap rate of the detection frames for each combination of the detection results for the first processing target image. In this example, the upper left coordinates of the detection frame of detection result A are (0, 62) and the lower right coordinates are (85, 156). The upper left coordinates of the detection frame of detection result B are (0, 65) and the lower right coordinates are (91, 159). Therefore, the overlap rate of the detection frames of detection results A and B is
IoU(A,B) = ((85-0) x (156-65)) ÷ ((85-0) x (156-62) + (91-0) x (159-65) - ((85-0) x (156-65))) ≒ 0.88
In other combinations, the overlap rate of the detection windows is 0.

In step S1305, the overlap determination unit 203 determines whether there is a combination of detection results for which the overlap rate calculated in step S1304 is equal to or greater than a threshold value. If the overlap determination unit 203 determines that there is a combination of detection results in which the overlap rate of the detection frames is equal to or higher than the threshold (YES in step S1305), the overlap determination unit 203 outputs the combination of detection results in which the overlap ratio is equal to or higher than the threshold; The process moves to step S1306. On the other hand, if the overlap determination unit 203 determines that there is no combination of detection results in which the overlap rate of the detection frames is equal to or greater than the threshold (NO in step S1305), the process proceeds to step S1311. In the example of this embodiment, the threshold value of the overlap rate is assumed to be 0.5. The overlap determining unit 203 moves to step S1306 if there is a set of detection results for which the overlap rate calculated in step S1304 is 0.5 or more, and moves to step S1311 if there is not. In this example, since the overlap rate of the detection frames of detection result A and detection result B is 0.5 or more, the overlap determination unit 203 outputs the combination (A, B) with an overlap rate of 0.5 or more. Then, the process moves to step S1306.

In step S1306, the image extraction unit 204 extracts the specified region from the input image using the set of the input image and the detection result output in step S1305, and extracts the specified region from the input image and the upper left corner of the region to be extracted with respect to the input image. Output the coordinates of. Specifically, first, the image extraction unit 204 converts the coordinates of two detection frames from the set of detection results into the coordinates of the input image using information on the range and transformation magnification extracted from the input image. In this example, the upper left coordinates of the detection frame of detection result A are (0, 62), so using the transformation magnification (224/720), the upper left coordinates of detection result A in the input image are (30+0÷( 224/720), 0+62÷(224/720))=(30,199). The results of calculations for other coordinates are shown in FIG. 14(e).

Next, the image extraction unit 204 calculates a circumscribing rectangle for the union of the two detection frames from the set of detection results, and calculates the top left and bottom right coordinates of the area to be extracted based on the calculated circumscribing rectangle. Details are the same as in step S305 shown in FIG. 3. In this example, the top left coordinate of the area to be extracted is (1, 168), and the bottom right coordinate is (352, 542). FIG. 14(f) shows an example of an extracted image 1404 extracted from the input image based on the top left and bottom right coordinates of the area to be extracted calculated in this way. Since the top left coordinates of the area to be extracted are (1, 168) and the bottom right coordinates are (352, 542), the extracted image 1404 is an image of 351 x 374 pixels.

In step S1307, the image processing unit 1208 transforms the 351×374 pixel image extracted in step S1306 into a 224×224 pixel image (hereinafter referred to as a second processing target image). At this time, the deformation magnification in the horizontal direction is (224/351), and the deformation magnification in the vertical direction is (224/374). An example of the second processing target image is shown in FIG. 14(g).

In step S1308, the object detection unit 1202 performs face detection processing to detect a person's face using the second processing target image in the same manner as step S306 shown in FIG. A "face wearing glasses" and a "face not wearing glasses" are detected. An example of the detection result of the face detection process for the second processing target image is shown in FIG. 14(h). In the example shown in FIG. 14(h), one detection result E is obtained, and the upper left coordinates (x1, y1) and lower right coordinates (x2, y2) of the detection frame, attributes, and reliability are output. Ru. Note that the coordinates of the detection frame in FIG. 14(h) are the coordinates in the second processing target image.

In step S1309, the result modification unit 205 reflects the detection result for the extracted image in the detection result for the input image, based on the detection result for the second processing target image obtained in step S1308. Specifically, similar to step S307 shown in FIG. 3, the result correction unit 205 converts the coordinates of the detection result for the second processing target image into the coordinates of the input image. At this time, the result correction unit 205 uses the upper left coordinates of the region to be extracted in the input image obtained in step S1306 and the information on the deformation magnification obtained in step S1307 to modify the detection result for the second processing target image. Convert coordinates to input image coordinates. An example of the detection result E in the input image is shown in FIG. 14(i).

In step S1310, the result output unit 206 determines whether processing has been completed for all combinations of detection results in which the detection frame overlap rate is equal to or greater than a threshold value. If the result output unit 206 determines that processing has been completed for all combinations of detection results for which the overlap rate is equal to or greater than the threshold (YES in step S1310), the process proceeds to step S1311. If the result output unit 206 determines that there is an unprocessed combination among the combinations of detection results for which the overlap rate is equal to or higher than the threshold (NO in step S1310), the process proceeds to step S1306, and the unprocessed combinations are processed from step S1306 onwards. Execute the process.

In step S1311, the result output unit 206 outputs an image in which the final detection result for the input image is superimposed on the input image, ends the process, and proceeds to process the next input image.

According to the fourth embodiment, the extracted image used in the second detection process is extracted from the input image rather than the image used in the first detection process. As a result, even if the image used in the first detection process has a lower resolution than the input image due to image transformation processing, etc., the final detection result of object detection for the input image will be more appropriate. can be output. In addition, even if a part of the object to be detected is cut out when cutting out the image to be used in the first detection process from the input image, the final result of object detection for the input image will be more appropriate. Detection results can be output.

(Modified example)
Although the image acquisition unit 1201 extracts one region from the input image, it may extract a plurality of regions and repeat the processing from step S1302 to step S1311 for each region. Furthermore, it is also possible to incorporate the processing of this embodiment into the second embodiment and third embodiment described above.

[Other embodiments]
The present invention provides a system or device with a program that implements one or more functions of the embodiments described above via a network or a storage medium, and one or more processors in a computer of the system or device reads and executes the program. This can also be achieved by processing. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

100, 500: Information processing device 201, 601, 1201: Image acquisition unit 202, 602, 1202: Object detection unit 203: Overlap determination unit 204, 904: Image extraction unit 205, 905: Result correction unit 206, 604: Result output Sections 207, 605: Storage section 603, 1208: Image processing section 908: Correspondence section

Claims (12)

Detection means for detecting objects corresponding to each of a first attribute and a second attribute different from the first attribute from a first image;
a detection area detected from the first image by the detection means, a first detection area in which an object corresponding to the first attribute is detected; and a detection area in which an object corresponding to the second attribute is detected; a determining means for determining an overlap between the second detection area and the second detection area;
In response to the determining means determining that the first detection area and the second detection area overlap, the first detection area and the second detection area overlap in the first image. As a result of the detection means performing object detection on a second image obtained by enlarging an area including an overlapping area, the detection means corresponds to the first attribute or the second attribute from the second image. and a correction means for correcting the detection result of the object detection performed by the detection means on the first image based on the detection result of the object detection on the second image when an object is detected. Information processing device. The determining means is configured to determine whether the first detection area and the second detection area overlap each other by a predetermined ratio or more in the first image. The information processing apparatus according to claim 1, wherein the information processing apparatus determines that two detection areas overlap. The information processing device according to claim 1 or 2 , wherein the first image is a captured image obtained by capturing an image with an imaging device. 4. The information processing apparatus according to claim 1, wherein the second image is an image corresponding to an area including a union of the first detection area and the second detection area. The information processing apparatus according to claim 4 , wherein the second image includes a union of the first detection area and the second detection area, and a predetermined margin. It is characterized by having an association means for associating past detection results with current detection results and updating the attributes corresponding to the current detection results based on information regarding the attributes corresponding to the past detection results. The information processing device according to any one of claims 1 to 5 . 6. The associating means obtains statistical information of attributes in the detection results of predetermined past images, and sets the attribute with the highest frequency of appearance in the statistical information as the attribute of the current detection result. The information processing device described in . 8. The information processing apparatus according to claim 1, wherein the first attribute and the second attribute indicate wearing of glasses and non-wearing of glasses, respectively. An information processing system including a first information processing device and a second information processing device,
The first information processing device includes:
a first detection means for detecting objects corresponding to each of a first attribute and a second attribute different from the first attribute from a first image;
a detection area detected from the first image by the first detection means, in which an object corresponding to the first attribute is detected; and a detection area corresponding to the second attribute. determining means for determining an overlap with a second detection area in which the object is detected;
When the determination means determines that the first detection area and the second detection area overlap, the area where the first detection area and the second detection area overlap in the first image is determined. Extracting means for extracting a second image corresponding to the region including;
When the second information processing device performs object detection using the second image and an object corresponding to the first attribute or the second attribute is detected from the second image, a correction means for correcting a detection result of object detection by the first detection means for the first image based on a detection result of object detection for the second image;
a first communication means for outputting the second image extracted by the extraction means to the second information processing device;
The second information processing device includes:
A second method for detecting an object corresponding to each of the first attribute and the second attribute from an image obtained by enlarging the second image output by the first communication means. detection means;
a second communication means for outputting a detection result of object detection on the image by the second detection means to the first information processing device as a detection result of object detection on the second image . An information processing system characterized by: The first information processing device is an imaging device,
The information processing system according to claim 9 , wherein the second information processing device is another device connected to the imaging device. a detection step of detecting objects corresponding to each of a first attribute and a second attribute different from the first attribute from the first image;
A detection area detected from the first image in the detection step, a first detection area in which an object corresponding to the first attribute is detected, and a first detection area in which an object corresponding to the second attribute is detected. a determination step of determining an overlap between the second detection area and the second detection area;
In response to determining that the first detection area and the second detection area overlap in the determination step, the first detection area and the second detection area overlap in the first image. As a result of performing object detection in the detection step on a second image obtained by enlarging an area including an overlapping area, the object corresponds to the first attribute or the second attribute from the second image. If an object is detected, the method further comprises a correction step of correcting the detection result of object detection in the detection step for the first image based on the detection result of the object detection for the second image. Information processing method. A program for causing a computer to function as the information processing device according to any one of claims 1 to 8 .

Images