JP7367540B2 - Image evaluation device and image evaluation program - Google Patents

Description

The present invention relates to an image evaluation device and an image evaluation program.

When restoring damaged cultural properties, it is preferable to reuse the original materials as much as possible. For example, when a stone wall collapses in a castle or the like, it is desirable to identify where on the wall the fallen stones were piled up, and to re-stack the stones in their original arrangement.

When carrying out such work, it is important to know the original appearance of the object being restored, and the public may be asked to provide images in order to obtain photographs or other images of the original appearance of the restoration object. be. Then, for example, a good-looking image to be repaired is selected from among the provided images, and the repair work is performed using that image as a reference.

In this regard, techniques related to evaluating image features are known (for example, Patent Documents 1 to 3).

International Publication No. 2017/006852 Japanese Patent Application Publication No. 2010-118030 JP2010-224987A

However, it may take a lot of man-hours to select from among a large number of images a useful image that shows a good image of the object from which image information is to be obtained.

In one aspect, an object of the present invention is to provide a technique that can evaluate the appearance of an object in an image.

An image evaluation device according to one aspect of the present invention includes a group of feature points specified in a region in which a target object appears in a reference image, and a plurality of feature points identified in a region in which a target object appears in a reference image. Among them, there is a specifying unit that identifies corresponding points, which are a set of corresponding feature points, between a standard image and a reference image, based on the feature values that characterize the feature points, and a specifying unit that identifies a corresponding point, which is a set of corresponding feature points, between a standard image and a reference image; a determining unit that determines a difference in scale between an object appearing in the reference image and an object appearing in the reference image based on the arrangement of the points and the arrangement of the plurality of corresponding points identified in the reference image; A modification section that changes the scale of the reference image to match the scale of the object in the standard image based on the difference, and an evaluation value that evaluates the degree of clarity of the object in the reference image after modification by the modification section. and an output section that outputs the.

It is possible to evaluate the appearance of objects in images.

FIG. 3 is a diagram illustrating a flow of evaluating the sharpness of an object appearing in an image according to an embodiment. 1 is a diagram illustrating a block configuration of an image evaluation device according to an embodiment. FIG. 3 is a diagram illustrating an operational flow of image evaluation processing according to the embodiment. FIG. 3 is a diagram illustrating an operation flow of a process for extracting features of a target object from an image according to an embodiment. FIG. 3 is a diagram illustrating identification of a target area according to an embodiment. FIG. 3 is a diagram illustrating identification of feature points according to the embodiment. FIG. 3 is a diagram illustrating image feature information according to an embodiment. FIG. 6 is a diagram illustrating an example in which feature extraction processing of a target object is performed on a reference image. FIG. 3 is a diagram illustrating changing the scale of a reference image. FIG. 3 is a diagram illustrating a histogram in which linear regions are detected from an image and angles are totaled. FIG. 6 is a diagram illustrating a change in the angle of a reference image. FIG. 3 is a diagram illustrating evaluation result information according to the embodiment. 1 is a diagram illustrating a hardware configuration of an information processing device for realizing an image evaluation device according to an embodiment.

Hereinafter, some embodiments of the present invention will be described in detail with reference to the drawings. Note that the same reference numerals are given to corresponding elements in a plurality of drawings.

As described above, it may take a lot of man-hours to select a useful image that clearly depicts the object from a large number of images such as photographs provided.

Here, a well-represented image of the object may be an image that satisfies at least one of the following conditions.
(Condition 1) A high-resolution image in which the object appears large in the image (Condition 2) An image in which the object is in focus (Condition 3) An image in which the object is photographed directly in front of the image

Here, regarding (condition 1), a high-resolution image in which the target object appears large in the image is a preferable image because detailed information regarding the target object can be obtained from the image. Regarding (condition 2), for example, even if the image is a high resolution image, if the focus is on something other than the target object, the target object may be out of focus. In this case, the object may appear blurry and detailed information about the object may not be collected from the image. Therefore, it is preferable that the image be in focus on the object. Regarding (condition 3), for example, when photographing an object from an angle, even if part of the object is in high resolution and in focus, the rest of the object may be far away or out of focus. There is a possibility that there is no such thing. In this case, only part of the information about the object in the image may be usefully used. Therefore, it is preferable that the image be taken facing the object from the front. Note that even if an image is taken from an angle, it is possible to convert it into a facing image using perspective transformation, etc., but if the parameters used during conversion are not accurate, the error will be large and the shape may differ from the actual shape. Different images may be generated. Therefore, it is preferable to use an image taken from the front, which can be used without perspective transformation.

For example, in consideration of the above points, it may be desired to extract an image that shows the object as well as possible from a large number of images. However, for example, even if images are taken of the same object, the size of the object in the image may vary; Photography conditions such as the photographic device used may differ. In such a case, simply selecting an image captured by an imaging device with high resolution does not necessarily identify an image that includes a lot of information about the object. Therefore, it is desired to provide a technique that can evaluate whether an object is clearly visible in an image.

In the embodiment described below, the scales of a plurality of images to be compared are converted so that the scales of the objects in the images are approximately the same, and the sharpness of the objects is evaluated in the obtained converted images. Therefore, it is possible to evaluate the quality of an object's appearance among a plurality of images based on the evaluation value.

FIG. 1 is a diagram illustrating the flow of processing for evaluating the sharpness of an image according to an embodiment. For example, assume that there is a standard image used as a standard when evaluating the sharpness of an object, and a reference image whose scale is adjusted based on the standard image (FIG. 1(a)).

In the embodiment, from these standard images and reference images, a region in which a target object is captured (in FIG. 1, a region in which a castle's stone wall is depicted) is extracted, a feature point is identified within the extracted target region, and the feature point is Determine the feature quantity to characterize (FIG. 1(b)). Note that the feature amount may be a local feature amount, for example.

Next, the feature points extracted from the object region of the reference image and the feature points extracted from the object region of the reference image are matched based on the feature amount, and the feature points of the reference image and the feature points of the reference image are matched. Identify the corresponding points among them. Then, based on the identified corresponding points, the difference in scale of the object in the two images is identified (FIG. 1(c)).

Then, the scale of the reference image is changed so that the scale of the object in the reference image matches the scale of the object in the reference image (FIG. 1(d)). In this manner, in the embodiment, the evaluation value of the sharpness of the object region is obtained after the scales of the objects are aligned to some extent (FIG. 1(e)). As the sharpness evaluation value, for example, the object region can be subjected to a filter that extracts edges such as a Laplacian filter or a Sobel filter to extract edges, and the variance of pixel values of the obtained edge image can be used. For example, if the object is photographed farther away and smaller in the reference image than in the reference image, the edges will become blurred due to the enlargement by adjusting the scale in FIG. 1(c), resulting in a lower sharpness evaluation value. Alternatively, assume that the target object is in focus in the standard image, while the target object is out of focus in the reference image. In this case, since the target object is blurred in the reference image, the sharpness evaluation value is low. Therefore, an image with a high sharpness evaluation value can be determined to be a useful image.

As described above, in the embodiment, the scale of the object shown in another reference image is changed in accordance with the scale of the object shown in the reference image, and the sharpness of the area where the object is shown after the change is evaluated. Therefore, it is possible to specify an image that shows a good object based on the evaluation value. Hereinafter, embodiments will be described in more detail.

FIG. 2 is a diagram illustrating a block configuration of the image evaluation device 200 according to the embodiment. The image evaluation device 200 may be an information processing device including a computer such as a server computer, a personal computer (PC), a notebook PC, a tablet terminal, or a smartphone. Image evaluation device 200 includes, for example, a control section 201 and a storage section 202. The control unit 201 includes, for example, a specifying unit 211, a determining unit 212, a changing unit 213, an output unit 214, and may also include other functional units. The storage unit 202 of the image evaluation device 200 stores information such as a standard image, a reference image, image feature information 700, and evaluation result information 1200, which will be described later. The details of each of these units and the information stored in the storage unit 202 will be described later.

Next, image evaluation processing according to the embodiment will be described.

FIG. 3 is a diagram illustrating an operational flow of image evaluation processing according to the embodiment. For example, the control unit 201 of the image evaluation device 200 may start the operation flow shown in FIG. 3 when an instruction to execute image evaluation processing is input together with designation of a plurality of images to be evaluated.

In step 301 (hereinafter, step will be abbreviated as "S", for example, S301), the control unit 201 selects a reference image from among a plurality of images to be evaluated. The reference image is preferably selected from among images that satisfy at least one of the above conditions (condition 1) to (condition 3), and in one example, the user selects a preferable image that satisfies these conditions as the reference image. You may choose. Thereby, as will be described later, it becomes possible to evaluate the image by adjusting the scale to the image photographed under favorable conditions. However, the embodiment is not limited to this, and in another embodiment, the control unit 201 may randomly select a reference image in S301.

In S302, the control unit 201 executes feature extraction processing of the object using the reference image as the processing target. FIG. 4 is a diagram illustrating an operation flow of a process for extracting features of an object from an image according to the embodiment.

In S401, the control unit 201 divides the image to be processed into a plurality of sections. For example, the control unit 201 may divide the image into a predetermined number of sections, or into a plurality of sections of a predetermined size.

In S402, the control unit 201 obtains image parameters for each of the plurality of sections. In one example, the image parameter may be an image obtained by performing two-dimensional Fourier transform on pixel values within a section. In another embodiment, the control unit 201 extracts a straight line area from an image within a section, aggregates the straight line areas extracted from the section for each angle, and uses a histogram in which the number for each angle is plotted as an image parameter. Good too. Note that an example of this histogram will be described later with reference to FIG. 10.

In S403, the control unit 201 groups the sections based on the image parameters. For example, if the image parameters of a section satisfy a predetermined similarity condition and are similar to the image parameters of adjacent sections, the control unit 201 may group those sections into the same group.

For example, the control unit 201 calculates a correlation coefficient between a Fourier transform image of a certain section and a Fourier transform image of an adjacent section, and if the correlation coefficient is greater than or equal to a predetermined threshold, the two Partitions may be grouped into the same group. Alternatively, the control unit 201 calculates, for example, a correlation coefficient between a histogram that totals the number of straight line regions for each angle for a section and a histogram that totals the number of straight line regions for each angle for the section and adjacent sections. Then, the control unit 201 may group the two sections into the same group when the correlation coefficient of the histogram is greater than or equal to a predetermined threshold. As a result, it is possible to extract a group of highly similar image areas, such as an area where a castle's stone wall is shown or an area where the sky is shown, from the image.

In S404, the control unit 201 identifies a group that satisfies a predetermined determination condition from among the grouped areas as a target object area in which the target object is captured. For example, the control unit 201 may specify the group with the largest area among the grouped regions as the target object region. Note that, for example, as described in (Condition 1) above, one of the conditions for a preferable image is that the object area is captured in a large size. Therefore, for example, even if it is assumed that the target object is captured in the largest area among the grouped areas and the subsequent processing is performed, it is possible to correctly evaluate a preferable image.

FIG. 5 is a diagram illustrating the identification of the target object area described above. For example, FIG. 5(a) shows the entire image. Then, the control unit 201 identifies a plurality of groups from the entire image based on the similarity of image parameters, and in FIG. has been identified.

Subsequently, in S405, the control unit 201 identifies a plurality of feature points from the identified target object area. Note that a plurality of feature points may also be referred to as a feature point group. For example, the control unit 201 may use corners detected using a corner detection method such as Harris or FAST (Features from Accelerated Segment Test) as feature points. In another embodiment, the control unit 201 may use a predetermined point in the object region, such as a lattice point when the object region is divided into a lattice shape at fixed pixel intervals, as the feature point.

FIG. 6 is a diagram illustrating identification of feature points according to the embodiment. For example, by using a corner detection method or the like from the image of the object region shown in FIG. 6(a), the corners of the stones of the stone wall are extracted as feature points as shown in FIG. 6(b).

In S406, the control unit 201 obtains a feature amount characterizing the feature point for each extracted feature point, and stores it in association with the feature point. For example, the control unit 201 may use algorithms such as AKAZE, SIFT, SURF, and BRIEF to obtain local features for feature points, and use the obtained local features as the features. Note that AKAZE is, for example, an abbreviation for Accelerated KAZE. SIFT is, for example, an abbreviation for Scale-invariant feature transform. For example, SURF is an abbreviation for Speed-Upped Robust Feature. BRIEF, for example, is an abbreviation for Binary Robust Independent Elementary Features. Then, the control unit 201 may register the feature amount of the obtained feature point in the image feature information 700 in association with the image, and may save it in the storage unit 202.

FIG. 7 is a diagram illustrating image feature information 700 according to the embodiment. In the image feature information 700, feature points and feature amounts such as local feature amounts that characterize the feature points are registered in association with each other for an image identified by an image identifier.

When the process of S406 is completed, the operation flow of FIG. 4 ends, and the flow advances to S303. According to the above-described operation flow of FIG. 4, for example, image feature information 700 including information on feature amounts of feature points identified from the target object area of the image to be processed is generated and stored in the storage unit 202. Ru.

Subsequently, in S303, the control unit 201 selects one unselected image other than the reference image as a reference image from among the plurality of evaluation target images specified at the start of the operation flow in FIG. 3.

In S304, the control unit 201 executes the feature extraction process of the object shown in FIG. 4, using the image selected in S303 as the processing target. Note that in S302, the operation flow of FIG. 4 is executed with the reference image as the processing target, but in S304, the operation flow of FIG. 4 is executed with the reference image selected in S303 as the processing target.

FIG. 8 is a diagram showing an example in which the feature extraction process of the object shown in FIG. 4 is executed on the selected reference image. As shown in Figure 8(a), the reference image shows two sides of the stone wall, and as a result of grouping based on image parameters, the side of the stone wall on the left, which has the largest size, is selected as the target area. There is. Then, in the process of FIG. 4, as shown in FIG. 8(b), feature points are extracted from the side of the stone wall on the left, and the feature points and the feature values obtained for the feature points are selected. The image feature information 700 may be generated in association with the image identifier of the reference image.

In S305, the control unit 201 matches the feature amounts of the feature points using the image feature information 700 of the reference image and the image feature information 700 of the reference image generated in S304. Then, the control unit 201 identifies corresponding points, which are a set of corresponding feature points, between the standard image and the reference image.

In S306, the control unit 201 determines the target object that appears in the reference image and the object that appears in the reference image based on the arrangement of the plurality of corresponding points specified in the reference image and the arrangement of the plurality of corresponding points specified in the reference image. Determine the difference in scale from the object. For example, the control unit 201 randomly selects two sets of corresponding points between the standard image and the reference image, and calculates the distance between the two corresponding points on the image plane. The control unit 201 may determine the difference in scale by determining the ratio of the distance between the corresponding points found in the reference image to the distance between the corresponding points found in the reference image. Alternatively, the control unit 201 randomly selects two sets of corresponding points and repeatedly executes the process of calculating the distance ratio between the corresponding points, and calculates the scale of the object by summing up the obtained multiple ratios. A representative value representing the ratio representing the difference between the two may be obtained. Note that the representative value representing the plurality of ratios representing the difference in scale may be, for example, a statistically determined statistical value such as an average value, a median value, or an intermediate value.

In S307, the control unit 201 changes the scale of the reference image so that the scale of the object shown in the reference image matches the scale of the object shown in the reference image, based on the difference in the obtained scales. For example, the control unit 201 changes the image size of the reference image by a magnification of (distance between two corresponding points in the reference image/distance between two corresponding points in the reference image). The scale of the object may be matched to the reference image.

FIG. 9 is a diagram illustrating changing the scale of the reference image in the process of S307. FIG. 9A shows an object area of the reference image, and FIG. 9B shows an example of changing the scale of the object area of the reference image. In FIG. 9(b), the object area of the reference image has been enlarged according to the difference in scale, and has been changed to the same scale as the object area of the reference image illustrated in FIG. 9(a). ing. Note that in FIG. 9(b), as a result of the enlargement, the edges of the stone wall reflected in the target object area are blurred. In this way, the edges of objects in images tend to become blurred when enlarged, and conversely become sharp when reduced. Then, as shown in FIG. 9(b), by converting the reference image so that the scale of the object in the reference image is approximately the same as the scale of the object in the standard image, the image of the object can be clearly seen. It becomes possible to uniformly evaluate the quality.

Subsequently, in S308, the control unit 201 acquires angle information regarding the difference in orientation between the object shown in the reference image and the object shown in the reference image. For example, the control unit 201 determines the arrangement of the object in the reference image based on the arrangement of the plurality of corresponding points specified in the reference image and the arrangement of the plurality of corresponding points specified in the reference image. An angle formed by the arrangement of objects in the reference image may be determined. As an example, the control unit 201 randomly selects two sets of corresponding points that correspond between the standard image and the reference image from the standard image and the scale-converted reference image obtained in S307. Then, the control unit 201 may obtain, as angle information, an angle between a straight line connecting the two corresponding points in the reference image and a straight line connecting the two corresponding points in the reference image. Alternatively, the control unit 201 may repeatedly select two sets of corresponding points and calculate the angle formed by the straight line connecting the corresponding points, and then calculate the angle between the two images by summing up the obtained angles. A representative value representing the difference in angle may be acquired as the angle information. Note that the representative value representing the plurality of angles may be, for example, a statistically determined value such as an average value, a median value, or an intermediate value.

Furthermore, the angle information according to the embodiment may be acquired using other methods. For example, in another embodiment, the control unit 201 detects a straight line area in the reference image and generates a histogram that totals the angles of the detected straight lines. Furthermore, the control unit 201 similarly detects straight line areas from the reference image and generates a histogram that totals the angles of the detected straight lines.

FIG. 10 is a diagram illustrating a histogram in which linear regions are detected from an image and angles are totaled. As shown in FIG. 10, in a histogram, the horizontal axis is the angle (for example, 0° to 360°) that a straight line area makes with respect to a predetermined axis direction such as the vertical or horizontal direction of the image, and the vertical axis is , is the number of straight lines having that angle. Then, the control unit 201 gradually shifts the angle of one of the histograms of the standard image or the histogram of the reference image, superimposes them on the other histogram, calculates the correlation, and uses the angle at which the correlation is highest as the angle information. It's fine.

In yet another embodiment, the control unit 201 performs two-dimensional Fourier transform on the object area for each of the standard image and the scale-converted reference image obtained in S307 to obtain a two-dimensional Fourier transformed image. . Then, the control unit 201 gradually rotates one of the two-dimensional Fourier-transformed image obtained from the standard image and the two-dimensional Fourier-transformed image obtained from the reference image, and superimposes it on the other two-dimensional Fourier-transformed image. At the same time, the sum of absolute differences and the sum of squared residuals of pixel values in the overlapping area are determined. Then, the control unit 201 may use the angle at which the calculated sum of absolute differences or sum of squared residuals is the minimum as the angle information.

Subsequently, in S309, the control unit 201 converts the reference image based on the obtained angle information. For example, the control unit 201 rotates the reference image using the angle information, and converts the arrangement of the object in the reference image to match the arrangement of the object in the reference image.

FIG. 11 is a diagram illustrating the rotation of the reference image in the process of S309. FIG. 11(a) shows the object area of the reference image, and FIG. 11(b) shows an example of the object area of the reference image after rotation by the angle of the angle information. There is. As shown in FIG. 11, the object area shown in the reference image has been changed to have approximately the same orientation as the reference image shown in FIG. 9(a). By arranging the orientation of the object in the images in this way, it is possible, for example, to facilitate the user's task of searching for an image in which the object is shown under desirable conditions.

In S310, the control unit 201 evaluates the sharpness of the reference image and outputs an evaluation value. For example, the control unit 201 applies a filter for extracting edges, such as a Laplacian filter or a Sobel filter, to the object region in the obtained converted image, and generates an edge image in which edges are extracted from the object region. Then, the control unit 201 may calculate the variance, the root mean square, etc. of the pixel values from the obtained edge image, and use the values of the variance and the root mean square as the sharpness evaluation value. For example, if an image in which the object is small is scaled up in the process of S307, or if the object is out of focus in the original image, the edges in the generated edge image will be blurred and extracted. be done. On the other hand, sharp edges are extracted from images where the object is in focus and the image is clear. In an edge image, the variance and root mean square of pixel values are higher when the edges are clear than when the edges are blurred. Therefore, it is possible to evaluate the quality of an object included in an image by using the variance, root mean square, or the like of the pixel values of an edge image as a sharpness evaluation value.

Subsequently, in S311, the control unit 201 determines whether processing has been completed for all images to be evaluated. If there are still unprocessed images other than the reference image, the flow returns to S303, selects the unprocessed images, and repeats the process. On the other hand, if processing has been completed for all images other than the reference image, the flow advances to S312.

In S312, the control unit 201 evaluates the sharpness of the reference image and outputs an evaluation value. Note that the sharpness evaluation value may be obtained in the same manner as described in S310, for example.

In S313, the control unit 201 outputs the image evaluation result based on the sharpness evaluation value, and this operation flow ends. For example, the control unit 201 may generate the evaluation result information 1200 by registering the sharpness evaluation value output in the processing of S310 and S312 in the evaluation result information 1200, and may output the evaluation result information 1200.

FIG. 12 is a diagram illustrating evaluation result information 1200 according to the embodiment. In the evaluation result information 1200, records related to a plurality of images including a reference image and a plurality of reference images whose sharpness was evaluated according to the operation flow of FIG. 3 are registered. In the example of FIG. 12, for example, an image identifier, a sharpness evaluation value, and an image are associated with each other in a record. The image identifier is, for example, information that identifies an image corresponding to a record. Further, the evaluation value may be, for example, a sharpness evaluation value obtained for an image corresponding to the record. For example, data of an image corresponding to a record may be registered in the image of the evaluation result information 1200. For example, the user can easily find useful images that clearly depict the object by referring to the information output to the evaluation result information 1200 and identifying images with high evaluation values.

Note that in the process of S<b>313 , the control unit 201 may select some of the images based on the evaluation value from among the plurality of images to be evaluated, and register the records in the evaluation result information 1200 . In one example, the control unit 201 selects a target object from among a plurality of images to be evaluated, such as a predetermined number of images with high sharpness evaluation values and images with sharpness evaluation values equal to or higher than a predetermined threshold. A clear reference image that satisfies a predetermined condition may be extracted and registered in the evaluation result information 1200. In addition, in the processing of S313, the control unit 201 controls, among the plurality of images to be evaluated, for images in which the correspondence of feature points is less than a certain level (for example, the number of corresponding points is less than a predetermined number) in S305, the control unit 201 records There is no need to register.

Furthermore, in the process of S313, the control unit 201 registers in the evaluation result information 1200, for example, an image in which the size of the object area included in the reference image after the scale conversion in the process of S307 is less than or equal to a predetermined size. You don't have to. As a result, if the object area shown in the image is narrow, even if the image is clear, it can be removed without being registered in the evaluation result information 1200. Alternatively, in the process of S313, the control unit 201 does not need to register the image in the evaluation result information 1200, for example, if the angle information is equal to or greater than the predetermined angle in the process of S308. As a result, for example, if the object area in the image is not taken from the front, but from an angle exceeding the allowable range, the evaluation result will be It can be removed without being registered in the information 1200.

In another embodiment, in the process of S313, the control unit 201 may, for example, rearrange the records in descending order of sharpness evaluation value and output the evaluation result information 1200.

Furthermore, for example, when outputting the evaluation result information 1200 by sorting records in descending order of sharpness evaluation values, suppose that there are records whose difference in sharpness evaluation value is close within a predetermined error range. . In this case, the control unit 201 may, for example, output the evaluation result information 1200 by rearranging those records in the process of S307 so that the image with the large size of the object area after conversion is placed at the top. good. Alternatively, for example, for records in which the difference in sharpness evaluation values is within a predetermined error range, the control unit 201 may prioritize images in which the angle of the angle information is less than a predetermined threshold and the object area is photographed from the front. The evaluation result information 1200 may be output by rearranging the evaluation result information 1200 so that it is arranged in the following manner. By doing the above, it becomes possible to arrange images for which more information can be collected or images that are easier to see for images with little difference in sharpness at the top and provide them to the user.

As described above, by carefully selecting and rearranging the data to be output as the evaluation result information 1200 in accordance with the conditions 1 to 3 above, the user can select the target object from the evaluation result information 1200. It is possible to more easily identify images that look good.

Note that in S313, the control unit 201 may, for example, output the evaluation result information 1200 to the storage unit 202, or in another embodiment, output display information for displaying the evaluation result information 1200, and output the evaluation result information 1200 to the display device. It may be displayed on the display screen.

Furthermore, the image registered in the evaluation result information 1200 may be the original image used as the evaluation target, as illustrated in FIG. The image may be an image after the conversion or an image after the angle is further converted. By registering and managing images whose scale and angle have been converted in the evaluation result information 1200, users can obtain easy-to-use images that have been converted to suit a specific purpose, such as restoring a collapsed stone wall of a castle. can be obtained from the evaluation result information 1200.

As described above, according to the embodiment, it is possible to identify a useful image that clearly depicts a target object from among a huge number of images. Therefore, for example, when we ask the general public to provide images in order to restore a collapsed stone wall of a castle, and a huge amount of images are collected, we can choose from among the images provided to repair the stone wall. It becomes possible to easily extract a good-looking image of a stone wall.

Although the embodiments have been illustrated above, the embodiments are not limited thereto. For example, the above-described operational flow is an example, and the embodiments are not limited thereto. If possible, the operational flow may be executed by changing the order of processing, may include additional processing, or may omit some processing. For example, in another embodiment, the processes of S308 and S309 in FIG. 3 may not be performed. Even if the reference image is not rotated in the processes of S308 and S309, as long as the scale is correct, the sharpness evaluation value can be obtained with high accuracy, and the effects of the above-described embodiment can be obtained.

In another embodiment, after identifying the corresponding points in the process of S305, a plurality of corresponding points (for example, four or more corresponding points) appearing in the reference image match with a plurality of corresponding points in the reference image. A homography matrix for transforming the reference image may be obtained, and the reference image may be perspectively transformed. In this case, the processes from S306 to S309 described above may be replaced with a process of obtaining a homography matrix and perspectively transforming the reference image.

Further, in the above-described embodiment, an example is described in which one sharpness evaluation value is obtained for a target object area in an image in the processing of S310 and S312, but the embodiment is not limited to this. isn't it. For example, in another embodiment, the control unit 201 determines the degree of sharpness of the image of the object for each of a plurality of sections obtained by dividing the object area in the reference image after the scale is converted. You may evaluate and output the evaluation value. For example, in an image of an object taken from an oblique angle rather than from the front, the sharpness may vary depending on the location of the object region within the image. In this case, a situation may occur in which useful image information about the object can be obtained for some regions within the object region, but no useful information can be obtained for other regions. Therefore, by dividing the object area of the image into multiple sections and evaluating the sharpness of each section, it is possible to obtain partially useful information based on the evaluation value. It becomes possible to extract.

Note that in the above-described embodiment, the control unit 201 operates as the specifying unit 211 in S305 and the process of FIG. 4, for example. Furthermore, in the processing of S306 and S308, the control unit 201 operates as the determining unit 212, for example. In the processing of S307 and S309, the control unit 201 operates as the changing unit 213, for example. In the processes of S310, S312, and S313, the control unit 201 operates as, for example, the output unit 214.

FIG. 13 is a diagram illustrating a hardware configuration of an information processing device 1300 such as a computer for realizing the image evaluation device 200 according to the embodiment. A hardware configuration for realizing the image evaluation device 200 in FIG. 13 includes, for example, a processor 1301, a memory 1302, a storage device 1303, a reading device 1304, a communication interface 1306, and an input/output interface 1307. Note that the processor 1301, memory 1302, storage device 1303, reading device 1304, communication interface 1306, and input/output interface 1307 are connected to each other via a bus 1308, for example.

The processor 1301 may be, for example, a single processor, a multiprocessor, or a multicore. The processor 1301 provides some or all of the functions of the control unit 201 described above by using the memory 1302 to execute, for example, a program that describes the procedure of the operation flow described above. For example, the processor 1301 of the image evaluation device 200 reads and executes a program stored in the storage device 1303, thereby operating as the specifying unit 211, the determining unit 212, the changing unit 213, and the output unit 214.

Memory 1302 is, for example, a semiconductor memory and may include a RAM area and a ROM area. The storage device 1303 is, for example, a hard disk, a semiconductor memory such as a flash memory, or an external storage device. Note that RAM is an abbreviation for Random Access Memory. Further, ROM is an abbreviation for Read Only Memory.

Reader 1304 accesses removable storage medium 1305 according to instructions from processor 1301 . The removable storage medium 1305 is realized by, for example, a semiconductor device (USB memory, etc.), a medium (magnetic disk, etc.) in which information is input/output by magnetic action, a medium in which information is input/output by optical action, etc. . Note that USB is an abbreviation for Universal Serial Bus. Further, media on which information is input/output by optical action include, for example, CD-ROM, DVD, Blu-ray Disc, etc. (Blu-ray is a registered trademark). CD is an abbreviation for Compact Disc. DVD is an abbreviation for Digital Versatile Disk.

The storage unit 202 described above may include, for example, a memory 1302, a storage device 1303, and a removable storage medium 1305. For example, the storage device 1303 of the image evaluation device 200 stores information such as a standard image, a reference image, image feature information 700, and evaluation result information 1200.

Communication interface 1306 communicates with other devices according to instructions from processor 1301, for example. In one example, communication interface 1306 may send and receive data to and from other devices over a network via wired or wireless communications.

The input/output interface 1307 may be, for example, an interface between an input device and an output device. The input device is, for example, a device such as a keyboard, mouse, or touch panel that receives instructions from a user. The output device is, for example, a display device such as a display, and a printing device such as a printer. In the embodiment described above, the processor 1301 may, for example, cause the display device connected to the input/output interface 1307 to display a display screen that displays the evaluation result information 1200 and the like. Further, the processor 1301 may, for example, cause a printing device connected to the input/output interface 1307 to output a medium such as paper on which the evaluation result information 1200 and the like are printed.

Each program according to the embodiment is provided to the image evaluation device 200 in the following format, for example.
(1) Installed in the storage device 1303 in advance.
(2) Provided by removable storage medium 1305;
(3) Provided by a server such as a program server.

Note that the hardware configuration of the information processing device 1300 for realizing the image evaluation device 200 described with reference to FIG. 13 is an example, and the embodiment is not limited to this. For example, some of the configurations described above may be deleted, or new configurations may be added. In another embodiment, for example, some or all of the functions of the control unit 201 described above may be implemented as hardware such as an FPGA, an SoC, an ASIC, and a PLD. Note that FPGA is an abbreviation for Field Programmable Gate Array. SoC is an abbreviation for System-on-a-chip. ASIC is an abbreviation for Application Specific Integrated Circuit. PLD is an abbreviation for Programmable Logic Device.

Above, several embodiments are described. However, the embodiments are not limited to the embodiments described above, but should be understood to include various modifications and alternative forms of the embodiments described above. For example, it will be understood that the various embodiments can be embodied by changing the components without departing from the spirit and scope thereof. Furthermore, it will be understood that various embodiments can be implemented by appropriately combining the plurality of components disclosed in the embodiments described above. Furthermore, various embodiments may be implemented by deleting some components from all the components shown in the embodiments or adding some components to the components shown in the embodiments. It will be understood by those skilled in the art.

Regarding the above embodiments, the following additional notes are further disclosed.
(Additional note 1)
Based on the feature amount that characterizes the feature points from among a group of feature points identified in the area where the target object is captured in the reference image and a plurality of feature points identified in the area where the target object is captured in the reference image. a specifying unit that specifies corresponding points that are a set of corresponding feature points between the reference image and the reference image;
Based on the arrangement of the plurality of corresponding points specified in the reference image and the arrangement of the plurality of corresponding points specified in the reference image, the object reflected in the reference image and the object reflected in the reference image a determining unit that determines the difference in scale from the object;
a changing unit that changes the scale of the reference image to match the scale of the object reflected in the reference image based on the difference in scale;
an output unit that outputs an evaluation value obtained by evaluating the degree of sharpness of the image of the object using the reference image changed by the changing unit;
An image evaluation device including:
(Additional note 2)
The output unit outputs the evaluation values for a plurality of reference images including the reference image, and determines whether the image of the object satisfies a predetermined condition and is clear based on the evaluation values output for the plurality of reference images. The image evaluation device according to supplementary note 1, which outputs information regarding a reference image.
(Additional note 3)
The identification unit further determines, for each of the plurality of divisions obtained by dividing the reference image, an image parameter representing a feature of the image in the division, and determines whether the image parameter satisfies a predetermined similarity condition. The image evaluation device according to appendix 1 or 2, wherein a group satisfying a predetermined determination condition is identified as a region in which the target object is captured from among the groups obtained by grouping similar sections.
(Additional note 4)
The output unit evaluates the degree of sharpness of the image of the object for each of a plurality of partitions obtained by dividing the area in which the object appears in the reference image after the change by the change unit. output the evaluated value,
The image evaluation device according to any one of Supplementary Notes 1 to 3, characterized in that:
(Appendix 5)
The determining unit further determines the arrangement of the object in the reference image based on the arrangement of the plurality of corresponding points specified in the reference image and the arrangement of the plurality of corresponding points specified in the reference image. , determine the angle formed by the arrangement of the object reflected in the reference image,
The changing unit further rotates the reference image based on the angle so that the arrangement of the object in the reference image approaches the arrangement of the object in the reference image.
The image evaluation device according to any one of Supplementary Notes 1 to 4, characterized in that:
(Appendix 6)
Based on the feature amount that characterizes the feature points from among a group of feature points identified in the area where the target object is captured in the reference image and a plurality of feature points identified in the area where the target object is captured in the reference image. identifying corresponding points that are a set of corresponding feature points between the reference image and the reference image;
Based on the arrangement of the plurality of corresponding points specified in the reference image and the arrangement of the plurality of corresponding points specified in the reference image, the object reflected in the reference image and the object reflected in the reference image Determine the difference in scale from the object,
Based on the difference in scale, changing the scale of the reference image to match the scale of the object reflected in the reference image;
outputting an evaluation value obtained by evaluating the degree of sharpness of the object in the reference image after the scale change;
An image evaluation program that causes a computer to perform processing.
(Appendix 7)
The outputting process includes outputting the evaluation values for a plurality of reference images including the reference image, and determining whether the image of the object satisfies a predetermined condition based on the evaluation values output for the plurality of reference images. The image evaluation program according to appendix 6, which outputs information regarding a clear reference image.
(Appendix 8)
The identifying process further includes determining, for each of the plurality of sections obtained by dividing the reference image, an image parameter representing the feature of the image in the section, and determining whether the image parameter satisfies a predetermined similarity condition. The image evaluation program according to appendix 6 or 7, which specifies a group that satisfies a predetermined determination condition from among groups obtained by grouping similar sections that satisfy the same as a region in which the target object is photographed.
(Appendix 9)
In the outputting process, the degree of sharpness of the image of the object is evaluated for each of a plurality of sections obtained by dividing the area in which the object is captured in the reference image after the scale has been changed. output the evaluation value,
9. The image evaluation program according to any one of appendices 6 to 8, characterized in that:
(Appendix 10)
The determining process further includes determining the target object in the reference image based on the arrangement of the plurality of corresponding points specified in the reference image and the arrangement of the plurality of corresponding points specified in the reference image. determining an angle formed by the arrangement of the object reflected in the reference image with respect to the arrangement;
The changing process further includes rotating the reference image based on the angle so that the arrangement of the object in the reference image approaches the arrangement of the object in the reference image.
The image evaluation program according to any one of appendices 6 to 9, characterized in that:

200: Image evaluation device 201: Control unit 202: Storage unit 211: Specification unit 212: Determining unit 213: Changing unit 214: Output unit 700: Image feature information 1200: Evaluation result information 1300: Information processing device 1301: Processor 1302: Memory 1303: Storage device 1304: Reading device 1305: Removable storage medium 1306: Communication interface 1307: Input/output interface 1308: Bus

Claims (6)

Based on the feature amount that characterizes the feature points from among a group of feature points identified in the area where the target object is captured in the reference image and a plurality of feature points identified in the area where the target object is captured in the reference image. a specifying unit that specifies corresponding points that are a set of corresponding feature points between the reference image and the reference image;
Based on the arrangement of the plurality of corresponding points specified in the reference image and the arrangement of the plurality of corresponding points specified in the reference image, the object reflected in the reference image and the object reflected in the reference image a determining unit that determines the difference in scale from the object;
a changing unit that changes the scale of the reference image to match the scale of the object reflected in the reference image based on the difference in scale;
an output unit that outputs an evaluation value obtained by evaluating the degree of sharpness of the image of the object using the reference image changed by the changing unit;
An image evaluation device including: The output unit outputs the evaluation values for a plurality of reference images including the reference image, and determines whether the image of the object satisfies a predetermined condition and is clear based on the evaluation values output for the plurality of reference images. The image evaluation device according to claim 1, which outputs information regarding a reference image. The identification unit further determines, for each of the plurality of divisions obtained by dividing the reference image, an image parameter representing a feature of the image in the division, and determines whether the image parameter satisfies a predetermined similarity condition. 3. The image evaluation device according to claim 1, wherein a group that satisfies a predetermined determination condition is identified from among the groups obtained by grouping similar sections as a region in which the target object is photographed. The output unit evaluates the degree of sharpness of the image of the object for each of a plurality of partitions obtained by dividing the area in which the object appears in the reference image after the change by the change unit. output the evaluated value,
The image evaluation device according to any one of claims 1 to 3, characterized in that: The determining unit further determines the arrangement of the object in the reference image based on the arrangement of the plurality of corresponding points specified in the reference image and the arrangement of the plurality of corresponding points specified in the reference image. , determine the angle formed by the arrangement of the object reflected in the reference image,
The changing unit further rotates the reference image based on the angle so that the arrangement of the object in the reference image approaches the arrangement of the object in the reference image.
The image evaluation device according to any one of claims 1 to 4, characterized in that: Based on the feature amount that characterizes the feature points from among a group of feature points identified in the area where the target object is captured in the reference image and a plurality of feature points identified in the area where the target object is captured in the reference image. identifying corresponding points that are a set of corresponding feature points between the reference image and the reference image;
Based on the arrangement of the plurality of corresponding points specified in the reference image and the arrangement of the plurality of corresponding points specified in the reference image, the object reflected in the reference image and the object reflected in the reference image Determine the difference in scale from the object,
Based on the difference in scale, changing the scale of the reference image to match the scale of the object reflected in the reference image;
outputting an evaluation value obtained by evaluating the degree of sharpness of the object in the reference image after the scale change;
A program that causes a computer to perform a process.

Images