JP4830950B2 - Information processing method, information processing apparatus, and program - Google Patents

Description

  The present invention relates to an information processing method, an information processing apparatus, and a program.

  Some digital still cameras have a mode setting dial for setting a shooting mode. When the user sets the shooting mode with the dial, the digital still camera determines shooting conditions (such as exposure time) according to the shooting mode and performs shooting. When shooting is performed, the digital still camera generates an image file. In this image file, additional data such as shooting conditions at the time of shooting is added to the image data of the shot image.

On the other hand, image processing is also performed on image data according to additional data. For example, when a printer performs printing based on an image file, the image data is corrected according to the shooting conditions indicated by the additional data, and printing is performed according to the corrected image data.
JP 2001-238177 A

When the digital still camera generates an image file, scene information corresponding to dial settings may be stored in the additional data. On the other hand, if the user forgets to set the shooting mode, scene information that does not match the content of the image data is stored in the additional data. For this reason, the scene of the image data may be identified by analyzing the image data without using the scene information of the additional data.
By the way, when the scene indicated by the additional data does not match the scene of the identification result, a confirmation screen may be displayed to the user. If the process is advanced after the user confirms the confirmation screen, the time until the process is started becomes longer.
An object of the present invention is to speed up processing while displaying a confirmation screen.

A main invention for achieving the above object is to obtain scene information of image data from additional data added to image data, identify an image scene indicated by the image data based on the image data, and An information processing method for comparing a scene indicated by information with an identified scene and displaying a confirmation screen when there is a mismatched image in which the scene indicated by the scene information does not match the identified scene, wherein the mismatched image Before the image processing is performed on the image, the image processing is started on the matching image in which the scene indicated by the scene information and the identified scene are not inconsistent.
Other features of the present invention will become apparent from the description of the present specification and the accompanying drawings.

  At least the following matters will become clear from the description of the present specification and the accompanying drawings.

From the additional data added to the image data, obtain the scene information of the image data,
Based on the image data, the scene of the image indicated by the image data is identified,
Comparing the scene indicated by the scene information with the identified scene;
An information processing method for displaying a confirmation screen when there is a mismatch image in which the scene indicated by the scene information and the identified scene do not match,
An information processing method is characterized in that before image processing is performed on the unmatched image, image processing is started on a matched image in which the scene indicated by the scene information and the identified scene do not match. .
According to such an information processing method, image processing can be started quickly.

  Further, it is desirable that image processing is performed on the matching image when the confirmation screen is displayed. Thereby, image processing can be started early.

  Further, before displaying the confirmation screen, create a job for the matching image, and after displaying the confirmation screen, create a job for the non-matching image, and the processing device, according to the priority of the job, It is desirable to execute the job. Thereby, image processing can be started early.

  In addition, it is desirable to change the priority of the job after creating a job for the mismatched image. As a result, the number of image files subjected to image processing in numerical order can be increased.

  In addition, if the job can be executed in the order of a plurality of image data after creating the job for the mismatched image, the priority order of the job is changed and the job is executed in the order of the plurality of image data. If not, it is desirable not to change the priority of the job. Thereby, a job can be executed in the order of a plurality of image data.

  In addition, it is desirable to display a warning screen if the job cannot be executed in the order of the plurality of image data after the job for the mismatched image is created. Thereby, a user can be alerted.

An information processing apparatus comprising a controller,
The controller is
From the additional data added to the image data, obtain the scene information of the image data,
Based on the image data, the scene of the image indicated by the image data is identified,
Comparing the scene indicated by the scene information with the identified scene;
When there is a mismatch image in which the scene indicated by the scene information does not match the identified scene, a confirmation screen is displayed,
Before the image processing is performed on the unmatched image, an information processing apparatus is disclosed that starts image processing on a matched image in which the scene indicated by the scene information and the identified scene do not match. .
According to such an information processing apparatus, image processing can be started quickly.

In the information processing device,
From the additional data added to the image data, the scene information of the image data is acquired,
Based on the image data, the scene of the image indicated by the image data is identified,
Comparing the scene indicated by the scene information with the identified scene;
When there is a mismatch image in which the scene indicated by the scene information and the identified scene do not match, a confirmation screen is displayed,
Before the image processing is performed on the unmatched image, a program that starts image processing on a matched image in which the scene indicated by the scene information and the identified scene do not match is clarified.
According to such a program, image processing can be started quickly.

=== Overall structure ===
FIG. 1 is an explanatory diagram of an image processing system. This image processing system includes a digital still camera 2 and a printer 4.

The digital still camera 2 is a camera that acquires a digital image by forming an image of a subject on a digital device (CCD or the like). The digital still camera 2 is provided with a mode setting dial 2A. The user can set the shooting mode according to the shooting conditions by using the dial 2A. For example, when the “night scene” mode is set by the dial 2A, the digital still camera 2 performs shooting under shooting conditions suitable for night scene shooting by decreasing the shutter speed or increasing the ISO sensitivity.
The digital still camera 2 stores an image file generated by photographing in the memory card 6 in accordance with the file format standard. In the image file, not only digital data (image data) of a captured image but also additional data such as a shooting condition (shooting data) at the time of shooting is stored.

  The printer 4 is a printing device that prints an image indicated by image data on paper. The printer 4 is provided with a slot 21 into which the memory card 6 is inserted. The user can take a picture with the digital still camera 2, remove the memory card 6 from the digital still camera 2, and insert the memory card 6 into the slot 21.

  The panel unit 15 includes a display unit 16 and an input unit 17 having various buttons. The display unit 16 is configured by a liquid crystal display. If the display unit 16 is a touch panel, the display unit 16 also functions as the input unit 17. The display unit 16 displays a setting screen for setting the printer 4, an image of image data read from the memory card, a screen for confirmation and warning to the user, and the like. Various screens displayed by the display unit 16 will be described later.

  FIG. 2 is an explanatory diagram of the configuration of the printer 4. The printer 4 includes a printing mechanism 10 and a printer-side controller 20 that controls the printing mechanism 10. The printing mechanism 10 includes a head 11 that ejects ink, a head control unit 12 that controls the head 11, a motor 13 for conveying paper, and a sensor 14. The printer-side controller 20 includes a memory slot 21 for transmitting and receiving data from the memory card 6, a CPU 22, a memory 23, a control unit 24 for controlling the motor 13, and a drive signal for generating a drive signal (drive waveform). And a generation unit 25. The printer-side controller 20 also includes a panel control unit 26 that controls the panel unit 15.

  When the memory card 6 is inserted into the slot 21, the printer-side controller 20 reads out the image file stored in the memory card 6 and stores it in the memory 23. Then, the printer-side controller 20 converts the image data of the image file into print data for printing by the printing mechanism 10, controls the printing mechanism 10 based on the printing data, and prints an image on paper. This series of operations is called “direct printing”.

  “Direct printing” is not only performed by inserting the memory card 6 into the slot 21, but also by connecting the digital still camera 2 and the printer 4 with a cable (not shown). Further, the panel unit 15 is used for setting direct printing (described later). The panel unit 15 is also used to display a confirmation screen during direct printing and to input confirmation (described later).

=== Image File Structure ===
The image file is composed of image data and additional data. The image data is composed of a plurality of pixel data. The pixel data is data indicating pixel color information (gradation value). An image is formed by arranging the pixels in a matrix. Therefore, the image data is data indicating an image. The additional data includes data indicating the characteristics of image data, shooting data, thumbnail image data, and the like.

Hereinafter, a specific structure of the image file will be described.
FIG. 3 is an explanatory diagram of the structure of the image file. The left side of the figure shows the overall configuration of the image file, and the right side shows the configuration of the APP1 area.

  The image file starts with a marker indicating SOI (Start of image) and ends with a marker indicating EOI (End of Image). After the marker indicating SOI, there is an APP1 marker indicating the start of the data area of APP1. The APP1 data area after the APP1 marker includes additional data such as shooting data and thumbnail images. In addition, image data is included after the marker indicating SOS (Start of Stream).

  After the APP1 marker, there is information indicating the size of the data area of APP1, followed by an EXIF header and a TIFF header, which becomes an IFD area.

  Each IFD area has a plurality of directory entries, a link indicating the position of the next IFD area, and a data area. For example, in the first IFD0 (IFD of main image), the position of the next IFD1 (IFD of thumbnail image) is linked. However, since there is no IFD next to IFD 1 here, IFD 1 does not link to another IFD. Each directory entry includes a tag and a data portion. When the amount of data to be stored is small, the actual data is stored as it is in the data portion. When the amount of data is large, the actual data is stored in the IFD0 data area, and the data portion indicates the data storage location. A pointer is stored. The IFD 0 includes a directory entry in which a tag (Exif IFD Pointer) indicating the storage location of the Exif Sub IFD and a pointer (offset value) indicating the storage location of the Exif Sub IFD are stored.

  The Exif SubIFD area has a plurality of directory entries. This directory entry also includes a tag and a data portion. When the amount of data to be stored is small, the actual data is stored as it is in the data portion, and when the amount of data is large, the actual data is stored in the ExifSubIFD data area, and the data portion indicates the data storage location. A pointer is stored. In the ExifSubIFD, a tag indicating the storage location of the Makernote IFD and a pointer indicating the storage location of the Makernote IFD are stored.

  The Makernote IFD area has a plurality of directory entries. This directory entry also includes a tag and a data portion. When the amount of data to be stored is small, the actual data is stored as it is in the data portion. When the amount of data is large, the actual data is stored in the Makernote IFD data area, and the data portion stores the data. Is stored. However, since the data storage format can be freely defined in the Makernote IFD area, it is not always necessary to store data in this format. In the following description, data stored in the Makernote IFD area is referred to as “MakerNote data”.

  FIG. 4A is an explanatory diagram of tags used in IFD0. As shown in the drawing, general data (data indicating the characteristics of image data) is stored in IFD0, and detailed photographing data is not stored.

  FIG. 4B is an explanatory diagram of tags used in the Exif SubIFD. As shown in the figure, Exif SubIFD stores detailed shooting data. Note that most of the shooting data extracted in the scene identification process is shooting data stored in the Exif SubIFD. The shooting scene type tag (Scene Capture Type) is a tag indicating the type of shooting scene. Further, the Makernote tag is a tag indicating the storage location of the Makernote IFD.

  If the data portion (shooting scene type data) for the shooting scene type tag in the Exif SubIFD area is “zero”, it means “standard”, “1” means “landscape”, and “2” means “person”. "3" means "night view". Since the data stored in the Exif SubIFD is standardized, anyone can know the contents of the shooting scene type data.

  In the present embodiment, shooting mode data is included in one of the Makernote data. In this shooting mode data, a different value is shown for each mode set by the mode setting dial 2A. However, since the MakerNote data has a different format for each manufacturer, the contents of the shooting mode data cannot be known unless the format of the MakerNote data is known.

  FIG. 5 is a correspondence table between the setting of the mode setting dial 2A and data. Since the shooting scene type tag used in the Exif SubIFD conforms to the file format standard, the scenes that can be specified are limited, and data for specifying a scene such as “evening scene” cannot be stored in the data portion. On the other hand, since the MakerNote data can be freely defined, the data specifying the shooting mode of the mode setting dial 2A can be stored in the data portion by the shooting mode tag which is one of the MakerNote data.

  The above-described digital still camera 2 shoots under shooting conditions corresponding to the setting of the mode setting dial 2 </ b> A, then creates the above image file and stores it in the memory card 6. In this image file, shooting scene type data and shooting mode data corresponding to the mode setting dial 2A are stored in the Exif SubIFD area and the Makernote IFD area, respectively, as scene information added to the image data.

=== Outline of automatic correction function ===
When printing a “person” photo, there is a demand to clean the skin tone. In addition, when printing a “landscape” photograph, there is a demand for emphasizing the blue of the sky and the green of trees and grass. Therefore, the printer 4 of the present embodiment includes an automatic correction function that analyzes an image file and automatically performs a suitable correction process.

  FIG. 6 is an explanatory diagram of the automatic correction function of the printer 4. Each element of the printer-side controller 20 in the figure is realized by software and hardware.

  The storage unit 31 is realized by a partial area of the memory 23 and the CPU 22. All or part of the image file read from the memory card 6 is developed in the image storage unit 31 </ b> A of the storage unit 31. In addition, the calculation result of each element of the printer-side controller 20 is stored in the result storage unit 31B of the storage unit 31.

  The face identification unit 32 is realized by the CPU 22 and a face identification program stored in the memory 23. The face identification unit 32 analyzes the image data stored in the image storage unit 31A and identifies the presence or absence of a face. When the face identifying unit 32 identifies that there is a face, the image to be identified is identified as belonging to the “person” scene. Since the face identification process by the face identification unit 32 is the same as a process that has already been widely performed, detailed description thereof is omitted.

  The face identifying unit 32 also calculates the probability (certainty factor) that the image to be identified belongs to the scene of “person”. This certainty factor is calculated from the ratio of the skin color pixels in the image, the shape of the skin color image, the degree of proximity between the color indicated by the pixel data and the memory color of the skin color, and the like. The identification result of the face identification unit 32 is stored in the result storage unit 31B.

  The scene identification unit 33 is realized by the CPU 22 and a scene identification program stored in the memory 23. The scene identification unit 33 analyzes the image file stored in the image storage unit 31A and identifies the scene of the image indicated by the image data. After the face identifying process by the face identifying unit 32, the scene identifying process by the scene identifying unit 33 is performed. As will be described later, the scene identification unit 33 identifies whether the image to be identified is an image of “landscape”, “evening scene”, “night scene”, “flower”, or “autumn leaves”. Note that the identification result of the scene identification unit 33 and information on the certainty factor are also stored in the result storage unit 31B.

FIG. 7 is an explanatory diagram of a relationship between an image scene and correction contents.
The image correction unit 34 is realized by the CPU 22 and an image correction program stored in the memory 23. The image correction unit 34 converts the image data of the image storage unit 31A based on the identification results (identification results of the face identification unit 32 and the scene identification unit 33) stored in the result storage unit 31B (described later) of the storage unit 31. to correct. For example, when the identification result of the scene identification unit 33 is “landscape”, correction is performed so that blue is emphasized and green is emphasized. However, when the scene indicated by the additional data of the image file and the scene indicated by the identification result do not match, the image correction unit 34 corrects the image data according to the confirmation result after performing a predetermined confirmation process described later.

  The printer control unit 35 is realized by a printer control program stored in the CPU 22, the drive signal generation unit 25, the control unit 24, and the memory 23. The printer control unit 35 converts the corrected image data into print data, and causes the printing mechanism 10 to print the image.

=== Scene Identification Processing ===
FIG. 8 is a flowchart of scene identification processing by the scene identification unit 33. FIG. 9 is an explanatory diagram of the function of the scene identification unit 33. Each element of the scene identification unit 33 in the figure is realized by software and hardware. The scene identification unit 33 includes a feature amount acquisition unit 40, an overall classifier 50, a partial classifier 60, and an integrated classifier 70 shown in FIG.

First, the feature amount acquisition unit 40 analyzes the image data developed in the image storage unit 31A of the storage unit 31 and acquires partial feature amounts (S101). Specifically, the feature amount acquisition unit 40 divides the image data into 8 × 8 64 blocks, calculates the color average and variance of each block, and acquires the color average and variance as partial feature amounts. Here, each pixel has gradation value data in the YCC color space, and the average value of Y, the average value of Cb, and the average value of Cr are calculated for each block, and the variance of Y and the variance of Cb are calculated. And the variance of Cr are calculated respectively. That is, three color averages and three variances are calculated as partial feature amounts for each block. These color averages and variances indicate the characteristics of the partial images in each block. Note that an average value or variance in the RGB color space may be calculated.
Since the color average and variance are calculated for each block, the feature amount acquisition unit 40 expands the image data for the blocks in the block order without expanding all the image data in the image storage unit 31A. For this reason, the image storage unit 31A does not necessarily have a capacity sufficient to expand all of the image files.

  Next, the feature amount acquisition unit 40 acquires the entire feature amount (S102). Specifically, the feature quantity acquisition unit 40 acquires the overall color average, variance, center of gravity, and shooting information of the image data as the overall feature quantity. Note that these color averages and variances indicate the overall characteristics of the image. The color average, variance, and center of gravity of the entire image data are calculated using the partial feature values calculated previously. For this reason, it is not necessary to re-expand the image data when calculating the entire feature amount, and the calculation speed of the entire feature amount is increased. Although the overall identification process (described later) is performed prior to the partial identification process (described later), the overall feature value is obtained after the partial feature value in order to increase the calculation speed. The shooting information is extracted from the shooting data of the image file. Specifically, information such as the aperture value, shutter speed, and the presence or absence of flash emission is used as the overall feature amount. However, not all shooting data of the image file is used as the entire feature amount.

  Next, the overall classifier 50 performs overall identification processing (S103). The overall identification process is a process for identifying (estimating) an image scene indicated by image data based on the overall feature amount. Details of the overall identification process will be described later.

  If the scene can be identified by the overall identification process (YES in S104), the scene identification unit 33 determines the scene by storing the identification result in the result storage unit 31B of the storage unit 31 (S109), and performs the scene identification process. finish. That is, when the scene can be identified by the overall identification process (YES in S104), the partial identification process and the integrated identification process are omitted. This increases the speed of the scene identification process.

  If the scene cannot be identified by the overall identification process (NO in S104), the partial classifier 60 performs the partial identification process (S105). The partial identification process is a process for identifying the scene of the entire image indicated by the image data based on the partial feature amount. Details of the partial identification processing will be described later.

  When the scene can be identified by the partial identification process (YES in S106), the scene identification unit 33 determines the scene by storing the identification result in the result storage unit 31B of the storage unit 31 (S109), and performs the scene identification process. finish. That is, when the scene can be identified by the partial identification process (YES in S106), the integrated identification process is omitted. This increases the speed of the scene identification process.

  If the scene cannot be identified by the partial identification process (NO in S106), the integrated discriminator 70 performs the integrated identification process (S107). Details of the integrated identification process will be described later.

  When the scene can be identified by the integrated identification process (YES in S108), the scene identification unit 33 determines the scene by storing the identification result in the result storage unit 31B of the storage unit 31 (S109), and performs the scene identification process. finish. On the other hand, when the scene cannot be identified by the integrated identification process (NO in S108), the scene identification unit 33 stores all the candidate scenes (scene candidates) in the result storage unit 31B (S110). At this time, the certainty factor is stored in the result storage unit 31B together with the scene candidate.

  As a result of the scene identification process (whole identification process / partial identification process / integrated identification process), if any of S104, S106, and S108 in FIG. 8 is YES, the printer-side controller 20 selects one scene with a relatively high certainty factor. Can be identified. If NO in S108, the printer-side controller 20 can identify at least one scene (scene candidate) with a relatively low certainty factor. If NO in S108, there may be one scene candidate or two or more scene candidates.

=== Overall identification processing ===
FIG. 10 is a flowchart of the overall identification process. Here, the overall identification process will be described with reference to FIG.

  First, the overall classifier 50 selects one sub-classifier 51 from the plurality of sub-classifiers 51 (S201). The overall classifier 50 is provided with five sub-classifiers 51 for identifying whether an image to be identified (identification target image) belongs to a specific scene. The five sub classifiers 51 identify scenes of scenery, evening scene, night scene, flowers, and autumn leaves, respectively. Here, the overall classifier 50 selects the sub classifier 51 in the order of landscape → evening scene → night scene → flower → autumn leaves. For this reason, first, the sub classifier 51 (landscape classifier 51L) for identifying whether or not the classification target image belongs to a landscape scene is selected.

Next, the overall classifier 50 refers to the classification target table and determines whether or not a scene should be identified using the selected sub-classifier 51 (S202).
FIG. 11 is an explanatory diagram of the identification target table. This identification target table is stored in the result storage unit 31B of the storage unit 31. In the identification target table, all fields are set to zero in the first stage. In the process of S202, the “No” column is referred to, and if it is zero, it is determined as YES, and if it is 1, it is determined as NO. Here, the overall classifier 50 refers to the “No” column in the “Scenery” column in the identification target table, and determines “YES” because it is zero.

  Next, the sub classifier 51 calculates the value of the discriminant based on the entire feature amount (S203). The value of this discriminant is related to the probability (confidence) that the identification target image belongs to a specific scene (described later). For the sub classifier 51 of this embodiment, a classification method using a support vector machine (SVM) is used. The support vector machine will be described later. When the classification target image belongs to a specific scene, the discriminant of the sub classifier 51 tends to be a positive value. When the classification target image does not belong to a specific scene, the discriminant of the sub classifier 51 tends to be a negative value. Further, the discriminant becomes a larger value as the certainty that the identification target image belongs to the specific scene is higher. For this reason, if the discriminant value is large, the probability (confidence) that the identification target image belongs to a specific scene is high, and if the discriminant value is small, the probability that the classification target image belongs to a specific scene is low. .

  Next, the sub discriminator 51 determines whether or not the value of the discriminant is larger than the positive threshold (S204). If the value of the discriminant is larger than the positive threshold, the sub discriminator 51 determines that the classification target image belongs to a specific scene.

  FIG. 12 is an explanatory diagram of an affirmative threshold value of the overall identification process. In the figure, the horizontal axis indicates an affirmative threshold, and the vertical axis indicates the probability of recall or precision. FIG. 13 is an explanatory diagram of Recall and Precision. If the discriminant value is greater than or equal to the positive threshold, the identification result is Positive. If the discriminant value is not greater than or equal to the positive threshold, the identification result is Negative.

  Recall indicates the recall rate and detection rate. Recall is the ratio of the number of images identified as belonging to a specific scene to the total number of images of the specific scene. In other words, Recall is the probability that the sub-identifier 51 identifies the image as a positive when the image of the specific scene is identified by the sub-identifier 51 (the probability that the image of the specific scene belongs to the specific scene. ). For example, when a landscape image is identified by the landscape classifier 51L, it indicates the probability that the landscape classifier 51L identifies it as belonging to a landscape scene.

  Precision indicates the correct answer rate and the correct answer rate. Precision is the ratio of the number of images in a particular scene to the total number of images identified as Positive. In other words, Precision indicates the probability that the image to be identified is a specific scene when the sub-classifier 51 that identifies the specific scene identifies it as Positive. For example, when the landscape classifier 51L identifies that it belongs to a landscape scene, it indicates the probability that the identified image is really a landscape image.

As can be seen from FIG. 12, the larger the positive threshold, the greater the Precision. For this reason, the larger the positive threshold value, the higher the probability that an image identified as belonging to a landscape scene, for example, is a landscape image. That is, the greater the positive threshold, the lower the probability of misidentification.
On the other hand, the larger the positive threshold, the smaller the Recall. As a result, for example, even when a landscape image is identified by the landscape classifier 51L, it is difficult to correctly identify it as belonging to a landscape scene. By the way, if the image to be identified can be identified as belonging to a landscape scene (YES in S204), the speed of the overall identification process is increased so as not to identify other remaining scenes (such as sunsets). For this reason, the larger the positive threshold, the lower the overall identification processing speed. Further, if the scene can be identified by the overall identification process, the partial identification process is not performed and the speed of the scene identification process is increased (S104). Therefore, as the positive threshold is increased, the scene identification process speed decreases. Become.
That is, if the positive threshold is too small, the probability of misidentification increases, and if it is too large, the processing speed decreases. In the present embodiment, since the accuracy rate (Precision) is set to 97.5%, the landscape affirmation threshold is set to 1.72.

  If the discriminant value is greater than the affirmative threshold value (YES in S204), the sub-classifier 51 determines that the classification target image belongs to a specific scene and sets an affirmative flag (S205). “Set an affirmative flag” means that the “affirmation” column in FIG. In this case, the overall discriminator 50 ends the overall discrimination process without performing discrimination by the next sub discriminator 51. For example, if the image can be identified as a landscape image, the entire identification process is terminated without identifying the sunset scene or the like. In this case, since the identification by the next sub-identifier 51 is omitted, the speed of the overall identification process can be increased.

  If the value of the discriminant is not greater than the positive threshold (NO in S204), the sub discriminator 51 cannot determine that the classification target image belongs to a specific scene, and performs the next process of S206.

  Next, the sub discriminator 51 compares the discriminant value with a negative threshold value (S206). Thereby, the sub classifier 51 determines whether the classification target image does not belong to a predetermined scene. There are two types of such determinations. First, if the value of the discriminant of the sub-identifier 51 of a specific scene is smaller than the first negative threshold, it is determined that the classification target image does not belong to the specific scene. For example, if the discriminant value of the landscape classifier 51L is smaller than the first negative threshold, it is determined that the classification target image does not belong to a landscape scene. Second, if the value of the discriminant of the sub-identifier 51 of a specific scene is larger than the second negative threshold, it is determined that the classification target image does not belong to a scene different from the specific scene. . For example, if the discriminant value of the landscape classifier 51L is larger than the second negative threshold, it is determined that the classification target image does not belong to the night scene.

  FIG. 14 is an explanatory diagram of the first negative threshold. In the figure, the horizontal axis indicates the first negative threshold, and the vertical axis indicates the probability. The bold line in the graph is a True Negative Recall graph, and indicates the probability of correctly identifying an image other than a landscape image as not a landscape image. The thin line in the graph is a False Negative Recall graph, which indicates the probability of erroneously identifying a landscape image that is not a landscape image.

As can be seen from FIG. 14, the False Negative Recall is reduced as the first negative threshold is reduced. For this reason, the smaller the first negative threshold, the lower the probability that an image identified as not belonging to a landscape scene is a landscape image, for example. That is, the probability of misidentification is reduced.
On the other hand, the True Negative Recall decreases as the first negative threshold decreases. As a result, it is difficult to identify an image other than a landscape image unless it is a landscape image. On the other hand, if it is possible to identify that the identification target image is not a specific scene, the process by the sub partial classifier 61 for the specific scene is omitted during the partial identification process to speed up the scene identification processing speed (described later in FIG. 17). S302). For this reason, the scene identification processing speed decreases as the first negative threshold is decreased.
That is, if the first negative threshold is too large, the probability of misidentification increases, and if it is too small, the processing speed decreases. In this embodiment, in order to set False Negative Recall to 2.5%, the first negative threshold is set to −1.01.

  By the way, if the probability that an image belongs to a landscape scene is high, the probability that the image belongs to a night scene is inevitably low. For this reason, when the discriminant value of the landscape discriminator 51L is large, it may be identified that the scene is not a night scene. In order to perform such identification, a second negative threshold is provided.

  FIG. 15 is an explanatory diagram of the second negative threshold. In the figure, the horizontal axis indicates the value of the landscape discriminant, and the vertical axis indicates the probability. In this figure, the Recall graph of the night view is drawn with a dotted line together with the Recall and Precision graph of FIG. When attention is paid to this dotted line graph, if the value of the discriminant of landscape is larger than −0.44, the probability that the image is a night scene image is 2.5%. In other words, if the landscape discriminant value is greater than −0.44, even if the image is identified as not a night scene image, the probability of misidentification is only 2.5%. Therefore, in the present embodiment, the second negative threshold is set to −0.44.

  When the discriminant value is smaller than the first negative threshold value, or when the discriminant value is larger than the second negative threshold value (YES in S206), the sub-classifier 51 determines that the classification target image belongs to a predetermined scene. It is determined not to do so, and a negative flag is set (S207). “Set a negative flag” means to set the “No” column in FIG. For example, when it is determined that the image to be identified does not belong to a landscape scene based on the first negative threshold, the “denial” column in the “landscape” column is 1. Further, when it is determined that the identification target image does not belong to the night scene based on the second negative threshold, the “Negation” field in the “Night scene” field is “1”.

  FIG. 16A is an explanatory diagram of threshold values in the landscape classifier 51L described above. An affirmative threshold value and a negative threshold value are preset in the landscape discriminator 51L. 1.72 is set as the positive threshold. The negative threshold includes a first negative threshold and a second negative threshold. -1.01 is set as the first negative threshold. In addition, a value is set for each scene other than the landscape as the second negative threshold.

  FIG. 16B is an explanatory diagram outlining the processing of the landscape classifier 51L described above. Here, for simplification of description, only the night view will be described for the second negative threshold. If the discriminant value is greater than 1.72 (YES in S204), the landscape classifier 51L determines that the classification target image belongs to a landscape scene. If the discriminant value is 1.72 or less (NO in S204) and is greater than −0.44 (YES in S206), the landscape classifier 51L determines that the classification target image does not belong to the night scene. To do. If the value of the discriminant is smaller than −1.01 (YES in S206), the landscape classifier 51L determines that the classification target image does not belong to a landscape scene. Note that the landscape classifier 51L also determines whether the image to be identified does not belong to the scene based on the second negative threshold for the evening scene and the autumn leaves. However, since the second negative threshold for flowers is larger than the positive threshold, the landscape discriminator 51L does not determine that the classification target image does not belong to the flower scene.

  In the case of NO in S202, in the case of NO in S206, or when the processing in S207 is completed, the overall discriminator 50 determines the presence or absence of the next sub discriminator 51 (S208). Here, since the process by the landscape classifier 51L is finished, the overall classifier 50 determines in S208 that there is a next sub-classifier 51 (evening scene classifier 51S).

  Then, when the process of S205 is finished (when it is determined that the identification target image belongs to a specific scene), or when it is determined in S208 that there is no next sub-classifier 51 (the identification target image is a specific image). When it cannot be determined that the scene belongs to the scene), the overall discriminator 50 ends the overall discrimination process.

  As already described, when the overall identification process is completed, the scene identification unit 33 determines whether or not the scene has been identified by the overall identification process (S104 in FIG. 8). At this time, the scene identification unit 33 refers to the identification target table in FIG. 11 and determines whether or not there is 1 in the “affirmation” column.

  If the scene can be identified by the overall identification process (YES in S104), the partial identification process and the integrated identification process are omitted. This increases the speed of the scene identification process.

=== Partial identification processing ===
FIG. 17 is a flowchart of the partial identification process. The partial identification process is performed when the scene cannot be identified by the overall identification process (NO in S104 of FIG. 8). As will be described below, the partial identification process is a process of identifying the scene of the entire image by identifying each scene of the divided divided image. Here, the partial identification process will be described with reference to FIG.

  First, the partial classifier 60 selects one sub partial classifier 61 from the plurality of sub partial classifiers 61 (S301). The partial discriminator 60 is provided with three sub partial discriminators 61. Each sub partial discriminator 61 discriminates whether or not each partial image divided into 8 × 8 64 blocks belongs to a specific scene. Here, the three sub partial classifiers 61 identify the scenes of sunset, flowers, and autumn leaves, respectively. The partial discriminator 60 selects the sub partial discriminator 61 in the order of evening scene → flower → autumn leaves. Therefore, first, the sub partial classifier 61 (evening scene partial classifier 61S) for identifying whether or not the partial image belongs to the sunset scene is selected.

  Next, the partial discriminator 60 refers to the discrimination target table (FIG. 11) and determines whether or not the scene should be discriminated using the selected sub partial discriminator 61 (S302). Here, the partial discriminator 60 refers to the “No” column of the “Evening Scene” column in the classification target table, and determines YES if it is zero, and NO if it is 1. When the evening scene classifier 51S sets a negative flag with the first negative threshold during the overall identification process or when another sub-classifier 51 sets a negative flag with the second negative threshold, in S302 It is judged as NO. If it is determined NO, the sunset partial identification process is omitted, and the partial identification process speed increases. However, for the convenience of explanation, it is assumed that YES is determined here.

Next, the sub partial discriminator 61 selects one partial image from the partial images divided into 8 × 8 64 blocks (S303).
FIG. 18 is an explanatory diagram of the order of partial images selected by the evening scene partial classifier 61S. When a scene of the entire image is identified from the partial image, it is desirable that the partial image used for identification is a portion where the subject exists. Therefore, in this embodiment, thousands of samples of sunset scene images are prepared, each sunset scene image is divided into 64 blocks of 8 × 8, and blocks including sunset scene partial images (sun and sky partial images of the sunset scene) are included. The presence probability of the sunset partial image in each block was calculated based on the extracted block position. And in this embodiment, a partial image is selected in an order from a block with a high existence probability. Note that the selection order information shown in the figure is stored in the memory 23 as part of the program.

  In the case of an evening scene image, since the sky of the evening scene often spreads from the vicinity of the center to the upper half, the existence probability increases in the upper half block from the vicinity of the center. In the case of an evening scene image, the lower 1/3 of the image is shaded by backlight, and the partial image alone often cannot be distinguished from the evening scene or the night scene, so the existence probability is lower in the lower 1/3 block. In the case of a flower image, since the composition is often such that a flower is arranged near the center, the probability of existence of a flower partial image near the center increases.

  Next, the sub partial classifier 61 determines whether or not the partial image belongs to a specific scene based on the partial feature amount of the selected partial image (S304). Similar to the sub classifier 51 of the overall classifier 50, the sub partial classifier 61 uses a discrimination method using a support vector machine (SVM). The support vector machine will be described later. If the discriminant value is a positive value, it is determined that the partial image belongs to a specific scene, and the sub partial classifier 61 increments the positive count value. If the discriminant value is a negative value, it is determined that the partial image does not belong to a specific scene, and the sub partial discriminator 61 increments the negative count value.

  Next, the sub partial discriminator 61 determines whether or not the positive count value is larger than the positive threshold value (S305). The positive count value indicates the number of partial images determined to belong to a specific scene. If the positive count value is larger than the affirmative threshold (YES in S305), the sub partial classifier 61 determines that the classification target image belongs to a specific scene, and sets an affirmative flag (S306). In this case, the partial discriminator 60 ends the partial discriminating process without performing discrimination by the next sub partial discriminator 61. For example, if the image can be identified as an evening scene image, the partial identification process is terminated without identifying flowers and autumn leaves. In this case, since the identification by the next sub partial classifier 61 is omitted, the speed of the partial classification process can be increased.

  If the positive count value is not greater than the positive threshold value (NO in S305), the sub partial classifier 61 cannot determine that the classification target image belongs to a specific scene, and performs the next process of S307.

  If the sum of the positive count value and the number of remaining partial images is smaller than the positive threshold (YES in S307), the sub partial discriminator 61 proceeds to the process of S309. If the sum of the positive count value and the number of remaining partial images is smaller than the positive threshold value, the positive count value does not become larger than the positive threshold value even if the positive count value is incremented by all the remaining partial images. By proceeding with the process, the remaining partial images are not identified by the support vector machine. Thereby, the speed of the partial identification process can be increased.

If the sub partial discriminator 61 determines NO in S307, the sub partial discriminator 61 determines whether there is a next partial image (S308). In the present embodiment, not all of the partial images divided into 64 are selected in order. In FIG. 18, only the top 10 partial images indicated by thick frames are selected in order. Therefore, when the identification of the tenth partial image is completed, the sub partial classifier 61 determines in S308 that there is no next partial image. (In consideration of this point, the “number of remaining partial images” in S307 is also determined.)
FIG. 19 is a Recall and Precision graph when an evening scene image is identified using only the top 10 partial images. If an affirmative threshold as shown in the figure is set, the accuracy rate (Precision) can be set to about 80%, the recall rate (Recall) can be set to about 90%, and identification with high accuracy is possible.

In this embodiment, the evening scene image is identified using only 10 partial images. For this reason, in the present embodiment, it is possible to increase the speed of the partial identification process compared to the case where the evening scene image is identified using all 64 partial images.
In this embodiment, the sunset scene image is identified using the top tenth partial image having a high existence probability of the sunset scene partial image. For this reason, in the present embodiment, it is possible to set both Recall and Precision higher than the identification of an evening scene image using 10 partial images extracted by ignoring the existence probability.
In this embodiment, the partial images are selected in descending order of the existence probability of the sunset partial image. As a result, the determination in S305 is likely to be YES at an early stage. For this reason, in the present embodiment, the speed of the partial identification process can be increased as compared with the case where the partial images are selected in the order in which the presence probability level is ignored.

  When it is determined YES in S307, or when it is determined that there is no next partial image in S308, the sub partial discriminator 61 determines whether or not the negative count value is larger than the negative threshold (S309). . Since this negative threshold performs substantially the same function as the negative threshold (S206 in FIG. 10) in the above-described overall identification process, detailed description thereof is omitted. If YES is determined in S309, a negative flag is set as in S207 of FIG.

  In the case of NO in S302, in the case of NO in S309, or when the process of S310 is completed, the partial discriminator 60 determines whether or not there is a next sub partial discriminator 61 (S311). In the case after the processing by the evening scene partial classifier 61S is finished, since the flower partial classifier 61F and the autumnal leaves partial classifier 61R are still present as the sub partial classifier 61, the partial classifier 60 determines the next sub partial classifier in S311. It is determined that there is a container 61.

  Then, when the process of S306 is completed (when it is determined that the identification target image belongs to a specific scene), or when it is determined in S311 that there is no next sub partial classifier 61 (the identification target image is specified). If it cannot be determined that the scene belongs to the scene), the partial discriminator 60 ends the partial discrimination processing.

As already described, when the partial identification process ends, the scene identification unit 33 determines whether or not the scene can be identified by the partial identification process (S106 in FIG. 8). At this time, the scene identification unit 33 refers to the identification target table in FIG. 11 and determines whether or not there is 1 in the “affirmation” column.
When the scene can be identified by the partial identification process (YES in S106), the integrated identification process is omitted. This increases the speed of the scene identification process.

=== Support vector machine ===
Before describing the integrated identification process, the support vector machine (SVM) used in the sub-identifier 51 for the overall identification process and the sub-partial identifier 61 for the partial identification process will be described.

FIG. 20A is an explanatory diagram of determination by the linear support vector machine. Here, the learning sample is shown in a two-dimensional space by two feature amounts x1 and x2. The learning sample is divided into two classes A and B. In the figure, samples belonging to class A are indicated by circles, and samples belonging to class B are indicated by squares.
A boundary that divides the two-dimensional space into two is defined by learning using the learning sample. The boundary is defined by <w · x> + b = 0 (where x = (x1, x2), w is a weight vector, and <w · x> is an inner product of w and x). However, the boundary is defined by learning using a learning sample so that the margin is maximized. That is, in the case of the figure, the boundary is not a thick dotted line but a thick solid line.
The discrimination is performed using the discriminant f (x) = <w · x> + b. It is determined that a certain input x (this input x is different from the learning sample) belongs to class A if f (x)> 0, and belongs to class B if f (x) <0. Determined.

  Here, the description is made using a two-dimensional space, but the present invention is not limited to this (that is, the feature amount may be two or more). In this case, the boundary is defined by a hyperplane.

  By the way, there are cases where the two classes cannot be separated by a linear function. In such a case, if the determination is performed by the linear support vector machine, the accuracy of the determination result is lowered. Therefore, if the feature quantity of the input space is nonlinearly transformed, that is, if the input space is nonlinearly mapped to a certain feature space, it can be separated by a linear function in the feature space. This is used in the nonlinear support vector machine.

  FIG. 20B is an explanatory diagram of discrimination using a kernel function. Here, the learning sample is shown in a two-dimensional space by two feature amounts x1 and x2. If the nonlinear mapping from the input space of FIG. 20B becomes a feature space as shown in FIG. 20A, it can be separated into two classes by a linear function. If the boundary is defined so that the margin is maximized in this feature space, the inverse mapping of the boundary in the feature space becomes the boundary shown in FIG. 20B. As a result, the boundary becomes nonlinear as shown in FIG. 20B.

In this embodiment, by using a Gaussian kernel, the discriminant f (x) becomes as follows (where M is the number of features and N is the number of learning samples (or contributes to the boundary): The number of learning samples), w _i is a weighting factor, y _j is the feature quantity of the learning sample, and x _j is the feature quantity of the input x).

  It is determined that a certain input x (this input x is different from the learning sample) belongs to class A if f (x)> 0, and belongs to class B if f (x) <0. Determined. Further, the larger the value of the discriminant f (x), the higher the probability that the input x (this input x is different from the learning sample) belongs to the class A. On the contrary, the smaller the value of the discriminant f (x), the lower the probability that the input x (this input x is different from the learning sample) belongs to the class A. In the sub-identifier 51 for the overall identification process and the sub-partial identifier 61 for the partial identification process, the value of the discriminant f (x) of the support vector machine is used.

  An evaluation sample is prepared separately from the learning sample. The above Recall and Precision graphs are based on the identification results for the evaluation samples.

=== Integrated identification processing ===
In the above-described overall identification process and partial identification process, the positive threshold value in the sub-classifier 51 and the sub-classifier 61 is set relatively high, and the Precision (correct answer rate) is set high. This is because, for example, if the accuracy rate of the landscape classifier 51L of the overall classifying unit is set low, the landscape classifier 51L misidentifies the autumnal image as a landscape image, and before the autumnal classifier 51R performs classification. This is because a situation occurs in which the entire identification process ends. In the present embodiment, by setting the Precision (accuracy rate) high, an image belonging to a specific scene is identified by the sub-classifier 51 (or sub-partial classifier 61) of the specific scene ( For example, the autumnal leaves image is identified by the autumnal leaves discriminator 51R (or the autumnal leaf partial discriminator 61R).

  However, if the Precision (accuracy rate) of the overall identification process or the partial identification process is set to be high, there is a high possibility that the scene cannot be identified by the overall identification process or the partial identification process. Therefore, in this embodiment, when the scene cannot be identified by the overall identification process and the partial identification process, the integrated identification process described below is performed.

  FIG. 21 is a flowchart of the integrated identification process. As will be described below, the integrated identification process is based on the discriminant value of each sub-classifier 51 in the overall identification process, and the certainty level is equal to or higher than a predetermined value (for example, 90% or higher). This is a process for selecting a high scene.

  First, the integrated discriminator 70 extracts a positive scene based on the discriminant values of the five sub discriminators 51 (S401). At this time, the value of the discriminant calculated by each sub classifier 51 during the overall identification process is used.

  Next, the integrated discriminator 70 determines whether or not there is a scene with a certainty factor equal to or greater than a predetermined value (S402). Here, the certainty factor indicates the probability that the identification target image belongs to the specific scene, and is obtained from the value of the discriminant. More specifically, the integrated discriminator 70 is provided with a table showing the relationship between the discriminant value and the precision, and a precision corresponding to the discriminant value is derived based on this table. The value is used as confidence. The predetermined value is set to 90%, for example, and is set to a value lower than Precision (97.5%) set by an affirmative threshold of the overall classifier or partial classifier. However, the certainty factor may not be Precision, and the value of the discriminant may be used as the certainty factor.

  If there is a scene with a certainty level equal to or greater than a predetermined value (YES in S402), an affirmative flag is set in the field for that scene (S403), and the integrated identification process is terminated. Note that when a scene having a certainty factor of 90% or more is extracted, a plurality of scenes are not extracted. This is because if the certainty of a certain scene is high, the certainty of another scene is inevitably low.

  On the other hand, if there is no scene with the certainty level equal to or higher than the predetermined value (NO in S402), the integrated identification process is terminated without setting an affirmative flag. As a result, one scene does not exist in the affirmative column of the identification target table in FIG. That is, it cannot be identified to which scene the identification target image belongs.

  As already described, when the integrated identification process is completed, the scene identification unit 33 determines whether or not the scene has been identified by the integrated identification process (S108 in FIG. 8). At this time, the scene identification unit 33 refers to the identification target table in FIG. 11 and determines whether or not there is 1 in the “affirmation” column. If YES is determined in S402, the determination in S108 is also YES. On the other hand, if NO is determined in S402, the determination in S108 is also NO.

  In the present embodiment, in the case of NO in S108 in FIG. 8, that is, in the case of NO in S402 in FIG. 21, all the scenes extracted in S401 are stored as scene candidates in the result storage unit 31B.

=== Display on display ===
<Overview>
As described above, the user can set the shooting mode with the mode setting dial 2A. The digital still camera 2 determines shooting conditions (exposure time, ISO sensitivity, etc.) based on the set shooting mode, photometry results at the time of shooting, and the like, and shoots the subject under the determined shooting conditions. After shooting, the digital still camera 2 stores shooting data indicating shooting conditions at the time of shooting together with the image data in the memory card 6 as an image file.

By the way, since the user forgot to set the shooting mode, shooting may be performed while the shooting mode inappropriate for the shooting condition is set. For example, a daytime landscape may be shot while the night view mode is set. In this case, although the image data of the image file is an image of a daytime landscape, data indicating the night view mode is stored in the shooting data (for example, the shooting scene type data in FIG. 5 is “3”). "become). In such a case, if the image data is corrected based on inappropriate shooting scene type data, printing with an image quality undesirable for the user is performed.
On the other hand, even if the image data is corrected based on the result of the identification processing (face identification processing and scene identification processing), printing with the image quality desired by the user may not be obtained. For example, when an erroneous identification occurs in the identification process, there is a case where a print having an image quality desired by the user cannot be obtained. Further, even when the shooting mode set by the user for a special effect is denied and the image data is corrected based on the identification result by the printer, printing as intended by the user cannot be performed.
Therefore, in this embodiment, a confirmation screen for prompting the user to confirm is displayed. Specifically, as described later, when the identification processing result and the scene indicated by the scene information (shooting scene type data and shooting mode data) of the additional data of the image file do not match, the display unit 16 of the panel unit 15 A confirmation screen is displayed.

<Reference example>
In the reference example, a plurality of image files are directly printed. As will be described later, in this reference example, images of all the image files are displayed on the confirmation screen, and printing is started after confirmation on the confirmation screen is completed.

  FIG. 22 is a flowchart showing the flow of the direct printing process of the reference example. Each process is realized by the printer-side controller 20 based on a program stored in the memory 23.

  First, the printer-side controller 20 performs a face identification process and a scene identification process for all of a plurality of image files to be directly printed (S601). Since these processes have already been described, a description thereof will be omitted.

  Next, the printer-side controller 20 determines whether or not the scene indicated by the additional data (shooting scene type data and shooting mode data) and the scene indicated by the identification processing result are inconsistent for each image file to be directly printed. Each is judged (S602). When a plurality of scene candidates are included in the identification processing result, determination is made using the scene candidate with the highest certainty factor.

  Next, the printer-side controller 20 determines whether or not there is even one image file in which the two scenes do not match (S603). If the two scenes match in all the image files to be directly printed (NO in S603), the user does not need confirmation and the process proceeds to S606. By not displaying the confirmation screen in this way, the processing time until printing can be shortened.

  If there is even one image file in which the two scenes do not match (YES in S603), the printer-side controller 20 displays a confirmation screen on the display unit 16 (S604).

23A and 23B are explanatory diagrams of an example of a confirmation screen of a reference example.
Nine images 162A are displayed on the confirmation screen 162 (only a square frame is drawn in the figure, but in reality, images are displayed in this frame). The nine images 162A are images of nine image files among a plurality of image files to be directly printed. Since the space in which each image can be displayed is small, each image 162A is displayed using thumbnail image data (see FIG. 3) of the image file. In addition, since the displayed image is small, it is difficult for the user to evaluate the image quality. Therefore, the thumbnail image data is not particularly subjected to image correction. On the upper left of each image 162A, numbers are assigned in order of shooting date and time or in order of data name of the image file. In the following description, an image is specified using this number. For example, of the nine images on the confirmation screen, the upper left image is referred to as “first image”, and the image file corresponding to this image is referred to as “first image file”.

  The printer-side controller 20 displays the image 162A without adding the mark 162B to those in which the two scenes match in S602 described above. In addition, the printer-side controller 20 displays the image 162B with the mark 162B added to those in which the two scenes do not match in S602 described above. Thereby, the user can easily know in which image the two scenes do not match depending on the presence or absence of the mark 162B.

  In the confirmation screen 162 in the figure, a mark 162B is displayed at the lower right of the first, fourth, fifth and ninth images. For this reason, in the first, fourth, fifth and ninth image files, it can be seen that the scene of the additional data and the scene to be identified do not match. On the other hand, the mark 162B is not displayed in the second, third, and sixth to eighth images. For this reason, in the second, third, and sixth to eighth image files, it can be seen that the scene of the additional data matches the scene to be identified.

  In the mark 162B, a scene as a result of identification processing is shown. When a plurality of scene candidates are included in the identification processing result, the scene candidate having the highest certainty factor is indicated in the mark 162B. For example, the scene (or the scene candidate with the highest certainty) as the identification processing result of the first image file is “landscape”.

  In the reference example, the mark 162B is not added to an image in which the scene of the additional data matches the scene to be identified. As a result, an image in which the two scenes do not match is conspicuous. On the other hand, if the mark 162B is not added, the image identification processing result is not displayed on the confirmation screen 162. However, if the two scenes match, the confirmation of the user should not be necessary in the first place, so there is no problem even if the identification processing result is not displayed.

  The user can determine whether to perform correction based on the scene of the additional data or correction based on the scene of the identification processing result by operating the input unit 17 while viewing the confirmation screen. For example, when the user operates the input unit 17 to select the mark 162B of the fifth image 162A, the mark 162B is highlighted with a bold line as shown in FIG. 23A. In this state, the user can further operate the input unit 17 to switch the scene shown in the mark 162B. At this time, in response to a user operation, the printer-side controller 20 displays the additional data scene and, if there are a plurality of scene candidates, other scene candidates in the mark 162B. FIG. 23B shows a state where “scenery”, which is a scene of additional data, is displayed in the mark 162B in response to a user operation. In this way, by switching and displaying the scene of the identification processing result and the scene of the additional data, the user's selection range is limited, so that the user's operation becomes easy. If the user selects a desired scene from all scenes (for example, all scenes in FIG. 7), the user's selection range becomes too wide and the operability is not very good.

  After the first to ninth image files are confirmed on the confirmation screen, the printer controller 20 further displays a confirmation screen for the tenth to eighteenth image files. In this way, the user confirmation is similarly performed for the remaining image files.

  When the determination button (not shown) of the input unit 17 is pressed by the user (YES in S605), the printer controller 20 corrects the image data in the correction mode according to the user's selection (S606). If the determination button is pressed in the state of FIG. 23B, the image data other than the fifth is corrected based on the scene of the identification processing result (the scene candidate having the highest certainty when there are a plurality of scene candidates). The second image data is corrected by the landscape mode (see FIG. 7) based on the scene of the additional data.

  The confirmation screen is displayed until the user inputs confirmation (NO in S605). If there is a confirmation input by the user (YES in S605), the printer-side controller 20 corrects the image data in the correction mode according to the user's selection (S606). However, even if there is no confirmation input by the user, when a predetermined time (for example, 20 seconds) has elapsed after the confirmation screen is displayed, the scene selected in the initial setting (here, the scene of the identification processing result) The image data may be corrected in a corresponding correction mode. Thereby, even if the user is away from the printer, the process can proceed. The scene selected in the initial setting may be changed according to the certainty of the identification processing result, or may be determined in advance so that the scene of the identification processing result is always selected regardless of the certainty.

  After the image correction process, the printer-side controller 20 prints an image based on the corrected image data (S607). As a result, a print image having an image quality according to the user's desire is obtained.

<First Embodiment>
In the first embodiment, only the image of the image file in which the scene of the additional data and the scene of the identification processing result do not match are displayed on the confirmation screen, and the image file that does not match is printed first. That is, compared to the case of the reference example, the image file to be displayed on the confirmation screen is different, and the timing of starting the printing process is different.

  In the following description, it is assumed that the objects of direct printing are the first to ninth image files. Similarly to the reference example, in the first, fourth, fifth, and ninth image files, the additional data scene and the scene to be identified do not match, and in the second, third, and sixth to eighth image files, the additional data It is assumed that the scene matches the scene to be identified.

  FIG. 24 is a flowchart showing the flow of direct print processing according to the first embodiment. Each process is realized by the printer-side controller 20 based on a program stored in the memory 23.

  First, the printer-side controller 20 acquires a first image file from a plurality of image files to be subjected to direct printing, and performs face identification processing and scene identification processing (S701). Since these processes have already been described, a description thereof will be omitted.

  Next, the printer-side controller 20 determines whether the scene indicated by the additional data can be compared with the scene indicated by the identification processing result (S702). When a plurality of scene candidates are included in the identification processing result, determination is made using the scene candidate with the highest certainty factor.

Note that the determination method in S702 is different between the case where the shooting scene type data is used to determine the mismatch in the next S703 and the case where shooting mode data which is MakerNote data is used.
When shooting scene type data is used in S703, when the shooting scene type data is neither “person”, “landscape”, or “night view”, for example, when shooting scene type data is “0” (see FIG. 5), in S703. Since the comparison with the identification processing result cannot be performed, the determination in S702 is NO. If the identification processing result is neither “person”, “landscape”, or “night view”, the comparison with the photographic scene type data cannot be made in S703, so the determination in S702 is NO. For example, when the identification processing result is “evening scene”, the determination in S702 is NO.
When shooting mode data is used in S703, when the shooting mode data is not “person”, “landscape”, “evening scene”, or “night scene”, for example, when the shooting mode data is “3 (close-up)” (see FIG. 5). Since the comparison with the identification processing result is impossible, the determination in S702 is NO. Further, when the identification processing result is neither “person”, “landscape”, “evening scene”, or “night scene”, the comparison with the shooting mode data is impossible, so the determination in S702 is NO.

  In the case of YES in S702, the printer-side controller 20 determines whether or not the scene indicated by the additional data (shooting scene type data and shooting mode data) and the scene indicated by the identification processing result do not match (S703). When a plurality of scene candidates are included in the identification processing result, determination is made using the scene candidate with the highest certainty factor.

  If they do not match in S703 (YES), the number of the image file, the identification processing result, etc. are stored in the memory 23 (S705). Then, the process proceeds to S706.

  If there is no mismatch in S703 (NO), the printer-side controller 20 creates a job (S704). The content of the job at this time is to correct the image data based on the scene of the identification processing result, and to perform the printing process based on the corrected image data. If a plurality of jobs are accumulated, the printer-side controller 20 executes the jobs in order according to the priority order. When the job is executed, the image data is corrected based on a predetermined scene (here, a scene as a result of the identification process) according to the contents of the job, and a printing process is performed based on the corrected image data. Note that the printer-side controller 20 performs the processes in FIG. 24 in parallel while executing a job.

  If YES is determined in step S702 for the first image file and YES is determined in step S703, the printer-side controller 20 determines the image file number, the identification processing result (here, “landscape”, a plurality of scene candidates). If there is, those scene candidates) are stored in the memory 23 (S705).

  Next, since the second to ninth image files still remain, NO is determined in S706, and the processing of S701 is performed on the second image file.

  In the second image file, if YES is determined in S702 and NO is determined in S703, the printer-side controller 20 creates a job for the second image file (S704). At this time, since there is no other job, the job is executed immediately after the job is created. That is, the image data of the second image file is corrected, and the printing process is started based on the corrected image data.

  In this way, the processes of S701 to S706 are similarly performed on the remaining third to ninth image files. Note that the printer-side controller 20 performs the processes in S701 to S706 on the third image file in parallel while executing the job of the second image file.

  After the process of S705 is performed on the ninth image file, there is no remaining image file, so the printer-side controller 20 determines YES in S706. Then, the printer-side controller 20 displays a confirmation screen (S707).

FIG. 25 is an explanatory diagram illustrating an example of the confirmation screen 164 according to the first embodiment.
On the confirmation screen 164, four images 164A are displayed (in the figure, only a square frame is drawn, but in reality, an image is displayed in this frame). The printer-side controller 20 determines which image should be displayed on the confirmation screen 164 based on the data stored in S705. In the four images 164A displayed on the confirmation screen 164, it is determined in S703 that the scene indicated by the additional data (shooting scene type data and shooting mode data) does not match the scene indicated by the identification processing result. The fourth, fifth, and ninth images.

  Compared with the above-described reference example, in the first embodiment, only mismatched images are displayed, so that the user can easily grasp the image to be confirmed. Further, in the first embodiment, since only the images where the two scenes do not match are displayed, the space in which each image 164A can be displayed is expanded. For this reason, the image 164A may be displayed using thumbnail image data, but the image data may be displayed after correction. When the image data 164A is displayed by correcting the image data, the correction corresponding to the scene indicated in the lower right mark 164B of the image 164A is performed. Thereby, the content of the correction applied to the image 164A matches the content indicated by the mark 164B at the lower right of the image 164A.

  As in the case of the reference example, the user operates the input unit 17 while viewing the confirmation screen 164 to perform correction based on the scene of the additional data for each image, or to the scene of the identification processing result. It is possible to decide whether to perform correction based on this. When the image data 164A is displayed by correcting the image data, the printer-side controller 20 switches and displays the correction applied to the image 164A every time the scene in the mark 164B is switched. For example, when the scene indicated by the mark 164B of the fifth image 164A is switched from “evening scene” (identification processing result scene) to “landscape” (additional data scene), the fifth image 164A is switched to the evening scene mode. Switch from the corrected image to the image corrected in landscape mode. This makes it easier for the user to confirm the effect of correction.

  The confirmation screen is displayed until confirmation is input by the user (NO in S708). However, even if there is no confirmation input by the user, the process may proceed to the next process after a predetermined time (for example, 20 seconds) has elapsed since the confirmation screen was displayed. Thereby, even if the user is away from the printer, the process can proceed. In this case, it is assumed that the scene selected by the initial setting is selected by the user. The scene selected in the initial setting may be changed according to the certainty of the identification processing result, or may be determined in advance so that the scene of the identification processing result is always selected regardless of the certainty.

  In the first embodiment, printing is already started while the user performs an operation on the confirmation screen. In the above-described reference example, if the user confirmation takes time, the printing start is delayed. However, in the first embodiment, since the printing is started while the confirmation screen is displayed (at least the second image is displayed). Since printing has started, the time until printing ends is shortened.

  When a determination button (not shown) of the input unit 17 is pressed by the user (YES in S708), the printer-side controller 20 creates jobs for printing the first, fourth, fifth, and ninth images, respectively (S709). ). The content of the job at this time is to correct the image data based on the scene selected by the user, and to perform a printing process based on the corrected image data.

  Next, the printer-side controller 20 determines whether or not printing is possible in numerical order (S710). Specifically, if the minimum number of the image file that created the job in S709 is larger than the number of the image that has already started printing, the printer-side controller 20 determines that printing is possible in the order of the numbers. Here, since the minimum number of the image file (first, fourth, fifth, ninth image file) that created the job in S709 is No. 1, and the second image has started printing, the determination in S710 is NO.

  If the two scenes do not match in the first image file, a job for printing the first to third images is created in S704, and printing of the first image is started. Usually, since it takes several seconds to several tens of seconds to print one image, if the operation on the confirmation screen by the user is quick, before the first to third images are printed (the sixth image Before printing starts, jobs for the fourth, fifth, and ninth image files are created (S709). In such a case, the printer-side controller 20 determines YES in S710, changes the job order (S711), and sets the job priority in the order of the image file numbers. Thus, after printing the third image, the printer 4 prints the fourth image instead of printing the sixth image. Then, the user can obtain images printed in the order of the numbers of the image files.

  If the determination in S710 is NO, the printer 4 does not print the images in the order of the image file numbers, so the printer-side controller 20 may display a warning screen informing the user of this on the display unit 16. Furthermore, displaying the print order on this warning screen is convenient when the user rearranges the printed images.

  Then, the printer-side controller 20 executes the stored jobs in order according to the priority order, and when the execution of all the jobs is completed (YES in S712), the process ends.

<Another confirmation screen of the first embodiment>
FIG. 26 is an explanatory diagram of another confirmation screen according to the first embodiment. Compared with the confirmation screen 164 of FIG. 25, the confirmation screen 165 is different in that three marks 1651 to 1653 are provided for each image.

  On this confirmation screen 165, three images 165A are displayed. As in the case of the confirmation screen 164 described above, the printer-side controller 20 determines which image should be displayed on the confirmation screen 165 based on the data stored in S705. In the three images 164A displayed on the confirmation screen 165, it is determined in S703 that the scene indicated by the additional data (shooting scene type data and shooting mode data) and the scene indicated by the identification processing result are inconsistent. The fourth and fifth images. The information on the ninth image is displayed by switching the confirmation screen by the user's operation.

  Also on this confirmation screen 165, since only images where the two scenes do not match are displayed, the user can easily grasp the image to be confirmed. However, since the space in which the image 165A can be displayed is small compared to the above-described confirmation screen 164, each image 165A is displayed using thumbnail image data (see FIG. 3) of the image file. In addition, since the displayed image is small, it is difficult for the user to evaluate the image quality. Therefore, the thumbnail image data is not particularly subjected to image correction.

  On the right side of each image 165, three marks 1651 to 1653 associated with each image are displayed. The three marks are, in order from the left, a mark indicating correction in the standard mode (correction by “Other” in FIG. 7), a mark indicating a scene of additional data, and a mark indicating the identification processing result. In the confirmation screen 164 described above, the user switches the scene in the mark. However, on the confirmation screen 165, the user selects one of the marks, and the printer-side controller 20 changes the scene according to the user's selection. Based on this, the image is corrected.

  The confirmation screen 165 has a mark indicating that correction is performed in the standard mode (correction by “Other” in FIG. 7), but this mark may not be present. In addition, if there is a sufficient space for displaying marks, marks indicating each scene candidate may be displayed when there are a plurality of scene candidates.

  Also in the confirmation screen 165, compared with the above-described reference example, only the mismatched images are displayed, so that the user can easily grasp the image to be confirmed. Further, in the present embodiment, since only images where the two scenes do not match are displayed, the space for displaying the mark 165B is expanded. For this reason, the amount of information that can be presented to the user on the confirmation screen can be increased.

Second Embodiment
In the first embodiment described above, a plurality of images are displayed in a list on the confirmation screen. On the other hand, in the following second embodiment, confirmation screens are displayed one by one.

  In the following description, as in the first embodiment, the direct print target is the first to ninth image files. In the first, fourth, fifth, and ninth image files, the additional data scene and the identification target scene do not match, and in the second, third, and sixth to eighth image files, the additional data scene and the identification target scene. And match.

  FIG. 27 is a flowchart showing the flow of direct print processing according to the second embodiment. Each process is realized by the printer-side controller 20 based on a program stored in the memory 23. In addition, since the process of S701-S706 of FIG. 27 in 2nd Embodiment is the same as the process of S701-S706 of FIG. 24 in above-mentioned 1st Embodiment, description is abbreviate | omitted.

  After the process of S705 is performed on the ninth image file, there is no remaining image file, so the printer-side controller 20 determines YES in S706. At this time, the memory 23 stores data relating to the first, fourth, fifth, and ninth images as the data stored in S703.

  Next, the printer-side controller 20 selects the first image from the first, fourth, fifth, and ninth images in which the additional data scene and the identification processing result scene do not match (S721). Then, the printer-side controller 20 displays a confirmation screen 166 for the first image (S722).

  FIG. 28A is an explanatory diagram illustrating an example of a confirmation screen according to the second embodiment. On the confirmation screen 166, two images indicated by the first image data are displayed side by side. The left image 166A is an image corrected based on the scene indicated by the additional data, and is corrected in the “landscape” mode here (see FIG. 7). On the other hand, the image 166B on the right side is an image corrected based on the scene of the identification processing result, and is corrected in the “person” mode here (see FIG. 7). Also, the certainty of the identification processing result is shown below the right image. Since the image displayed on the display unit 16 is not large, the user's judgment material is supplemented by displaying the certainty factor together with the image. The printer-side controller 20 highlights the edge of the image selected by the user with a bold line in response to the user's operation on the input unit 17. In the initial setting, the right image 166B is selected. If the OK button 166C is selected by the user (YES in S723), the printer-side controller 20 creates a job for correcting and printing the image data in the correction mode according to the user's selection (S724). If the OK button 166C is pressed in the state of FIG. 28A, a job is created in which the image data is corrected and printed in the “person” mode, which is the identification processing result.

  FIG. 28B is an explanatory diagram of another example of the confirmation screen according to the second embodiment. On the confirmation screen 167, one image indicated by the first image data is displayed. However, the left and right image quality are different at the boundary 167C. On the left side of the boundary, an image 167A corrected based on the scene indicated by the additional data is displayed. On the right side of the boundary, an image 167B corrected based on the scene of the identification processing result is displayed. By displaying in this way, the user can compare the left and right image quality at the boundary, so it is easier to compare the two images (167A, 167B) than comparing the two separated images as shown in FIG. 28A. . Normally, the printer-side controller 20 sets the boundary 167C at the center of the screen, but when the identification processing result is “person”, the printer-side controller 20 sets the boundary at the center of the identified person as shown in the figure. To do. When the image is corrected in the “person” mode, the skin color is corrected (see FIG. 7). Therefore, by setting a boundary at the center of the person, the user can use the “person” mode. It can be easily determined whether or not image correction is appropriate. On this confirmation screen, a button 167D for displaying the scene indicated by the additional data and a button 167E for displaying the scene of the identification processing result are displayed. When the user selects one of the two buttons (167D, 167E) (YES in S723), the printer-side controller 20 corrects the image data in a correction mode according to the user's selection and prints a job. Create (S724).

  The confirmation screen is displayed until confirmation is input by the user (NO in S723). However, even if there is no confirmation input by the user, the process may proceed to the next process after a predetermined time (for example, 20 seconds) has elapsed since the confirmation screen was displayed. Thereby, even if the user is away from the printer, the process can proceed. In this case, it is assumed that the scene selected by the initial setting is selected by the user. The scene selected in the initial setting may be changed according to the certainty of the identification processing result, or may be determined in advance so that the scene of the identification processing result is always selected regardless of the certainty.

  Also in the second embodiment, printing is already started while the user performs an operation on the confirmation screen. In the above-described reference example, if the user confirmation takes time, the printing start is delayed. However, in the second embodiment, since the printing is started while the confirmation screen is displayed (at least for the second image). Since printing has started, the time until printing ends is shortened.

  If there is a confirmation by the user (YES in S723), the printer-side controller 20 creates a job for correcting and printing the image data in the correction mode according to the user's selection (S724). In S709 of the first embodiment described above, four jobs are created. In S724 of the second embodiment, one job is created. Here, first, a job for the first image is created.

  Next, the printer-side controller 20 determines whether printing is possible in numerical order (S725). Specifically, if the number of the image file that created the job in S724 is larger than the number of the image that has already started printing, the printer-side controller 20 determines that printing is possible in the order of the numbers. Here, since the job of the first image file has been created in S724, and the second image has started printing, the determination in S710 is NO.

  Next, in S727, since there is still an inconsistent image (since the fourth, fifth, and ninth images are not yet), the printer-side controller 20 determines YES.

  Next, the printer-side controller 20 selects the fourth image from the fourth, fifth, and ninth images (S721). Then, the printer-side controller 20 similarly displays a confirmation screen for the fourth image (S722).

  The user confirms the fourth image (YES in S723), and after the job of the fourth image file is created (S724), the printer-side controller 20 can print the fourth image in numerical order. Whether or not (S725). If printing of the fourth image file has been created before printing of the third image is completed (before printing of the sixth image is started), the printer-side controller 20 determines YES in S725. Judge. Then, the job of the fourth image file is inserted between the job of the third image file and the job of the sixth image file. Thus, after printing the third image, the printer 4 prints the fourth image instead of printing the sixth image. That is, the fourth image can be printed in the order of the image file numbers.

  If the determination in S725 is NO, the printer 4 does not print the images in the order of the image file numbers. Therefore, the printer-side controller 20 stores the image numbers determined as NO in S725, and displays the warning screen. It may be displayed on the display unit 16 that the image files are not printed in the order of the numbers of the image files and the numbers of the image files that are not printed in the order of the numbers. Such a warning display is convenient when the user rearranges the printed images.

  In this way, jobs for the remaining fifth and ninth image files are created (S724), and the job order is changed as necessary (S726). Then, the printer-side controller 20 executes the stored jobs in order according to the priority order, and when the execution of all the jobs is completed (YES in S712), the process ends.

=== Add scene information to additional data ===
If the user selects a scene on the confirmation screen, the scene desired by the user can be determined. Therefore, in this embodiment, when the user confirms on the confirmation screen, the printer-side controller 20 stores the scene selected by the user in the additional data of the image file. Here, a case where the user selects a scene as a result of the identification process on the confirmation screen will be described.

  FIG. 29 is an explanatory diagram of the configuration of the APP1 area when the identification result is added to the additional data. In FIG. 29, a portion different from the image file of FIG. 3 is indicated by a bold line.

  Compared with the image file of FIG. 3, Makernote IFD is added to the image file of FIG. Information of the identification processing result is stored in the second Makernote IFD.

  A new directory entry is also added to the Exif SubIFD. The added directory entry is composed of a tag indicating the second Makernote IFD and a pointer indicating the storage location of the second Makernote IFD.

  In addition, since a new directory entry is added to the Exif Sub IFD, the storage location of the Exif Sub IFD data area is shifted, so the pointer indicating the storage location of the Exif Sub IFD data area is changed.

  Further, since the second Makernote IFD is added, the IFD1 area is shifted, so that the link indicating the position of IFD1 is also changed in IFD0. In addition, since the second Makernote IFD is added, the size of the data area of APP1 is changed, so the size of the data area of APP1 is also changed.

  Thus, by saving the user-selected scene (in this case, the scene of the identification processing result) in the additional data of the image file, when the image of this image file is printed again, the identification processing and the confirmation screen are displayed. It becomes unnecessary. Further, when the user takes out the memory card 6 from the printer 4 of the present embodiment and inserts the memory card 6 into another printer, this printer does not have a scene identification processing function but performs automatic correction processing. However, the image data can be corrected appropriately.

=== Other Embodiments ===
Although a printer or the like as one embodiment has been described, the above embodiment is for facilitating understanding of the present invention, and is not intended to limit the present invention. The present invention can be changed and improved without departing from the gist thereof, and it is needless to say that the present invention includes equivalents thereof. In particular, the embodiments described below are also included in the present invention.

<About the printer>
In the above-described embodiment, the printer 4 performs scene identification processing, confirmation screen display, and the like. However, the digital still camera 2 may perform scene identification processing, confirmation screen display, and the like. Further, the information processing apparatus that performs the scene identification process and the confirmation screen display is not limited to the printer 4 or the digital still camera 2. For example, an information processing apparatus such as a photo storage that stores a large amount of image files may perform the above-described scene identification processing and confirmation screen display. Of course, a personal computer or a server installed on the Internet may perform the above-described scene identification processing and confirmation screen display.

<About image files>
The image file described above is in the Exif format, but the image file format is not limited to this. Further, the above-described image file is a still image, but may be a moving image. In short, if the image file includes image data and additional data, the scene identification process as described above can be performed.

<About Support Vector Machine>
For the above-described sub classifier 51 and sub partial classifier 61, a classification method using a support vector machine (SVM) is used. However, the method for identifying whether or not the identification target image belongs to a specific scene is not limited to using a support vector machine. For example, pattern recognition such as a neural network may be employed.

<About scene candidate extraction method>
In the above-described embodiment, a scene having a certainty level equal to or higher than a predetermined value is extracted as a scene candidate only when a scene cannot be identified by any of the overall identification process, the partial identification process, and the integrated identification process. However, the method for extracting scene candidates is not limited to this.
FIG. 30 is an explanatory diagram of another processing flow. This process is performed instead of the scene identification process described above.
First, as in the above-described embodiment, the printer-side controller 20 calculates an overall feature amount based on image file information (S801). Next, the landscape discriminator 51L calculates the discriminant value or the Precision corresponding to the discriminant as the certainty factor as in the above-described discrimination processing (S802). Note that the landscape classifier 51L of the above-described embodiment also identifies whether or not the classification target image belongs to a landscape scene. However, the landscape classifier 51L here has a certainty factor based on a discriminant. Just calculate. Similarly, the other sub classifiers 51 also calculate the certainty factor (S803 to S806). Then, the printer-side controller 20 extracts a scene having a certainty factor of a predetermined value or more as a scene candidate (S807), and stores the scene candidate (and the certainty factor) (S808).
Even in this way, it is possible to identify the scene of the image indicated by the image data. It is also possible to compare the identified scene with the scene of the additional data and display a confirmation screen when there is a mismatch.

=== Summary ===
(1) In the above-described embodiment, the printer-side controller 20 acquires shooting scene type data and shooting mode data that are scene information from the additional data added to the image data. Further, the printer-side controller 20 acquires the identification result obtained by the face identification process or the scene identification process (see FIG. 8).
There are cases where the scene indicated by the shooting scene type data or shooting mode data and the scene of the identification result of the scene identification process do not match. In such a case, a confirmation screen for prompting the user to confirm is displayed.
However, if image processing (image correction processing, printing processing, etc.) is advanced after waiting for user confirmation on the confirmation screen, it takes time to start the processing, and it takes time to complete the processing.
Therefore, in the above-described embodiment, before the image correction process (an example of image processing) and the printing process (an example of image processing) are performed on the first image in which the two scenes do not match, the two scenes are An image correction process is started on the second image that is not inconsistent based on the identification process result (see S704 in FIG. 24 and FIG. 25). As a result, the processing of the second image can proceed without waiting for confirmation of the first image, so that image correction and printing processing can be started earlier than in the comparative example.

(2) In the above-described embodiment, when the confirmation screen for the first image in which two scenes do not match is displayed (see S707 in FIG. 24, FIG. 25), the two scenes are not inconsistent 2 The second image is subjected to image correction processing and printing processing based on the identification processing result (S704). As a result, the processing of the second image can proceed without waiting for confirmation of the first image, so that image correction and printing processing can be started earlier than in the comparative example.

(3) The printer-side controller 20 described above executes a job according to the priority order of the job. In the above-described embodiment, for the first image in which the two scenes do not match, a job is created after displaying the confirmation screen (S709). On the other hand, for the second image in which the two scenes are not inconsistent, a job is created before displaying the confirmation screen (S704). As a result, a job for the second image can be created first without waiting for confirmation of the first image, so that image correction and printing processing can be started earlier than in the comparative example.

(4) In the above-described embodiment, the printer-side controller 20 changes the job priority after creating a job in S709 (after creating a job of an image in which two scenes do not match). As a result, the number of images that can be printed in the order of numbers can be increased.

(5) After creating the job in S709, the printer-side controller 20 determines whether or not printing can be performed in the order of the numbers of a plurality of image files to be directly printed (S710). The printer-side controller 20 changes the job order if YES in S710. Thus, the job execution order can be set in the order of the numbers of a plurality of image files to be directly printed.
However, in the first embodiment described above, it is determined in S710 whether or not all objects of direct printing can be printed in numerical order, but the present invention is not limited to this. For example, if the first image cannot be printed in order, but the remaining fourth, fifth, and ninth images can be printed in order, YES is determined in S710 and the job order may be changed. . In this way, the number of images that can be printed in numerical order can be increased (for example, images other than the first can be printed in numerical order).

(6) In the above-described embodiment, if the determination in S710 is NO, the printer 4 does not print the images in the order of the number of the image file. You may display. Thus, the user can notice that the printed images are not arranged in order.

(7) The printer 4 (corresponding to the information processing apparatus) includes the printer-side controller 20 and the display unit 16. Then, the printer-side controller 20 acquires shooting scene type data and shooting mode data that are scene information from the additional data added to the image data. Further, the printer-side controller 20 acquires the identification result obtained by the face identification process or the scene identification process (see FIG. 8). Further, the scene indicated by the shooting scene type data and shooting mode data is compared with the scene obtained as a result of the identification process. If there is an image in which the additional data scene and the identification processing result scene do not match, the printer-side controller 20 displays a confirmation screen on the display unit 16 (see S707).

  The above-described printer-side controller 20 performs two processing before image correction processing (an example of image processing) and printing processing (an example of image processing) are performed on the first image in which two scenes do not match. Image correction processing is started on the second image whose scenes are not inconsistent based on the result of identification processing (see S704 in FIG. 24 and FIG. 25). As a result, the processing of the second image can proceed without waiting for confirmation of the first image, so that image correction and printing processing can be started earlier than in the comparative example.

(8) The above-described memory 23 stores a program for causing the printer 4 to execute the process of FIG. This program identifies, for each image data of a plurality of image data, a code for obtaining scene information of the image data from the additional data added to the image data, and an image scene indicated by the image data based on the image data A code for comparing the scene indicated by the scene information with the identified scene, and a code for displaying a confirmation screen when there is a mismatch image in which the scene indicated by the scene information does not match the identified scene Yes. In addition, this program starts image processing for a matching image in which the scene indicated by the scene information and the identified scene do not match before image processing is performed on the mismatching image. Thereby, image processing can be started early.

It is explanatory drawing of an image processing system. 2 is an explanatory diagram of a configuration of a printer. FIG. It is explanatory drawing of the structure of an image file. FIG. 4A is an explanatory diagram of tags used in IFD0. FIG. 4B is an explanatory diagram of tags used in the Exif SubIFD. It is a correspondence table of mode setting dial settings and data. It is explanatory drawing of the automatic correction function of a printer. It is explanatory drawing of the relationship between the scene of an image, and the correction content. It is a flowchart of the scene identification process by a scene identification part. It is explanatory drawing of the function of a scene identification part. It is a flowchart of a whole identification process. It is explanatory drawing of an identification object table. It is explanatory drawing of the affirmation threshold value of the whole identification process. It is explanatory drawing of Recall and Precision. It is explanatory drawing of a 1st negative threshold value. It is explanatory drawing of a 2nd negative threshold value. FIG. 16A is an explanatory diagram of threshold values in the landscape classifier. FIG. 16B is an explanatory diagram of an outline of the process of the landscape classifier. It is a flowchart of a partial identification process. It is explanatory drawing of the order of the partial image which an evening scene partial identifier selects. It is a Recall and Precision graph when the evening scene image is identified only by the top 10 partial images. FIG. 20A is an explanatory diagram of determination by the linear support vector machine. FIG. 20B is an explanatory diagram of discrimination using a kernel function. It is a flowchart of an integrated identification process. It is a flowchart which shows the flow of the process of the direct print in a reference example. 23A and 23B are explanatory diagrams of an example of a confirmation screen of a reference example. It is a flowchart which shows the flow of the process of the direct print of 1st Embodiment. It is explanatory drawing of an example of the confirmation screen 164 of 1st Embodiment. It is explanatory drawing of another confirmation screen of 1st Embodiment. It is a flowchart which shows the flow of the process of the direct print of 2nd Embodiment. FIG. 28A is an explanatory diagram illustrating an example of a confirmation screen according to the second embodiment. FIG. 28B is an explanatory diagram of another example of the confirmation screen according to the second embodiment. It is explanatory drawing of a structure of an APP1 area | region when an identification result is added to additional data. It is explanatory drawing of another processing flow.

Explanation of symbols

2 Digital still camera, 2A mode setting dial,
4 Printer, 6 Memory card,
10 printing mechanism, 11 head, 12 head control unit, 13 motor,
14 sensors, 15 panel section, 16 display section, 17 input section,
20 printer-side controller, 21 slots, 22 CPU, 23 memory,
24 control unit, 25 drive signal generator,
31 storage unit, 31A image storage unit, 31B result storage unit,
32 face identification unit, 33 scene identification unit, 34 image correction unit, 35 printer control unit,
40 feature quantity acquisition unit, 50 global classifier, 51 sub classifier, 51L landscape classifier,
51S evening scene classifier, 51N night scene classifier, 51F flower classifier, 51R autumn leaves classifier,
60 partial classifiers, 61 sub partial classifiers, 61S evening scene partial classifiers,
61F Flower partial classifier, 61R Autumn colored partial classifier, 70 Integrated classifier,
162 Confirmation screen of reference example, 164 confirmation screen of the first embodiment,
165 Another confirmation screen of the first embodiment,
166 Confirmation screen of the second embodiment, 167 Another confirmation screen of the second embodiment

Claims (8)

From the additional data added to the image data, obtain the scene information of the image data,
Based on the image data, the scene of the image indicated by the image data is identified,
Comparing the scene indicated by the scene information with the identified scene;
An information processing method for displaying a confirmation screen when there is a mismatch image in which the scene indicated by the scene information and the identified scene do not match,
An information processing method, comprising: starting image processing for a matching image in which the scene indicated by the scene information and the identified scene are not mismatched before image processing is performed on the mismatching image. An information processing method according to claim 1,
An information processing method, wherein image processing is performed on the matching image when the confirmation screen is displayed. An information processing method according to claim 1 or claim 2, wherein
Before displaying the confirmation screen, create a job for the matching image,
After displaying the confirmation screen, create a job for the mismatched image,
An information processing method, wherein the processing device executes the job according to the priority order of the job. An information processing method according to claim 3,
An information processing method comprising: changing a priority of the job after creating a job for the mismatched image. An information processing method according to claim 4,
After creating a job for the mismatched image,
If the job can be executed according to the order of a plurality of image data, the priority of the job is changed,
An information processing method, wherein the priority of the job is not changed if the job cannot be executed in the order of the plurality of image data. An information processing method according to claim 5,
An information processing method comprising: displaying a warning screen if the job cannot be executed in the order of the plurality of image data after creating a job for the mismatched image. An information processing apparatus comprising a controller,
The controller is
From the additional data added to the image data, obtain the scene information of the image data,
Based on the image data, the scene of the image indicated by the image data is identified,
Comparing the scene indicated by the scene information with the identified scene;
When there is a mismatch image in which the scene indicated by the scene information does not match the identified scene, a confirmation screen is displayed,
An image processing apparatus that starts image processing on a matching image in which the scene indicated by the scene information and the identified scene do not match before image processing is performed on the mismatching image. In the information processing device,
From the additional data added to the image data, the scene information of the image data is acquired,
Based on the image data, the scene of the image indicated by the image data is identified,
Comparing the scene indicated by the scene information with the identified scene;
When there is a mismatch image in which the scene indicated by the scene information and the identified scene do not match, a confirmation screen is displayed,
A program for starting image processing for a matching image in which the scene indicated by the scene information and the identified scene do not match before image processing is performed on the mismatching image.