JP6003541B2 - Image processing apparatus and program - Google Patents

Description

  The present invention relates to an image processing apparatus and a program for performing processing related to a stored photographic image.

  There is a technique for holding an image and displaying it in various modes. For example, in the technique disclosed in Patent Document 1, on the event occurrence date for a certain user, a decoration image representing the event is combined with the image of the user. The technique disclosed in Patent Document 2 discloses a technique for extracting a frame including a user and his / her spouse on a date set as a wedding anniversary and outputting the frame to a photo frame.

JP 2011-22420 A JP 2011-10007 A

  However, in the above-described conventional technology, the processing is limited to displaying an image that satisfies a predetermined condition on a predetermined day, and therefore, for example, an entertainment according to the usage mode of the device is not performed by the user. Lack of sex.

  The present invention has been made in view of the above circumstances, and an object thereof is to provide an image processing apparatus and a program capable of improving entertainment.

  The present invention for solving the problems of the above conventional example is an image processing apparatus, which is connected to a holding unit that holds a photographic image, and is photographed for each of the photographic images held in the holding unit. Person specifying means for specifying a person, means for storing in association with a photographic image, person identification information representing a person specified as being photographed in the photographic image, and a photograph in which a plurality of person identification information is associated A means for calculating the number of photographic images having a plurality of associated person identification information and a number of photographic images having a plurality of associated person identification information are predetermined. Means for grouping more photographic images than a threshold value, and processing means for processing the grouped photographic images based on the grouped photographic images are included.

  Further, here, the holding unit holds tag information associated with each of the photographic images, and the grouped photographic images are related to the tag information associated with each of the grouped photographic images. Means for estimating tag information that should be commonly associated with the grouped photographic images, and the estimated tag for the photographic images without the estimated tag information among the grouped photographic images. And means for associating information with each other and holding the information in the holding unit. The tag information may include information on the location where the photographic image is captured.

  Furthermore, it includes means for identifying information on a camera used for taking a photographic image held in the holding unit, and acquiring information on a person who is a camera user set in advance in association with the information on the camera. The acquired information on the camera user may be provided for a process of specifying a person related to shooting by the person specifying means.

  According to another aspect of the present invention, there is provided a program for identifying a person who is photographing a computer connected to a holding unit that holds a photographic image for each of the photographic images held in the holding unit. A photographic image in which identification means, a photographic image, and means for storing in association with person identification information representing a person identified as being photographed in the photographic image, and a plurality of person identification information, The means for calculating the number of photographic images having a plurality of associated person identification information in common and the number of photographic images having a plurality of associated person identification information in common are greater than a predetermined threshold. It is intended to function as means for grouping photographic images and processing means for processing the grouped photographic images based on the grouped photographic images.

  According to the present invention, entertainment can be improved.

It is a block diagram showing the example of a structure of the image processing apparatus which concerns on embodiment of this invention. It is explanatory drawing showing the example of the content of the holding | maintenance part which hold | maintains the photographic image made into the object of the image processing apparatus which concerns on embodiment of this invention. It is a functional block diagram showing the example of the image processing apparatus which concerns on embodiment of this invention. It is a flowchart showing the example of the process by the image processing apparatus which concerns on embodiment of this invention. It is a functional block diagram showing another example of the image processing apparatus which concerns on embodiment of this invention. It is a flowchart figure showing the example of the estimation process of the tag information in the image processing apparatus which concerns on embodiment of this invention.

  Embodiments of the present invention will be described with reference to the drawings. As illustrated in FIG. 1, the image processing apparatus 1 according to the embodiment of the present invention includes a control unit 11, a storage unit 12, an operation unit 13, a display unit 14, and a communication unit 15. Is done. Here, the control unit 11 is a program control device such as a CPU, and operates according to a program stored in the storage unit 12.

  The control unit 11 according to the present embodiment specifies a person who is photographed for each of the held photographic images. The control unit 11 stores each photographic image in association with personal identification information representing a person identified as being photographed in the photographic image, and is a photographic image in which a plurality of personal identification information is associated. Thus, the number of photographic images having a plurality of associated person identification information in common is calculated. Then, the control unit 11 groups the photographic images in which the number of photographic images having a plurality of associated person identification information in common is greater than a predetermined threshold value, and groups the photographic images thus grouped. A predetermined process based on the photographic image is executed. The specific processing contents of the control unit 11 will be described later.

The storage unit 12 includes a memory device, a disk device, and the like. The storage unit 12 stores a program executed by the control unit 11. This program is a DVD-ROM (Digital Versatile Disc
The program may be provided by being stored in a computer-readable recording medium such as a read-only memory, or may be distributed via a network and stored in the storage unit 12. The storage unit 12 also operates as a work memory of the control unit 11, and in the present embodiment, the storage unit 12 also functions as a holding unit that holds a photographic image. The holding unit for holding the photographic image may be an external disk device connected to the image processing apparatus 1 via a USB (Universal Serial Bus) or the like, or accessible via the communication unit 15. It may be a storage device.

  The operation unit 13 includes a remote controller and a receiver that receives a signal representing the content of the user's instruction operation in the remote controller. This receiver outputs to the control unit 11 the contents of the user's instruction operation represented by the signal received from the remote controller.

  The display unit 14 may be a display or an output terminal for a home television device. The display unit 14 displays and outputs an image on a display or a television device in accordance with an instruction input from the control unit 11.

  The communication unit 15 is a network interface or the like, and outputs information received via the network to the control unit 11. The communication unit 15 transmits information to the instructed destination via the network according to the instruction input from the control unit 11.

  Here, the operation of the control unit 11 will be described. Prior to that, a photographic image stored in the storage unit 12 which is a holding unit of the present embodiment will be described. As illustrated in FIG. 2, this photographic image is so-called Exif information such as tag information (T; shooting date information, etc.), which is incidental information related to the image information, with respect to the main body (B) of the image information. Are included in association with each other and recorded in the storage unit 12. Further, as will be described later, personal identification information (P) representing a person (or a group of persons) identified as being photographed in the photographic image is associated with this photographic image. Here, the information representing a person or a group of persons may be, for example, a list of the names of the persons or the names of persons included in the group of persons.

  The control unit 11 is connected to a holding unit that holds the photographic image, and can read the photographic image and information associated therewith from the holding unit. As the control unit 11 executes the program stored in the storage unit 12, the image processing apparatus 1 according to the present embodiment functionally has a person specifying unit 21 and a person as illustrated in FIG. A setting unit 22, a frequency calculation unit 23, a grouping unit 24, and a processing unit 25 are included.

  The person specifying unit 21 performs recognition processing of a photographed person for each of the photographic images held in the holding unit. As this recognition processing, what is widely known as face recognition processing may be adopted. The person identifying unit 21 uses, for example, the person's name of the person photographed in each photographic image as person identification information representing the person recognized from the photographic image to be processed (that is, the person photographed in the photographic image). Outputs string information. The person setting unit 22 associates the person identification information output by the person specifying unit 21 with the photographic image processed by the person specifying unit 21 and records it in the holding unit.

  The frequency calculation unit 23 calculates the number of photographic images that are associated with a plurality of person identification information and that have a plurality of associated person identification information in common (the combination of the person identification information is the same). . For example, if a person A, B, C, a person A, D, a person D, a person A, B, C are photographed in the photographic images P1, P2, P3, P4, respectively, the same person group is photographed. The number of photographic images for each photographic image is calculated. In this example, the person groups A, B, and C are “2”, and the person group “A, D” is “1”.

  The grouping unit 24 refers to the result of the calculation by the frequency calculation unit 23, and groups the photographic images in which the number of photographic images having a plurality of associated person identification information in common is greater than a predetermined threshold value. To do. Specifically, the grouping unit 24 uses, as tag information, a common group identifier for photo images in which the number of photo images having a plurality of associated person identification information in common is greater than a predetermined threshold. You can associate them. As an example, if this threshold value is “2”, the same group identifier is assigned to the photographic images P1, P4 in which the persons A, B, C are photographed for the previous photographic images P1, P2, P3, P4. Associate as tag information.

  The processing unit 25 performs processing based on the grouped photographic images for the photographic images grouped by the grouping unit 24. An example of processing by the processing unit 25 is as follows. That is, when the grouping unit 24 associates a common group identifier as tag information, the processing unit 25 adds the group identifier (the group identifier associated as the tag information is referred to as a target identifier here) to the photographic image. Acquire the associated person identification information. The processing unit 25 displays and outputs the acquired person identification information (plurality) on the display unit 14. And the process part 25 requests | requires a user to input the name showing the group of the displayed several person identification information. When the processing unit 25 receives the input of this name from the user, the processing unit 25 associates the input name with the displayed group of person identification information and stores them in the storage unit 12, and a photographic image associated with the target identifier. In addition, the input name is stored in association with the tag information.

  The present embodiment is configured as described above, and operates as follows. The image processing apparatus 1 receives input of photographic images P1, P2, P3, and P4 in which the persons A, B, and C, the persons A and D, the person D, and the persons A, B, and C are respectively photographed. A photographic image is stored in the storage unit 12 (FIG. 4: S1).

  Then, the image processing apparatus 1 recognizes the person photographed in each photographic image, and stores the personal identification information for identifying the photographed person in association with each photographic image (FIG. 4: S2). The image processing apparatus 1 groups photographic images in which the number of photographic images having a common set of associated person identification information is greater than a predetermined threshold value. In the above example, for the previous photographic images P1, P2, P3, and P4, the same group identifier (for example, group identifier “x”) is used for the photographic images P1 and P4 in which the persons A, B, and C are taken. ) As tag information (FIG. 4: S3).

  The image processing apparatus 1 displays a group of person identification information associated with the photographic image associated with the group identifier “x”, and accepts input of name information representing the group of person identification information from the user. . Here, it is assumed that a character string such as “family” is input from the user. The image processing apparatus 1 uses the inputted name as tag information and associates it with the photographic images P1 and P4 associated with the group identifier “x” (FIG. 4: S4).

  The image processing apparatus 1 also reads out a photographic image associated with tag information corresponding to the key from a character string input by the user from the storage unit 12 and displays the read out photographic image. For example, the user inputs a character string “family” as a key. Then, the image processing apparatus 1 reads out the photographic images P1 and P4 associated with the character string “family” as tag information from the storage unit 12 and displays them.

  In the description of the present embodiment so far, as an example, a character string input from a user is associated as tag information with a grouped photographic image. However, the tag information associated with the grouped photographic images may be estimated by the image processing apparatus 1 instead of receiving input from the user.

  Specifically, the image processing apparatus 1 of this example functionally includes a person specifying unit 21, a person setting unit 22, a frequency calculating unit 23, a grouping unit 24, and a processing unit as illustrated in FIG. 25 'and a tag information estimation unit 26. In addition, about the thing which has the structure similar to what was shown in FIG. 3, the same code | symbol is attached | subjected and detailed description is abbreviate | omitted.

  The tag information estimation unit 26 refers to a group of person identification information associated with the photographic images grouped by the grouping unit 24 (hereinafter referred to as grouped photographic images), and is common to these photographic images. And tag information to be associated is estimated. The processing unit 25 ′ associates the estimated tag information with the photographic images without the tag information estimated by the tag information estimation unit 26 among the grouped photographic images.

  Specific tag information estimation processing by the tag information estimation unit 26 is, for example, as follows. The tag information estimation unit 26 refers to the shooting date information of the grouped photographic images and checks whether or not the shooting date is periodic. Specifically, the tag information estimation unit 26 extracts the shooting date information of the grouped photographic images, and rearranges the extracted shooting date information in ascending order or descending order. The tag information estimation unit 26 eliminates duplicate information from the rearranged shooting date information, and there is no duplication and rearranged shooting date information T1, T2,... (For example, Ti <Ti + 1, i = 1). , 2..., ΔTi = Ti + 1−Ti is calculated. The tag information estimation unit 26 then examines the frequency in the range of ΔTi. The range of this value is, for example, “1”, “2”... “7” (1 week), “8”,..., “12”, “13 to 15” (about 2 weeks),. ”(About January),...,“ 360 to 370 ”(about one year)...

  The tag information estimation unit 26 takes out the range of the value that is the maximum value (the most frequent value). For example, if the range of the value corresponding to the maximum value of ΔTi is “28 to 31”, the tag information estimation unit 26 assumes that the grouped photographic images are taken from an event that occurs every month. "Is estimated as tag information to be associated with the grouped photographic images.

  When the range of the value corresponding to the maximum value of ΔTi is “360 to 370 (about 1 year)”, the tag information estimation unit 26, for example, a person associated with the grouped photographic images A photograph in which specific information is extracted, and the birthday of the person represented by any of the extracted person specific information (the person identification information and the birthday information are stored in association with each other) is grouped Check whether it is within one week before and after the image shooting date, and the birthday of the person represented by any of the extracted person identification information is within one week before and after the shooting date of the grouped photo image For example, a tag to associate a character string “b birthday” with the person identification information “p” representing a person with a birthday between one week before and after the shooting date with a grouped photo image Estimate as information.

  In addition, the tag information estimation processing by the tag information estimation unit 26 is not limited to using the shooting date in this way. In another example of tag information estimation processing performed by the tag information estimation unit 26, the tag information estimation unit 26 uses a so-called geotag (representing a place where the photographic image is taken on a photographic image grouped by the grouping unit 24. If the tag information is associated with various types of position information (latitude and longitude information, etc.), as well as position information set arbitrarily, such as latitude and longitude, and position information that is arbitrarily set Get the name of the place. In addition, since the technique which obtains the name of a place from a geotag is widely known, detailed description is abbreviate | omitted.

  The tag information estimation unit 26 extracts a character string portion (for example, “sport field”) included in common in the acquired name. Then, a tag information candidate (for example, “sports”, etc.) determined in advance in association with this character string is estimated as tag information to be associated with the grouped photographic images.

  Another tag information estimation process of the tag information estimation unit 26 will be described next. This tag information estimation process is illustrated in FIG. That is, as illustrated in FIG. 6, the tag information estimation unit 26 acquires person identification information (plurality) associated with the photographic images grouped by the grouping unit 24 (S11). Then, the tag information estimation unit 26 selects any of the acquired person identification information (person identification information that has not been selected in the process so far) as attention information (S12). The tag information estimation unit 26 extracts, from the storage unit 12 (that is, the holding unit), a photographic image that is different from the grouped photographic images and is associated with the person identification information that has become attention information (S13). ).

  And the tag information estimation part 26 calculates the appearance frequency of the tag information linked | related with the extracted photograph image (S14). If there is an appearance frequency of the tag information calculated using the other person identification information as the attention information, it is cumulatively added to the information on the appearance frequency. If all of the person identification information acquired in process S11 has not been selected as the attention information, the tag information estimation unit 26 returns to process S12 and continues the process (loop).

By this processing, the tag information estimation unit 26 uses the frequency information of the tag information already attached to other photographic images in which at least one person photographed in the grouped photographic images is captured. obtain. As an example,
・ People A, B, and C are captured in the photographic image P11,
・ People A and B are captured in photo image P12,
A person C is captured in the photographic image P13,
・ People A and C are captured in photo image P14,
It is assumed that the person A is captured in the photographic image P15. At this time,
The tag information “A-kun” and “members of baseball team X” are associated with the photo image P11,
-Tag information "members of baseball team X" and "B-kun" are associated with the photo image P12,
-Tag information "C" is associated with the photo image P13,
-Tag information "member of baseball team X" is associated with the photo image P14,
It is assumed that tag information “A” is associated with the photo image P15.

  Then, in the above processing of the tag information estimation unit 26, assuming that information representing the persons A, B, and C as person identification information is associated with the grouped photographic images, the tag information “A-kun” The appearance frequency is “4” (because the person identification information corresponding to each person A, B, and C is counted as attention information redundantly for the photo image P11), and the tag information “member of baseball team X” The appearance frequency information is obtained as “7” for the appearance frequency and “2” for the tag information “B-kun”.

  The tag information estimation unit 26 estimates at least one tag information as the tag information to be associated with the grouped photographic images in the descending order of appearance frequency (S15), and ends the process. In the case of the above specific example, for example, tag information “member of baseball team X” having a relatively high appearance frequency is estimated as tag information to be associated with grouped photographic images. Although the tag information arbitrarily set by the user is taken as an example here, the tag information for which the appearance frequency is compared may be tag information set based on Geotag or other Exif information. Good.

  If the appearance frequency is lower than the threshold value determined in advance, even if the appearance frequency is relatively high, the tag information to be associated with the grouped photographic images may not be used. Here, as a method of determining the threshold value, for example, a ratio to the number of photographic images held in the holding unit may be used, or a predetermined absolute number may be used.

  Further, the tag information estimation unit 26 recognizes character strings included in the grouped photographic images by OCR (optical character recognition) processing, and detects character strings recognized at a relatively high frequency from the grouped photographic images. Further, estimation based on grouped photographic images, such as estimation as tag information, may be performed.

  In the above description, the person specifying unit 21 recognizes a person from a photographic image. However, in addition to the person recognized from the photographic image, the person specifying unit 21 is present on the spot as follows. A person may be specified. In other words, in this example, camera information (which may be information indicating a model or unique identification information (serial number, etc.)) and a person who is a camera user who uses the camera represented by the information in advance. Information (person identification information) is associated and stored in the storage unit 12 as a camera database. This camera database may be set in advance by the user and stored in the storage unit 12.

  The person specifying unit 21 performs processing for recognizing a person photographed in the photographic image with respect to the photographic image to be processed and is attached to the photographic image to be processed. Get camera information. Then, the personal identification information of the camera user of the camera represented by the acquired information is acquired with reference to the camera database stored in the storage unit 12. The person specifying unit 21 then sets the union of the person identification information of the person photographed in the photographic image and the person identification information acquired by referring to the camera database (if the camera user is photographed, Is recorded in the holding unit in association with the photographic image to be processed. According to this example, it is possible to perform processing including the person who was at the shooting site.

  In addition, when the image processing apparatus 1 reads out a photographic image from the storage unit 12 and displays it according to an instruction from the user, the image processing apparatus 1 refers to tag information associated with the photographic image to be displayed, and the tag information satisfies a predetermined condition. When satisfied, when an effect method is set in advance in association with the condition, the effect process may be performed by the preset effect method.

  Specifically, assuming that the tag information includes a production method such as playing the song “Happy Birthday” in advance for the condition that the tag information includes the character string “birthday”, the tag information When displaying a photographic image that includes a character string including “birthday”, the image processing apparatus 1 reproduces a song “Happy Birthday”.

  In addition, regarding the condition that the tag information includes the character string “Marriage Anniversary”, the music that was popular in the year set separately is played, or the headline information of the incident that occurred in that year is An effect method may be set, such as obtaining (for example, obtaining from wikipedia) via the display.

  According to the present embodiment, when a group of persons to be repeatedly photographed is recognized, tag information related to the group of persons can be set, and the tag information can be used in various ways such as performing effects based on the set tag information. Since it can be used, entertainment can be improved.

DESCRIPTION OF SYMBOLS 1 Image processing apparatus, 11 Control part, 12 Storage part, 13 Operation part, 14 Display part, 15 Communication part, 21 Person specific part, 22 Person setting part, 23 Frequency calculation part, 24 Grouping part, 25, 25 'Process Part, 26 tag information estimation part.

Claims (4)

Connected to the holding unit that holds the photographic image,
For each of the photographic images held in the holding unit, person specifying means for specifying the person being photographed,
Means for identifying information of a camera used for taking a photographic image held in the holding unit, and acquiring information of a person who is a camera user set in advance in association with the information of the camera;
For each photographic image, the union of the person identification information of the person identified as being photographed in the photographic image and the person identification information of the person who is the camera user of the camera used for photographing the photographic image Means for storing in association with the photographic image;
Means for calculating the number of photographic images associated with a plurality of person identification information and having a plurality of associated person identification information in common;
Means for grouping photographic images in which the number of photographic images having a plurality of associated person identification information in common is greater than a predetermined threshold;
Processing means for performing processing based on the grouped photo images for the grouped photo images;
An image processing apparatus comprising: The image processing apparatus according to claim 1,
The holding unit holds tag information associated with each photographic image,
For the grouped photographic images, based on tag information associated with each of the grouped photographic images, means for estimating tag information that should be commonly associated with the grouped photographic images;
Among the grouped photographic images, for the photographic image without the estimated tag information, means for associating the estimated tag information with the holding unit,
An image processing apparatus further comprising: The image processing apparatus according to claim 2,
An image processing apparatus in which the tag information includes information on a place where a photographic image is taken. A computer connected to a holding unit that holds photographic images
For each of the photographic images held in the holding unit, person specifying means for specifying the person being photographed,
Means for identifying information of a camera used for taking a photographic image held in the holding unit, and acquiring information of a person who is a camera user set in advance in association with the information of the camera;
For each photographic image, the union of the person identification information of the person identified as being photographed in the photographic image and the person identification information of the person who is the camera user of the camera used for photographing the photographic image Means for storing in association with the photographic image;
Means for calculating the number of photographic images associated with a plurality of person identification information and having a plurality of associated person identification information in common;
Means for grouping photographic images in which the number of photographic images having a plurality of associated person identification information in common is greater than a predetermined threshold;
Processing means for performing processing based on the grouped photo images for the grouped photo images;
Program to function as.

Images