JP6891744B2 - Image map creation device, display device and program - Google Patents

Description

The present invention relates to an image map creating device, a display device and a program.

It is not easy to find the most suitable data for the purpose desired by the user from a large amount of data. Therefore, conventionally, a technique for creating a data map capable of grasping an overall bird's-eye view position has been disclosed so that the optimum data for a purpose can be searched from a large number of data (for example, Patent Document 1). ..

Japanese Unexamined Patent Publication No. 2008-84151

By the way, the need to find the most suitable data for the purpose desired by the user from a large amount of data is not limited to document data. For example, there is a need to find the most suitable image data for a purpose desired by a user from a large number of image data.

Therefore, an object of the present invention is to provide an image map creation device, a display device, and a program for finding an optimum image for a purpose desired by a user.

The present invention solves the above problems by the following solutions.
The first invention comprises a word extraction means for analyzing an image to extract a plurality of words, a classification word generation means for clustering a plurality of words extracted by the word extraction means to generate a classification word, and the classification word. It is an image map creation device including a map creation means for creating a map by counting the taxonomy generated by the generation means.
The second invention includes, in the image map creating apparatus of the first invention, a classification means for dividing a plurality of words extracted by the word extraction means into a plurality of groups, and the classification word generation means is based on the classification means. The words of each of the divided groups are clustered to generate a classification word, and the map creation means creates the map classified by each classification word of at least two groups out of the plurality of groups divided by the classification means. It is an image map creation device characterized by creating.
The third invention is the image map creating apparatus of the first invention or the second invention, in which the classified word generating means uses a word corresponding to a higher concept as the classified word from a plurality of words extracted by the word extracting means. It is an image map creation device characterized by generating.
The fourth invention is in any of the image map creating devices from the first invention to the third invention, in which the classified word generating means is based on the appearance frequency of a plurality of words extracted by the word extracting means. It is an image map creation device characterized by generating the classification word.
In a fifth aspect of the present invention, in any of the image map creating devices from the first invention to the third invention, an image is analyzed to extract a plurality of feature images representing the features of the image, and the image is located in the area of the image. The image ratio calculating means for calculating the ratio occupied by the feature image extracted with respect to the feature image is provided, the word extracting means extracts a word representing the feature of the feature image, and the classified word generating means is the image ratio calculating means. The image map creating apparatus is characterized in that the classified word is generated from the word extracted by the word extracting means according to the ratio calculated by the above.
A sixth aspect of the present invention is the image map creating apparatus according to any one of the first to fifth inventions, wherein the word extraction means generates an explanatory text by referring to words associated with each feature of the image. It is an image map creation device characterized by extracting a plurality of words from the generated explanatory text.
According to a seventh aspect of the present invention, in any of the image map creating devices from the first invention to the sixth invention, the classified word generating means has a plurality of words extracted from each of the plurality of images by the word extracting means. It is an image map creation device characterized by generating classification words by clustering.
The eighth invention is a display device communicatively connected to any of the image map creating devices from the first invention to the seventh invention, and the map created by the map creating means is displayed. Corresponds to the map output means to be output, the element receiving means shown in the map output by the map output means and accepting the selection of the element associated with the classification term, and the element received by the element receiving means. The display device is provided with an image output means for outputting an image to be output.
According to a ninth aspect of the present invention, in the display device of the eighth invention, when there are a plurality of images corresponding to the elements received by the element receiving means, the image output means responds to the similarity of the plurality of images. It is a display device characterized in that each image is arranged and the plurality of images are output as related images.
A tenth aspect of the invention is characterized in that, in the display device of the ninth invention, the similarity is calculated based on at least one of a feature amount of each image and a word corresponding to each image. It is a display device.
The eleventh invention is the display device of the ninth invention or the tenth invention, in which the related image has a similarity based on the length or thickness of a line between images or the depth direction from an image as a reference. It is a display device characterized by being represented by the distance of.
A twelfth invention is a program for operating a computer as an image map creating device according to any one of the first to seventh inventions.
The thirteenth invention is a program for operating a computer as any display device from the eighth invention to the eleventh invention.

According to the present invention, it is possible to provide an image map creation device, a display device, and a program for finding an optimum image for a desired purpose by a user.

It is an overall schematic diagram of the image map output system which concerns on this embodiment. It is a functional block diagram of the image map creation device and the display device which concerns on this embodiment. It is a figure which shows the example of the map created by the image map making apparatus which concerns on this embodiment. It is a flowchart which shows the map output processing of the image map output system which concerns on this embodiment. It is a figure which shows the display example in the display device which concerns on this embodiment. It is a flowchart which shows the map creation process of the image map creation apparatus which concerns on this embodiment. It is a figure which shows the specific example in the map making process by the image map making apparatus which concerns on this embodiment. It is a figure which shows the specific example in the map making process by the image map making apparatus which concerns on this embodiment. It is a flowchart which shows the related image output processing of the image map output system which concerns on this embodiment. It is a flowchart which shows the related image generation processing of the image map making apparatus which concerns on this embodiment. It is a figure which shows the display example in the display device which concerns on this embodiment.

Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. It should be noted that this is only an example, and the technical scope of the present invention is not limited to this.
(Embodiment)
<Image map output system 100>
FIG. 1 is an overall schematic view of the image map output system 100 according to the present embodiment.
FIG. 2 is a functional block diagram of the image map creating device 1 and the display device 4 according to the present embodiment.
FIG. 3 is a diagram showing an example of a map 80 created by the image map creating device 1 according to the present embodiment.

The image map output system 100 shown in FIG. 1 is a system for classifying image data stored in an image DB (database) 3 by classification words based on image characteristics and creating a map that visualizes the classification of images. is there. Further, when the image map output system 100 accepts the selection of one of the elements representing the set of image data constituting the map, the image map output system 100 outputs the image data included in the selected element.
In the following description, it is assumed that the image data stored in the image DB 3 is, for example, an image data of a painting or the like exhibited in a museum or a museum. Then, the image data is assumed to have, for example, data including a museum ID (IDentification) for identifying a museum or the like that owns the image as metadata.
As an example, in the image map output system 100, a map related to a painting held in a museum or a museum is created for each museum or the museum.

The image map output system 100 includes an image map creation device 1, an image DB 3, and a display device 4. The image map creating device 1, the image DB 3, and the display device 4 are communicably connected via the communication network N. The communication network N is, for example, an Internet line or the like, regardless of whether it is wired or wireless.

<Image map creation device 1>
The image map creating device 1 uses the image data stored in the image DB 3. The image map creating device 1 analyzes the image indicated by the image data and extracts a plurality of words representing the contents of the image. The image map creating device 1 may associate the extracted plurality of words with the image data and store them in the image DB 3. Then, the image map creation device 1 generates, for example, a word corresponding to a superordinate concept from the extracted plurality of words, uses the generated word as a classification word to be used in the map, and uses the classification word, for example, in a bubble chart. Create a represented map.

The image map creating device 1 is, for example, a server. The image map creating device 1 may also be a personal computer (PC) or the like.
As shown in FIG. 2A, the image map creating device 1 includes a control unit 10, a storage unit 20, and a communication interface unit 29.
The control unit 10 is a central processing unit (CPU) that controls the entire image map creation device 1. The control unit 10 appropriately reads and executes the operating system (OS) and the application program stored in the storage unit 20 to cooperate with the above-mentioned hardware and execute various functions.

The control unit 10 includes an image reception unit 11, a word extraction unit 12 (word extraction means), a classification unit 13 (classification means), a classification word generation unit 14 (classification word generation means), and a map creation unit 15 (map). A creation means), a map registration unit 16, and a related image generation unit 17.
The image receiving unit 11 receives image data to be created as a map. The image receiving unit 11 receives, for example, image data of a painting owned by the target museum stored in the image DB 3 based on the museum ID contained in the metadata of the image data.

The word extraction unit 12 extracts a word representing the content of the received image data.
The word extraction unit 12 uses, for example, "DenseCap", which is known explanatory text generation software, to generate explanatory text regarding image data. When "DenseCap" is used, the explanatory text generated by the word extraction unit 12 is written in English. The word extraction unit 12 may use known explanatory text generation software "NeuralTalk2" or the like instead of "DenseCap". However, since "DenseCap" that generates a plurality of explanatory texts divides the image into a plurality of regions and generates explanatory texts for each region, many words can be obtained from the explanatory texts related to the image.

Next, when the explanation is written in English, the word extraction unit 12 translates the explanation in English into Japanese. The word extraction unit 12 can translate the explanatory text using, for example, known automatic translation software. In addition, the word extraction unit 12 may translate from English to Japanese by communicating with a server (not shown) that provides a translation web service connected via the communication network N. ..

In the case of "DenseCap" and "NeuralTalk2", a learning device learned using the current image data is used to generate the explanatory text. Therefore, "DenseCap" or the like generates an explanatory text using an ordinary object from the image data as a learning result. On the other hand, since the image data of paintings in museums and the like includes the image data of paintings of old times, erroneous conversion may occur in "DenseCap" and the like. For example, a person's hand may erroneously convert the Bible into a tablet, which is now commonplace. Therefore, in that case, it is necessary to replace the tablet included in the generated explanation with the Bible, or delete the explanation of things that are not in the era when the painting is the motif.

After that, the word extraction unit 12 extracts a plurality of words from the explanatory text. Specifically, the word extraction unit 12 extracts a plurality of words from the explanatory text, for example, by performing morphological analysis on the explanatory text. Here, the words extracted by the word extraction unit 12 include nouns, adjectives, verbs, and the like.
In the above description, the generation of the explanatory text and the extraction of words using "DenseCap" have been described, but the present invention is not limited to this.

The classification unit 13 divides words extracted from a plurality of image data for which a map is created into a plurality of groups. Here, the plurality of image data for which the map is created is, for example, image data of a plurality of paintings owned by the target museum.
The classification unit 13 can be divided into groups such as nouns, adjectives, and verbs by filtering by part of speech from words to be processed that are acquired based on a plurality of image data for which a map is to be created, for example.
The classification unit 13 excludes pronouns such as "he" and "kore", suffixes such as "side" and "made", and adverb and count nouns such as "middle" and "part". You may. Further, the classification by the classification unit 13 is not limited to the part of speech. As an example, the classification by the classification unit 13 may be based on a predetermined category (genre).

The classification word generation unit 14 clusters the words to be processed to generate the classification word.
For example, the classification word generation unit 14 superordinates the word to be processed into a classification word. As an example, when the words "girl" and "male" are extracted by the word extraction unit 12, the classification word generation unit 14 uses "person", which is a superordinate concept word common to "girl" and "male". , May be generated as a taxonomy.
Further, the classification word generation unit 14 may use a word having a high frequency of occurrence as a classification word based on the frequency of appearance of the word to be processed. Specifically, after storing the words that have appeared in association with the number of occurrences, the classification word generation unit 14 has the highest n words (n is an integer of 1 or more) from the one with the largest number of occurrences, or Words having a number of occurrences equal to or greater than a certain threshold may be acquired as classification words.

Further, the classification word generation unit 14 may select a word to be super-conceptualized from the words having a high frequency of occurrence of the word to be processed, and use the word of the super-concept as the classification word.
Furthermore, when the words to be processed are classified by the classification unit 13, the classification word generation unit 14 may generate the classification words by clustering for each classified group.
In addition, the classification word generation unit 14 may generate classification words by clustering based on the similarity of word meanings.

The map creation unit 15 arranges the classification words around each group grouped by the classification unit 13, counts the number of each corresponding image data at the position corresponding to the classification word, and makes a circle area (element). A map 80, which is a bubble chart represented by the size of, is created.
Here, the map creation unit 15 determines whether or not there is an explanatory text including all the classification words when the position corresponding to the classification word includes a plurality of classification words, that is, when there are two or more axes. Judge about the data. Then, when there is an explanatory text including all the classification words, the map creation unit 15 associates the image ID of the image data with the element ID of the corresponding circular region. Here, the image ID is identification information for identifying the image data. The element ID is identification information for identifying the circular area included in the map 80.

FIG. 3 shows an example of the map 80 when two groups are selected from the grouping by the classification unit 13. In the map 80, the vertical axis is a cluster representative word, the horizontal axis is an adjective, and the number of image data classified by both classification words is represented by the size of a circular region at the intersection of the vertical axis and the horizontal axis. It is a display mode in the two-dimensional matrix arranged in the above.
Here, when a plurality of classification words correspond to one image data, a certain image data may be included in a plurality of circular regions. For example, when a certain image data includes a description of "round and red fruit", the image data includes "round" and "fruit" and "red" and "fruit" as classification words. Therefore, this image data is included in both the circular area at the position where the "round" and the "fruit" intersect and the circular area at the position where the "red" and the "fruit" intersect.

Returning to FIG. 2A, the map registration unit 16 stores the map 80 created by the map creation unit 15 in the map storage unit 22. Further, the map registration unit 16 stores the image ID of the image data constituting the map 80 in the map storage unit 22 in association with the element ID representing the circular area.
The related image generation unit 17 generates related image data in which image data associated with the circular region included in the map 80 is arranged based on the degree of similarity.
Here, the degree of similarity represents the degree of similarity of the image data of each selected combination by selecting a combination of two image data from a plurality of image data associated with the circular region.
The similarity may be calculated based on the features of the two image data. The feature amount is, for example, the degree of similarity of the color tones constituting the two image data. Further, the degree of similarity may be the degree of matching of words corresponding to the two image data or the degree of similarity. The degree of similarity indicates that the two image data are similar as the degree is higher.

More specifically, the related image generation unit 17 includes a plurality of image data associated with the circular region, and expresses the calculated similarity by the length of the line connecting the two image data and the thickness of the line. Generate related image data. The related image data may be connected by a line when the similarity between the two image data is equal to or higher than the threshold value. In addition, the related image data may express the degree of similarity by the distance in the depth direction from the reference image.

The storage unit 20 is a storage area for a hard disk, a semiconductor memory element, or the like for storing programs, data, and the like necessary for the control unit 10 to execute various processes.
The storage unit 20 includes a program storage unit 21 and a map storage unit 22.
The program storage unit 21 is a storage area for storing various programs. The program storage unit 21 stores the map creation program 21a in addition to the above-mentioned known explanatory text generation software and known automatic translation software. The map creation program 21a is a program for performing various functions executed by the control unit 10 of the image map creation device 1. The map creation program 21a may include known explanatory text generation software and known automatic translation software.

The map storage unit 22 is a storage area for storing the data of the map 80 created by the map creation unit 15.
In addition to the data of the map 80, the map storage unit 22 stores the image ID of each image included in the circular area for each circular area constituting the map 80.
The communication interface unit 29 is an interface for performing communication with the image DB 3 and the display device 4 via the communication network N.

<Image DB3>
The image DB 3 shown in FIG. 1 is, for example, an image database server having a storage area such as a hard disk and a semiconductor memory element. The image DB 3 stores image data in association with the image ID. In this example, the image DB 3 may physically or logically store the image data for each museum. Further, the image DB 3 may receive and store image data from another terminal or the like (not shown). Although not shown, the image DB 3 includes a control unit, a storage unit, a communication interface unit, and the like.

<Display device 4>
The display device 4 is, for example, a portable device having a computer function represented by a tablet. The display device 4 may also be a smartphone, a mobile phone, or the like, or a PC or the like.
The display device 4 displays a map 80 of an image desired by the user. Further, the display device 4 receives the displayed elements of the map 80, extracts the image data corresponding to the elements from the image DB 3, and displays the elements.
As shown in FIG. 2B, the display device 4 includes a control unit 40, a storage unit 50, a touch panel display 57, and a communication interface unit 59.

The control unit 40 is a CPU that controls the entire display device 4. The control unit 40 executes various functions in cooperation with the above-mentioned hardware by appropriately reading and executing the OS and application programs stored in the storage unit 50.
The control unit 40 includes a map output unit 41 (map output means), an element reception unit 42 (element reception means), and an image output unit 43 (image output means).
The map output unit 41 displays the data of the map 80 created by the map creation unit 15 of the image map creation device 1 or registered in the map storage unit 22 by the map registration unit 16 of the image map creation device 1 on the touch panel display 57. Output to. As a result, the user of the display device 4 can visually recognize the map 80 displayed on the touch panel display 57.
The element receiving unit 42 receives one circular area selected by the user on the screen on which the map 80 is displayed.
The image output unit 43 outputs the related image data corresponding to the received circular area generated by the related image generation unit 17 of the image map creation device 1 to the touch panel display 57.

The storage unit 50 is a storage area such as a semiconductor memory element for storing programs, data, and the like necessary for the control unit 40 to execute various processes. The storage unit 50 includes a program storage unit 51.
The program storage unit 51 is a storage area for storing various programs. The program storage unit 51 stores the image display program 51a. The image display program 51a is a program for performing various functions executed by the control unit 40.
The touch panel display 57 has a function as a display unit composed of a liquid crystal panel or the like and a function as an input unit for detecting a touch input by a finger or the like from a user.
The communication interface unit 59 is an interface for communicating with the image map creating device 1 and the like via the communication network N.

Here, the computer means an information processing device including a control unit, a storage device, and the like, and the image map creation device 1 and the display device 4 are information processing devices including a control unit, a storage unit, and the like, respectively, and are computers. Included in the concept of.

<Processing of image map output system 100>
Next, a process of displaying the map 80 on the display device 4 will be described.
FIG. 4 is a flowchart showing a map output process of the image map output system according to the present embodiment.
FIG. 5 is a diagram showing a display example on the display device 4 according to the present embodiment.
FIG. 6 is a flowchart showing a map creation process of the image map creation device 1 according to the present embodiment.
7 and 8 are diagrams showing a specific example in the map creation process by the image map creation device 1 according to the present embodiment.

In step S (hereinafter, simply referred to as “S”) 10 of FIG. 4, the control unit 40 of the display device 4 executes the image display program 51a. Then, the control unit 40 outputs the museum selection screen 70 to the touch panel display 57 of the display device 4.
FIG. 5A shows an example of the museum selection screen 70. After executing the image display program 51a, the control unit 40 may make an inquiry to the image map creation device 1 and output the museum selection screen 70 transmitted by the image map creation device 1. Further, the control unit 40 may output the museum selection screen 70 prepared in advance in the image display program 51a.
The user of the display device 4 checks the check box on the left side of the name of the museum to which the map is to be displayed, and selects the execute button. By doing so, in S11, the control unit 40 accepts the selection of the museum name and transmits the museum ID of the museum to the image map creation device 1. Of course, the control unit 40 may transmit data that can identify the museum other than the museum ID.
In S12, the control unit 10 of the image map creation device 1 receives the museum ID.
In S13, the control unit 10 performs a map creation process.

Here, the map creation process will be described with reference to FIG.
In S20 of FIG. 6, the control unit 10 (image reception unit 11) of the image map creation device 1 extracts the image data having the received museum ID from the image DB 3.
In S21, the control unit 10 (word extraction unit 12) selects one image data from the extracted image data, verbalizes the content of the image indicated by the image data, and extracts a word.
FIG. 7A is an example of the image 60 shown by the image data to be verbalized.
FIG. 7B is an example of the result image 61 and the caption 62 when "DenseCap" is used for the image 60. In "DenseCap", a sentence is output for each area of the image 60. Then, as the result image 61 shows, a plurality of regions are generated in the image 60, so that a plurality of sentences are also generated. When the number of pixels of the image 60 is 640 px (pixels), "DenseCap" generates about 80 captions 62.

FIG. 7C is an example of a translated sentence (explanatory sentence) 63 translated from the caption 62.
The control unit 10 may store the translated sentence 63 in the image DB 3 in association with the image data.
FIG. 8A shows result data 64 obtained by performing morphological analysis on the translated sentence 63 shown in FIG. 7C.
FIG. 8B shows an example of a word table 65 in which a word created based on the result data 64 and a part of speech are associated with each other.

Returning to FIG. 6, in S22, the control unit 10 determines whether or not all the extracted image data has been verbalized to extract words. When processing is performed on all the extracted image data (S22: YES), the control unit 10 shifts the processing to S23. On the other hand, when all the extracted image data has not been processed (S22: NO), the control unit 10 shifts the processing to S21 and performs a process of verbalizing the image data that has not yet been verbalized. Do.

In S23, the control unit 10 (classification unit 13, classification word generation unit 14) divides the words extracted from all the extracted image data into two groups and clusters them in each group. The control unit 10 is divided into two groups, for example, a noun and an adjective. As a result, verbs and the like that are not included in the two groups are excluded. Further, the control unit 10 performs clustering for each group and extracts the classification words. At that time, the control unit 10 may target words whose frequency of appearance is equal to or higher than a certain threshold value for clustering. Further, the number of classified words may be limited to a predetermined number according to the frequency of occurrence of words.
In S24, the control unit 10 (map creation unit 15) sets the two groups on the vertical axis and the horizontal axis, and counts the number of image data including the classification words obtained by counting at the positions where the classification words of each axis intersect. A map 80 of a bubble chart represented by the size of a circular area is created.
In S25, the control unit 10 (map registration unit 16) stores the correspondence between the created map 80 data, the element ID indicating the circular area, and the image data ID in the map storage unit 22. After that, the control unit 10 ends this process and shifts the process to FIG.

In this example, the map 80 is created after the image data is verbalized and stored in the map storage unit 22 as an example, but the present invention is not limited to this. For example, the image data stored in the image DB 3 in advance may be verbalized, words may be associated with the image data, and the map creation process may be performed after S23.

Returning to FIG. 4, in S14, the control unit 10 transmits the created map 80 data to the display device 4. After that, the control unit 10 ends this process.
In S15, the control unit 40 (map output unit 41) of the display device 4 receives the data of the map 80 and outputs the map display screen 71 including the map 80 to the touch panel display 57.
FIG. 5B shows an example of the map display screen 71 displayed on the touch panel display 57. In the map 80 displayed on the map display screen 71, the characteristics of the painting of the selected museum are shown in an easy-to-understand manner by the classification word, so that the user can easily find the desired image.
After that, the control unit 40 ends this process.

Next, a process of outputting image data included in the circle area by selecting one circle area shown in the map 80 will be described.
FIG. 9 is a flowchart showing a related image output process of the image map output system 100 according to the present embodiment.
FIG. 10 is a flowchart showing a related image generation process of the image map creating device 1 according to the present embodiment.
FIG. 11 is a diagram showing a display example in the display device 4 according to the present embodiment.

For example, as shown in FIG. 5B, when the user selects one circular area (bubble) on the map display screen 71, in S30 of FIG. 9, the control unit 40 (element reception unit 42) of the display device 4 is selected. ) Accepts selections for the circular area on the map 80.
In S31, the control unit 40 transmits an element ID that identifies the selected circular region to the image map creation device 1.
In S32, the control unit 10 of the image map creation device 1 receives the element ID.
In S33, the control unit 10 (related image generation unit 17) performs the related image generation process.

Here, the related image generation process will be described with reference to FIG.
In S40, the control unit 10 extracts the image data associated with the element ID from the image DB 3. Specifically, the control unit 10 refers to the map storage unit 22 to specify the image ID of the image data associated with the element ID. Then, the control unit 10 extracts the image data corresponding to the image ID from the image DB 3.
In S41, the control unit 10 calculates the similarity of the extracted image data. When a plurality of image data are extracted, the control unit 10 calculates the similarity for each combination of the two image data.

In S42, the control unit 10 generates a related image 90 in which each image data is arranged according to the calculated similarity.
FIG. 11 shows an example of a related image screen 72 including the related image 90. The related image 90 is formed by arranging seven image data included in the circular region selected by the user in FIG. 5B based on the degree of similarity. Here, in the related image 90, of the seven image data included in the circular region selected by the user, two image data are connected by lines having different thicknesses. The thickness of this line represents the similarity between the two image data.
Returning to FIG. 10, after that, the control unit 10 ends this process and shifts the process to FIG.

Returning to FIG. 9, in S34, the control unit 10 transmits the generated data of the related image 90 to the display device 4. After that, the control unit 10 ends this process.
In S35, the control unit 40 of the display device 4 receives the data of the related image 90 and causes the touch panel display 57 to display the related image 90. After that, the control unit 40 ends this process.

As described above, the image map output system 100 of the present embodiment has the following effects.
(1) The image map creation device 1 performs morphological analysis on the explanatory text relating to the image data, extracts the classified words from the obtained words, and creates a map 80 using the classified words. Therefore, the classification word extracted from the extracted word can represent the feature of the image. Further, since the map 80 shows the features of the image data by the classification word, it is possible to find the most suitable image for the purpose desired by the user by referring to the map 80.
(2) The image map creation device 1 divides a plurality of words into a plurality of groups, clusters the words in each group to generate a classification word, and uses each classification word of at least two groups out of the divided plurality of groups. Create a classified map 80. Therefore, the map 80 can be made into two or more dimensions.

(3) The image map creating device 1 extracts a superordinate concept from a plurality of extracted words and generates a classified word. Therefore, the map 80 using the words of the superordinate concept can be created, and the number of the classified words can be suppressed to the optimum number.
(4) Since the image map creation device 1 generates a classification word by clustering a plurality of words extracted from each of the plurality of image data, it is possible to generate a common classification word for the plurality of image data, and it is easy to classify. it can.
(5) The image map creation device 1 outputs the image data corresponding to the circle area according to the acceptance of the selection of the circle area associated with the classification word shown in the map 80. You can see the corresponding image data.

(6) When there are a plurality of image data corresponding to the circular region, the image map creating device 1 arranges each image data according to the similarity of the plurality of image data, and the related image 90 including the plurality of image data. Is output to the display device 4, so that the display device 4 can show the degree of similarity of each image data in an easy-to-understand manner by arranging the image data corresponding to the degree of similarity.
(7) Since the similarity is calculated based on at least one of the feature amount of each image data and the word corresponding to each image data, the feature of each image data can be reflected.

Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments. Moreover, the effects described in the embodiments are merely a list of the most preferable effects arising from the present invention, and the effects according to the present invention are not limited to those described in the embodiments. The above-described embodiment and the modified form described later can be used in combination as appropriate, but detailed description thereof will be omitted.

(Transformed form)
(1) In the present embodiment, the example of dividing into two groups and clustering in each group has been described as an example, but the present invention is not limited to this. It may be clustered as a whole without being divided into groups, or it may be divided into three or more groups and clustered in each group.
(2) In the present embodiment, the example of creating a map of a bubble chart has been described, but the present invention is not limited to this. A map that can be displayed by other methods may be used as long as the overall position can be grasped from a bird's-eye view.
(3) In the present embodiment, a map for creating a map for a plurality of image data has been described, but the present invention is not limited to this. A map for one image data may be created. In that case, the number in the circle area displayed on the map is all 1.

(4) In the present embodiment, a superordinate concept is extracted from a plurality of extracted words to generate a classified word as an example, but the present embodiment is not limited to this. For example, a classification word may be generated based on the frequency of occurrence of a plurality of extracted words. Further, for example, the control unit is provided with an image ratio calculation means for calculating the ratio of the area of the image (feature image) that is the source of the extracted word to the total area of the image data. The classification word may be acquired according to the calculated ratio, such as acquisition as the classification word.
(5) In the present embodiment, the image DB has been described as being connected via a communication network, but the present invention is not limited to this. For example, in the present embodiment, it may be stored in the storage unit of the image map creating device.

(6) In the present embodiment, an example in which image data of a painting held in a museum or a museum is targeted has been described, but the present invention is not limited to this. It may be any data that has been imaged, and may be, for example, image data of a patent drawing or image data such as a photograph.
(7) In the present embodiment, it has been described that the metadata of the image data has a museum ID of a museum or the like that holds the image data, but the present invention is not limited to this. When the image data is a painting, for example, the painter of the painting, the age of the motif of the painting, and the like may be included in the metadata. Further, when the image data is a photograph, for example, the shooting date, the shooting location, and the like may be included in the metadata.
Then, the similarity may be calculated using the metadata. For example, in the case of paintings, if they belong to the same painter, the degree of similarity may be increased.

(8) In the present embodiment, when displaying the related image, the similarity is expressed by the thickness of the line as an example, but the present invention is not limited to this. For example, the similarity may be expressed by the distance between each image. At that time, the distance between each image may be expressed by the size of the image. As a specific example, when displaying a related image from a map, the one described in this embodiment is output, and when one image is selected from the displayed related images by the user, the selected image is displayed. The similarity may be expressed in the depth direction and output at the center. By doing so, the degree of similarity with the selected image can be shown in an easy-to-understand manner.

1 Image map creation device 3 Image DB
4 Display device 10, 40 Control unit 12 Word extraction unit 14 Classification word generation unit 15 Map creation unit 17 Related image generation unit 20, 50 Storage unit 21a Map creation program 22 Map storage unit 41 Map output unit 42 Element reception unit 43 Image output Part 51a Image display program 57 Touch panel display 80 Map 90 Related images 100 Image map output system

Claims (13)

A word extraction method that analyzes images to extract multiple words,
A classification word generation means for generating a classification word by clustering a plurality of words extracted by the word extraction means,
A map creation means that creates a map by counting the classification words generated by the classification word generation means, and
An image map creation device equipped with. In the image map creating apparatus according to claim 1,
A classification means for dividing a plurality of words extracted by the word extraction means into a plurality of groups is provided.
The classification word generation means clusters the words of each group divided by the classification means to generate a classification word.
The map creation means creates the map classified by each classification word of at least two groups out of the plurality of groups divided by the classification means.
An image map creation device characterized by. In the image map creating apparatus according to claim 1 or 2.
The classification word generation means generates a word corresponding to a superordinate concept as the classification word from a plurality of words extracted by the word extraction means.
An image map creation device characterized by. In the image map creating apparatus according to any one of claims 1 to 3.
The classification word generation means generates the classification word based on the appearance frequency of a plurality of words extracted by the word extraction means.
An image map creation device characterized by. In the image map creating apparatus according to any one of claims 1 to 3.
An image ratio calculation means for analyzing an image, extracting a plurality of feature images representing the features of the image, and calculating the ratio of the extracted feature image to the region of the image is provided.
The word extraction means extracts a word representing a feature of the feature image and obtains a word.
The classification word generation means generates the classification word from the words extracted by the word extraction means according to the ratio calculated by the image ratio calculation means.
An image map creation device characterized by. In the image map creating apparatus according to any one of claims 1 to 5.
The word extraction means generates a descriptive text by referring to the words associated with each feature of the image, and extracts a plurality of words from the generated descriptive text.
An image map creation device characterized by. In the image map creating apparatus according to any one of claims 1 to 6.
The classification word generation means generates a classification word by clustering a plurality of words extracted by the word extraction means from each of a plurality of images.
An image map creation device characterized by. A display device communicatively connected to the image map creating device according to any one of claims 1 to 7.
A map output means for outputting the map created by the map creation means, and a map output means.
An element receiving means that is shown in the map output by the map output means and accepts the selection of the element associated with the classification word.
An image output means that outputs an image corresponding to the element received by the element reception means, and an image output means.
To prepare
A display device characterized by. In the display device according to claim 8,
When there are a plurality of images corresponding to the elements received by the element receiving means, the image output means arranges each image according to the similarity of the plurality of images, and uses the plurality of images as related images. To output,
A display device characterized by. In the display device according to claim 9,
The similarity is calculated based on at least one of the feature amount of each image and the word corresponding to each image.
A display device characterized by. In the display device according to claim 9 or 10.
The related images represent the similarity by the length or thickness of the lines between the images or the distance in the depth direction from the reference image.
A display device characterized by. A program for operating a computer as the image map creating device according to any one of claims 1 to 7. A program for operating a computer as a display device according to any one of claims 8 to 11.

Images