JP7528715B2 - Font setting device, font setting method, and computer program - Google Patents

Description

The present invention relates to a typeface setting device, a typeface setting method, and a computer program.

Technology for superimposing text onto images has been known in the past and is widely used, for example, as advertising images with catchy slogans and images with text used in presentation software (see, for example, Patent Document 1).

JP 2006-172118 A

There are many different types of text fonts, and the text font and the image may not match, causing the user to feel uncomfortable. However, the technology described in Patent Document 1 is a technology for overlaying text in a position that does not damage the image, and is not a technology for overlaying a font that does not look unnatural.

The present invention has been made in consideration of the above circumstances, and aims to provide a font setting device, a font setting method, and a computer program for suppressing incongruity between the font of text and an image when the text is superimposed on the image.

A typeface setting device according to one embodiment of the present invention includes a material acquisition unit that acquires a material image, material text to be superimposed on the material image, and information on an area in which the material text is superimposed on the material image, an image selection unit that selects a similar image that is similar to the material image or a combination of the material image and the material text from a plurality of teacher images on which text is superimposed, a typeface identification unit that identifies a typeface to be applied to the material text based on the typeface of the text superimposed on the selected similar image, and a superimposition unit that superimposes the material text on the area of the material image using the identified typeface.

According to the present application, when text is superimposed on an image, it is possible to reduce the sense of discomfort felt by the user by suppressing the incongruity between the typeface of the text and the image.

FIG. 2 is an explanatory diagram outlining a typeface setting method according to an embodiment. FIG. 1 is a block diagram showing a configuration of an information processing device. FIG. 13 is a conceptual diagram illustrating an example of a font table. 1A and 1B are schematic diagrams showing examples of material images and material texts. FIG. 2 is a schematic diagram showing an example of a teacher image. FIG. 2 is a schematic diagram showing a configuration of a learning model used for object detection. FIG. 13 is a schematic diagram showing an example of object detection. FIG. 13 is a diagram showing a calculation result of the degree of difference. FIG. 1 is an explanatory diagram illustrating a method of recognizing text. FIG. 11 is an explanatory diagram for explaining a method of estimating a typeface. FIG. 13 is a schematic diagram showing an image after a font style has been set. 11 is a flowchart illustrating a procedure of a process executed by an information processing device. 11 is a flowchart illustrating a procedure for selecting a similar image. 13 is a flowchart illustrating a typeface estimation procedure. FIG. 11 is an explanatory diagram illustrating a method for selecting a similar image in the second embodiment. 13 is a flowchart illustrating a procedure for selecting a similar image in the second embodiment.

Hereinafter, the present invention will be described in detail with reference to the drawings showing embodiments thereof.
(Embodiment 1)
FIG. 1 is an explanatory diagram for explaining an outline of a font setting method in an embodiment. In the embodiment, a font setting method when arranging text superimposed on an image will be explained. The information processing device 1 shown in FIG. 1 is a dedicated or general-purpose computer used for, for example, DTP (Desktop Publishing). The information processing device 1 includes software (hereinafter referred to as layout software) for creating magazine pages, flyers, etc. on a computer. The layout software is software that can appropriately arrange images and text. An example of the layout software is page layout software for typesetting. Alternatively, graphic software, word processing software, presentation software, etc. can be used.

DTP workers use layout software to place text on an image and create data for magazine pages, flyers, and the like. In the following, an image before text is placed on it is called a material image, and text before it is placed on an image is called material text. Any image can be used as a material image. For example, the material image can be a photograph (image) taken with a digital camera, a two-dimensional or three-dimensional image created using graphics software, or an image obtained via a communications network N. Material text is character string data entered using layout software, and is written in character code inside the information processing device 1.

The worker uses layout software to place the source text superimposed on the source image. At this time, the information processing device 1 searches for an image similar to the source image (hereinafter referred to as a similar image) from among multiple teacher images with superimposed text present on the communication network N, and sets the typeface of the source text to be placed on the source image by referring to the typeface information of the text contained in the similar image. In this embodiment, the information processing device 1 automatically sets the typeface to be applied to the source text, so the workload can be reduced compared to when the worker manually selects the typeface. Also, in this embodiment, the typeface of the source text is set by referring to an image created by another worker, so it is possible to reduce the sense of incongruity caused by the content of the source image and the typeface of the source text not matching.

Figure 2 is a block diagram showing the configuration of the information processing device 1. The information processing device 1 includes a control unit 11, a storage unit 12, a communication unit 13, an input unit 14, an operation unit 15, and a display unit 16.

The control unit 11 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), etc. The ROM included in the control unit 11 stores control programs and the like for controlling the operation of each of the above hardware components. The CPU in the control unit 11 executes the control programs stored in the ROM and the various programs stored in the memory unit 12, and controls the operation of each of the above hardware components, causing the entire device to function as a typeface setting device according to the present invention. The RAM included in the control unit 11 stores data that is temporarily used while the various programs are being executed.

The control unit 11 is not limited to the above configuration, and may be one or more processing circuits or control circuits including a single-core CPU, a multi-core CPU, a GPU (Graphics Processing Unit), a microcomputer, a volatile or non-volatile memory, etc. The control unit 11 may also have functions such as a clock that outputs date and time information, a timer that measures the elapsed time from when an instruction to start measurement is given to when an instruction to end measurement is given, and a counter that counts numbers.

The storage unit 12 includes storage devices such as a hard disk drive (HDD) and a solid state drive (SSD). The storage unit 12 stores various programs to be executed by the control unit 11. The programs stored in the storage unit 12 include a font setting program PG for selecting an image similar to the material image and setting the font of the material text to be superimposed on the material image based on the font information of the text contained in the selected similar image. Note that the font setting program PG does not need to be a single computer program, but may be a program group consisting of multiple computer programs. The font setting program PG may also be a macro or plug-in used in an appropriate application program.

The computer program stored in the memory unit 12 may be provided by a recording medium M on which the computer program is recorded in a readable manner. The recording medium M is, for example, a portable memory such as a Secure Digital (SD) card, a micro SD card, or a Compact Flash (registered trademark). In this case, the control unit 11 reads the program from the recording medium M using a reading device (not shown) and stores the read program in the memory unit 12. The program stored in the memory unit 12 may also be provided by communication. In this case, the control unit 11 obtains the program through the communication unit 13 and stores the obtained program in the memory unit 12.

The storage unit 12 also includes a font table TB that stores information about fonts included in the information processing device 1. FIG. 3 is a conceptual diagram showing an example of the font table TB. The fonts in this embodiment are digital fonts, and represent typeface data that can be used for display and printing. A typeface is a style of characters that is designed in a unified manner based on aesthetic sensibility. Typefaces include Japanese typefaces such as Ming, Gothic, and cursive, and Western typefaces such as serif and sans serif. In this embodiment, for the sake of simplicity, it is assumed that only three fonts, "Font A", a Ming font, "Font B", a Gothic font, and "Font C", a cursive font, are installed in the information processing device 1.

The communication unit 13 has a communication interface for connecting to a communication network N such as the Internet. The communication interface provided in the communication unit 13 may be a wired interface or a wireless interface. The communication unit 13 transmits various information to be notified to the outside, and receives various information transmitted from the outside to the device itself.

The input unit 14 has an interface for connecting an imaging device such as a digital camera or a scanner. The control unit 11 can acquire an image generated by an imaging device as a material image through the input unit 14. The control unit 11 stores the material image acquired through the input unit 14 in the storage unit 12.

The operation unit 15 has an interface for connecting input devices such as a keyboard and a touch panel. The control unit 11 accepts various operations and settings by the operator through the operation unit 15. The control unit 11 can also accept source text through the operation unit 15.

The display unit 16 is equipped with a display device such as a liquid crystal display panel or an organic EL (Electro Luminescence) display panel, and displays information and data to be notified to the worker based on a control signal output from the control unit 11.

In addition, the information processing device 1 does not have to be a single computer, but may be a computer system composed of multiple computers and peripheral devices. In addition, the information processing device 1 may be a virtual machine that is virtually constructed by software.

The process executed by the information processing device 1 will now be described in detail.
(1) Obtaining material images and material text The worker starts the layout software on the information processing device 1, operates the layout software to display the desired material image on the display unit 16, and uses the operation unit 15 to input material text to be placed on the material image. Fig. 4 is a schematic diagram showing an example of a material image and material text. The material image 110 shown as an example in Fig. 4 is a photograph (image) captured with an illuminated temple as the subject. The material image 110 is a digital image, and is composed of a plurality of pixels, each having a pixel value for each of the RGB colors. The material image 110 is imported into the layout software and displayed on the display unit 16 of the information processing device 1.

When placing text in the material image 110, a text placement area 110A indicating the position of the text to be placed is set by the operator. Text entered using the operation unit 15 is displayed in the text placement area 110A. The material text 120 shown as an example in FIG. 4 is composed of the character string "The main hall at night lit by warm lights." Immediately after input, a predetermined font (typeface) or a font (typeface) selected by the operator is set in the material text 120.

(2) Search for Similar Image Candidates When the material image 110 and the material text 120 are given, the information processing device 1 searches for an image similar to the material image 110 (similar image) from a plurality of teacher images on which text is superimposed and which exist on the communication network N. The information processing device 1 may search for a similar image triggered by the provision of the material image 110 and the material text 120, or may search for a similar image triggered by a search instruction given by an operator. In this embodiment, the similarity between the texts as well as the similarity between the images is taken into consideration, and ultimately, a similar image closest to the material image 110 is selected.

In order to take into account similarities between texts, the information processing device 1 performs morphological analysis on the given material text 120 and extracts words contained in the material text 120. For example, five words, "warm," "light," "illuminated," "night," and "main hall," are extracted from the material text 120, "The main hall at night lit by warm lights." The information processing device 1 searches for images that contain at least one of these extracted words via the communication network N.

Figure 5 is a schematic diagram showing an example of a teacher image. Teacher image 210 shown in Figure 5A is an image of a suspension bridge captured with a sunset in the background, and includes the text "Sunset over a suspension bridge, fantastic lights." This text includes the word "lights," which is the same as the word extracted from material text 120, so teacher image 210 shown in Figure 5A is searched for as a candidate for a similar image.

The teacher image 220 shown in FIG. 5B is an image captured of a deciduous tree that is beginning to turn red, and includes the text "Autumn leaves in Japan, a warm scene." Because this text includes the word "warm," which is the same as the word extracted from the material text 120, the teacher image 220 shown in FIG. 5B is searched for as a candidate for a similar image.

The teacher image 230 shown in FIG. 5C is an image captured with a temple as the subject, and includes the text "Main hall open to the public." Because this text includes the word "Main hall," which is the same as the word extracted from the material text 120, the teacher image 230 shown in FIG. 5C is searched for as a candidate for a similar image.

The teacher image 240 shown in FIG. 5D is an image of a landscape with a floating balloon, and includes the text "We are looking for people interested in air travel." Since this text does not include the same words as those extracted from the material text 120, the teacher image 240 shown in FIG. 5D is excluded from the candidates for similar images.

The information processing device 1 according to the present embodiment is configured to search for teacher images that contain at least one of the words extracted from the material text 120 as candidates for similar images, but teacher images that contain not only the same word but also synonyms or similar words may be searched for as candidates for similar images. For example, the information processing device 1 may search for teacher images that contain words such as "warm," "tepid," "mild," "clear," and "warm" as candidates for similar images for the word "warm" extracted from the material text 120. Furthermore, when a sufficient number of candidates for similar images (e.g., one or more) are not obtained as a result of searching for teacher images that contain the same word, the information processing device 1 may expand the search range to include synonyms or similar words.

Even if the teacher image acquired via the communication network N contains text, the text is often embedded as an image (i.e., expressed as a bitmap integrally with the image). In this case, the information processing device 1 cannot recognize the text in the teacher image as a character string (character code), and cannot determine the similarity with the material text 120. Therefore, the information processing device 1 performs OCR (Optical Character Recognition) on the teacher image acquired from the communication network N to recognize the text contained as an image as a character string, and extracts words by performing morphological analysis on the recognized character string. The information processing device 1 can determine the similarity between the texts by comparing the words extracted from the material image 110 with the words extracted from the image on the communication network N.

(3) Selection of Similar Images Next, the information processing device 1 selects the image most similar to the material image 110 from among the teacher images selected as candidates for similar images as the final similar image. Specifically, the information processing device 1 detects one or more objects (also called objects) contained in each of the material image 110 and the candidate similar image, and determines the similarity between the two by combining the dissimilarity of the entire image and the dissimilarity between the objects. The information processing device 1 selects the image with the lowest dissimilarity (i.e., the image with the highest similarity) as the similar image to the material image 110.

Figure 6 is a schematic diagram showing the configuration of a learning model MD1 used for object detection. The information processing device 1 detects one or more objects contained in each of the material image 110 and candidate similar images. Here, detecting an object means detecting a rectangular area (bounding box) that surrounds an object in an image and classifying the object contained within that rectangular area. The information processing device 1 detects objects using the learning model MD1 based on deep learning.

The learning model MD1 is a learning model based on deep learning such as R-CNN (Region-based Convolutional Neural Network), Fast R-CNN, SSD (Single Shot Detector), Mask R-CNN, YOLO (You Only Look Once), and RetinaNet. The learning model MD1 includes an input layer, an intermediate layer, and an output layer. An image to be processed is input to the input layer. The intermediate layer includes a convolution layer, a pooling layer, a fully connected layer, and the like, and performs calculations for selecting candidates for object-like regions included in the input image and calculations for identifying objects based on features extracted from the input image. The output layer outputs object detection results based on the calculation results by the intermediate layer. The learning model MD1 is configured to perform calculations in the intermediate layer according to the image input to the input layer, and to output, for example, information on class names, prediction reliability of classes, and bounding boxes from the output layer.

The learning model MD1 is generated by providing a large number of images containing known objects, the class names of the objects contained in each image, and the coordinate values of the bounding box surrounding the objects as training data, and performing machine learning using a predetermined algorithm. The learning model MD1 may be generated inside the information processing device 1, or may be generated in an external server. The generated learning model MD1 is stored in the storage unit 12 of the information processing device 1. In this case, the control unit 11 of the information processing device 1 inputs the image to be processed into the learning model MD1 and executes a calculation using the learning model MD1 to obtain the object detection result. Alternatively, the learning model MD1 may be stored in an external server. In this case, the information processing device 1 transmits the image to be processed to the external server, and obtains the calculation result (object detection result) by the external server through communication.

In this embodiment, the object is detected using a learning model MD1 based on deep learning, but the method for detecting the object is not limited to a specific method. For example, the object may be detected using a classical method such as template matching.

Figure 7 is a schematic diagram showing an example of object detection. Figure 7A shows an example of object detection for material image 110. Material image 110 includes a main hall (temple), a five-story pagoda (temple), and two trees as objects. By using the learning model MD1 described above, the information processing device 1 detects bounding boxes S1 to S4 that surround these objects, and classifies the objects within the bounding boxes S1 to S4. For example, bounding box S1 indicates that the class of the object within the area is identified as "temple", and the prediction reliability is "98%". The other bounding boxes S2 to S4 are as shown in Figure 7A.

Figure 7B shows an example of object detection for image 230 (see Figure 5C), which is a candidate for a similar image. Image 230 includes a main hall (temple) and a five-story pagoda (temple) as objects. By using the learning model MD1 described above, the information processing device 1 detects bounding boxes T1 to T2 that surround these objects and performs class identification of each of the objects within the bounding boxes T1 to T2. For example, bounding box T1 indicates that the class of the object within the region is identified as "temple" and that the prediction reliability is "100%". The other bounding box T2 is also as shown in Figure 7B.

Next, the information processing device 1 calculates the degree of dissimilarity between the material image 110 and the candidate similar image. In this embodiment, the information processing device 1 calculates the degree of dissimilarity between the material image 110 and the candidate similar image by combining the degree of dissimilarity of the entire image and the degree of dissimilarity between the objects.

The dissimilarity d(I, J) between any two images (image I and image J) that have the same number of pixels in both the vertical and horizontal directions is calculated, for example, using the following formula:

Here, I _yx is the pixel value of the coordinates (x, y) in image I, and J _yx is the pixel value of the coordinates (x, y) in image J. In other words, Equation 1 represents the sum of the absolute values of the differences between all pixel values of the two images.

When applied between the material image 110 shown in FIG. 7A and the candidate similar image (image 230) shown in FIG. 7B, the dissimilarity between the material image 110 and image 230 is calculated based on equation 1 as follows:

Here, d(S1, T1) represents the degree of difference between the image in the bounding box S1 detected in the material image 110 and the image in the bounding box T1 detected in the image 230. Since the bounding boxes are not of a fixed size, the sizes of the bounding boxes S1 and T1 are not necessarily the same. For this reason, as a pre-processing before calculating the degree of difference, the information processing device 1 resizes each bounding box to a predetermined size so that the vertical and horizontal sizes are the same. The same is true for d(S1, T2), d(S2, T1), and d(S2, T2). In other words, the first to fourth terms on the right-hand side of equation 2 represent the degree of difference between the object included in the material image 110 and the object included in the image 230.

The fifth term on the right-hand side of Equation 2, d(S, T), represents the degree of dissimilarity between material image 110 and image 230. Material image 110 and image 230 are not necessarily the same size, so as a pre-processing step before calculating the degree of dissimilarity, the images are resized to a predetermined size so that their horizontal and vertical sizes are the same. The first through fourth terms on the right-hand side of Equation 2 represent the degree of dissimilarity between objects, while the fifth term represents the degree of dissimilarity of the entire image.

p(S3) and p(S4) in the sixth and seventh terms on the right-hand side of equation 2 represent penalties. Since image 230 does not contain the two tree objects contained in material image 110, a penalty is added that increases the degree of dissimilarity. The penalty may be a fixed value (e.g., 100) or a value that varies depending on the size of bounding boxes S3 and S4. A penalty may also be added if image 230 contains an object that does not exist in material image 110.

An example of calculating the degree of dissimilarity between material image 110 and image 230 has been described with reference to FIG. 7. In a similar manner, information processing device 1 can calculate the degree of dissimilarity between material image 110 and other candidate similar images.

Figure 8 shows the results of calculating the dissimilarity. Figure 8 shows the results of calculating the dissimilarity between material image 110 and images 210, 220, and 230 selected as candidates for similar images. Since the dissimilarity is calculated based on the dissimilarity of the entire image and the dissimilarity between objects, the dissimilarity of images 210 and 220, which have a different overall impression from material image 110 and have objects that differ from material image 110, is relatively high. On the other hand, the dissimilarity of image 230, which has a relatively similar overall impression to material image 110 and has the same type of object as material image 110, is calculated to be relatively low.

The information processing device 1 selects the image with the lowest degree of dissimilarity (i.e., the image with the highest degree of similarity) from among the images selected as candidates for similar images as the similar image of the material image 110. In the example of FIG. 8, image 230 is selected as the similar image.

The information processing device 1 according to this embodiment is configured to calculate the dissimilarity between images and between objects, and select similar images based on the calculated dissimilarity, but it may also be configured to select similar images using a learning model that is trained to input two images to be compared and output a value related to the dissimilarity (or dissimilarity).

(4) Setting of Font The information processing device 1 estimates the font of the text included in the similar image in order to determine the font to be applied to the material text 120. At this time, the information processing device 1 cuts out a text area from the similar image and recognizes each character included in the cut-out text area.

Figure 9 is an explanatory diagram explaining a method of recognizing text. The image shown as an example in Figure 9 is image 230, which has been selected as a similar image to material image 110. Image 230 has the text "Main hall open to the public" superimposed on it. Since this text is embedded as an image (i.e., it is represented as a bitmap integral with the image), the information processing device 1 cannot recognize each character that makes up the text as a character code that can be identified on a computer.

The information processing device 1 according to this embodiment extracts a text area including text from the image 230, and recognizes each character included in the extracted text area. In this embodiment, a text area represents a rectangular area surrounding each character. Extraction of a text area and recognition of each character included in a text area can be achieved by using existing OCR technology. In the example of Figure 9, text areas surrounding each of the six characters "本", "堂", "一", "普通", "公", and "開" are extracted, and the characters (character codes) included in each text area are recognized.

In addition, the information processing device 1 performs image segmentation to separate a text region including each character into a character region and a background region. In this embodiment, the character region represents the region inside the outline that defines the shape of each character, and the background region represents the region outside the outline. The information processing device 1 performs image segmentation using a learning model (not shown) constructed by a neural network such as SegNet, FCN (Fully Convolutional Network), U-Net (U-Shaped Network), or PSPNet (Pyramid Scene Parsing Network). The learning model for image segmentation may be stored in the storage unit 12 of the information processing device 1, or may be stored in an external server accessible via the communication network N.

Furthermore, the information processing device 1 calculates the average value of pixel values in the character region and the average value of pixel values in the background region for each text region. In this embodiment, due to space limitations, black characters are drawn against a white background, so the average value of pixel values in the character region is (0,0,0), which indicates black, and the average value of pixel values in the background region is (255,255,255), which indicates white.

Next, the information processing device 1 estimates the typeface of the text contained in the similar image. FIG. 10 is an explanatory diagram explaining a typeface estimation method. The recognition result of the text contained in the similar image includes the recognition result (character code) of each character constituting the text, color information of the character area, and color information of the background area. The information processing device 1 creates a typeface sample that reflects the recognition result of the text contained in the similar image using fonts installed in the device. In this embodiment, since fonts for Ming, Gothic, and cursive are installed, the information processing device 1 creates typeface samples for Ming, Gothic, and cursive as bitmap data that include the same character string as the text of the similar image and reflect the color of the character area and the color of the background area.

The information processing device 1 calculates the degree of difference between the group of images of the text included in the similar images and the group of images of the created font sample, and estimates the font of the text included in the similar images based on the calculated degree of difference. The information processing device 1 can calculate the degree of difference d(I, J) based on the following formula, for example.

Here, I ⁿ _yx is the pixel value of the pixel at coordinates (x, y) in the image of the text contained in the nth similar image, and J ⁿ _yx is the pixel value of the pixel at coordinates (x, y) in the image of the nth font sample. n represents the number of characters. In other words, Equation 3 shows the result of adding up the sum of the absolute values of the differences in all pixel values of the two images for the number of characters.

If the calculated difference between the image and the cursive typeface sample is the lowest (i.e., the similarity is the highest), the information processing device 1 estimates that the typeface of the text contained in the similar image is "cursive typeface."

The information processing device 1 sets the typeface of the text estimated from the similar image as the typeface of the material text 120 to be superimposed on the material image 110. The information processing device 1 applies the set typeface to the material text 120 and places it in the text placement area 110A of the material image 110 to obtain the final image. Figure 11 is a schematic diagram showing the image after the typeface has been set.

The procedure of the process executed by the information processing device 1 will be described below.
12 is a flowchart illustrating the procedure of the process executed by the information processing device 1. The control unit 11 of the information processing device 1 reads out the typeface setting program PG from the storage unit 12 and executes it to execute the following process.

The control unit 11 accepts the input of a material image and material text (step S101). The material image is input by importing it through layout software launched on the information processing device 1. The material text is input by accepting an input operation by the worker at the operation unit 15. In addition, when accepting the input of the material text, a text placement area is set that determines the placement position of the material text on the material image.

The control unit 11 then acquires candidates for similar images similar to the material image via the communication network N (step S102). At this time, the control unit 11 performs an image search using the material image as a search key, and acquires, from among the searched images, images that contain the same words as those contained in the material text as candidates for similar images. A plurality of candidates for similar images may be acquired. In step S102, images that do not contain any of the words that make up the material text are excluded from the candidates for similar images. As an example, the control unit 11 is configured to perform an image search using the material image as a search key, but the material image on which the material text is superimposed may be used as the search key, or words contained in the material text may be used as the search key.

Next, the control unit 11 calculates the degree of dissimilarity between the material image and the candidate similar images, and selects one similar image based on the calculated degree of dissimilarity (step S103). The procedure for selecting a similar image in step S103 will be described in detail with reference to the flowchart shown in FIG. 13.

Next, the control unit 11 extracts a text region from the selected similar image and estimates the typeface of the text contained in the extracted text region (step S104). The typeface estimation procedure in step S104 will be described in detail with reference to the flowchart shown in FIG. 14.

Next, the control unit 11 applies the font estimated in step S104 to the material text and displays it on the display unit 16 (step S105). At this time, the material text after the font application is superimposed on the text placement area on the material image that was set in advance.

Figure 13 is a flowchart explaining the procedure for selecting similar images. The control unit 11 detects objects contained in the material image (step S121). Classical methods such as template matching and learning model MD1 constructed using a neural network such as R-CNN are used to detect the objects. As the object detection result, the control unit 11 obtains information on the bounding box surrounding the object, the class name of the object present within the bounding box, and the predicted reliability of the class.

The control unit 11 detects objects included in the candidates for similar images (step S122). The method of detecting the objects is the same as in step S121. In this embodiment, for convenience, the procedure is such that objects included in the candidate similar images are detected after objects included in the material image are detected, but the order of these procedures is not limited to the flowchart shown in FIG. 13, and they may be executed simultaneously in parallel.

Next, the control unit 11 calculates the dissimilarity between the material image and the candidate similar images, taking into account the detected objects (step S123). The dissimilarity is given, for example, as the sum of the absolute values of the differences for each pixel value. The control unit 11 calculates the dissimilarity between the material image and the candidate similar images by adding up the dissimilarity calculated for the entire image and the dissimilarity calculated for each object. If an object included in the material image is not included in the candidate similar images, or if an object not included in the material image is included in the candidate similar images, a penalty (positive value) may be added to the dissimilarity.

Next, the control unit 11 selects a similar image based on the calculated dissimilarity (step S124). That is, the control unit 11 may select, from among the candidates for similar images, the candidate with the lowest calculated dissimilarity as the similar image.

Figure 14 is a flowchart explaining the typeface estimation procedure. The control unit 11 extracts a text region from a similar image (step S141) and performs character recognition (step S142). Existing OCR technology is used for the extraction of the text region and character recognition. The control unit 11 also separates the text region into a character region and a background region by image segmentation, and calculates the average pixel value in each region (step S143).

Next, the control unit 11 creates a typeface sample using fonts installed in the information processing device 1 based on the results of the character recognition and the calculated average pixel values of the character area and background area (step S144). The typeface sample is created as an image in bitmap format.

Next, the control unit 11 calculates the degree of dissimilarity between the created font sample and the image of the text region extracted from the similar image (step S145). In this embodiment, the font sample is created as a bitmap image, so the control unit 11 can calculate the degree of dissimilarity based on the difference in pixel values between the images. In other words, the control unit 11 can calculate the degree of dissimilarity as the sum of the absolute values of the differences for each pixel value.

Next, the control unit 11 estimates the typeface of the text included in the similar image based on the calculated degree of difference (step S146). That is, the control unit 11 selects the typeface sample with the lowest calculated degree of difference from among the created typeface samples, and estimates the typeface indicated by that typeface sample as the typeface of the text included in the similar image.

As described above, in this embodiment, by providing a material image and material text, the typeface of the material text to be placed on the material image is automatically set. The typeface of the material text is set by referring to an existing image that can be obtained via the communication network N, so that the material text can be made to blend in with the material image, and an image that looks less unnatural can be provided to the worker.

In this embodiment, we have described an application example in which layout software is used to place text in an image and create data for magazine pages, flyers, etc., but the software can also be used widely, for example, as advertising images with catchy slogans, images with text for use in presentation software, images with comments for use in social networks, etc.

Although the present embodiment is configured to execute each process in the information processing device 1, each process may be executed in an external server accessible from the information processing device 1. In this case, the information processing device 1 accepts input of the material image and material text, and uploads the accepted material image and material text data to the external server. The external server executes each of the above-mentioned processes based on the received material image and material text data, and sets the typeface of the material text to be placed in the material image. The external server simply returns the material image with the material text after the typeface setting to the information processing device 1.

In addition, in this embodiment, a similar image that is similar to the combination of the material image and the material text is selected by taking into consideration the similarity between the images and the similarity between the texts, but a similar image that is similar only to the material image may be selected without taking into consideration the similarity between the texts. In the latter case, the information processing device 1 may select a similar image that is most similar to the material image based on the degree of dissimilarity of the entire image and the degree of dissimilarity between the objects.

Furthermore, in this embodiment, candidates for similar images are searched for via a communication network, but it is also possible to configure the system so that teacher images that may be used as similar images are collected in advance and registered in a database. In this case, the information processing device 1 can access the database to obtain candidates for similar images.

(Embodiment 2)
In the first embodiment, the criteria for selecting similar images are the identity of words contained in the images and the similarity of the entire image and objects contained in the image. In the second embodiment, a configuration will be described in which similar images are selected taking into consideration the size of text in the image in addition to these selection criteria.

Figure 15 is an explanatory diagram explaining a method for selecting similar images in embodiment 2. The image 110 shown in Figure 15 is the same as the material image 110 explained in embodiment 1. When material text 120 is placed in material image 110, text placement area 110A is set. The proportion (area ratio) of text placement area 110A in material image 110 is set as S0.

Images 231 to 233 are a group of candidate similar images obtained via communication network N. Images 231 to 233 have the same image and text content, but differ only in the size of the text that is placed. The proportion of text area 231A in image 231 is S1, the proportion of text area 232A in image 232 is S2, and the proportion of text area 233A in image 233 is S3.

The information processing device 1 according to the second embodiment compares the proportion of the text placement area 110A in the material image 110 with the proportion of the text areas 231A to 233A in each of the images 231 to 233, and selects the candidate with the closest proportion as the similar image. In the example of FIG. 15, the proportion S2 of the image 232 is closest to the proportion S0 of the material image 110, so the information processing device 1 selects the image 232 as the similar image to the material image 110.

Figure 16 is a flowchart explaining the procedure for selecting similar images in embodiment 2. The control unit 11 detects objects contained in the material image and the candidate similar images using the same procedure as in embodiment 1, and calculates the degree of dissimilarity between the material image and the candidate similar images, taking into account the detected objects (steps S201 to S203).

Next, the control unit 11 selects similar image candidates whose degree of dissimilarity with the material image is relatively small (step S204). For example, the control unit 11 selects a predetermined number of similar image candidates in order from the one with the smallest degree of dissimilarity. The control unit 11 may also compare the calculated degree of dissimilarity with a preset threshold value, and select similar image candidates whose degree of dissimilarity is smaller than the threshold value.

Next, the control unit 11 calculates the proportion of the text placement area in the material image (step S205). The control unit 11 calculates the ratio between the size (total number of pixels) of the material image and the size (total number of pixels) of the text placement area set by the worker.

Next, the control unit 11 extracts a text area from each of the similar image candidates selected in step S204 (step S206), and calculates the proportion of the text area in each of the similar image candidates (step S207). The method of extracting the text area is the same as in embodiment 1, but it is not necessary to extract a text area for each character, and it is sufficient to extract an area that includes the entire character string. In addition, the control unit 11 calculates the ratio between the size (total number of pixels) of the similar image candidate and the size (total number of pixels) of the extracted text area.

Next, the control unit 11 selects a similar image based on the ratios calculated in steps S205 and S207 (step S208). That is, the control unit 11 compares the ratio of the text placement area in the material image with the ratio of the text area in each of the similar image candidates, and selects the candidate that is closest to the two as the similar image.

The process after similar images are selected is the same as in the embodiment. For example, in the example of FIG. 15, a font (cursive) estimated from the text included in the text area 232A is applied to the source text.

As described above, in the second embodiment, similar images are selected taking into consideration the proportion of the text area in the image, so that similar images that give a relatively similar impression to the text in the image can be selected, and the typeface of the source text can be set based on such similar images.

The embodiments disclosed herein are to be considered in all respects as illustrative and not restrictive. The scope of the present invention is indicated by the claims, not by the above meaning, and is intended to include all modifications within the meaning and scope of the claims.

Reference Signs List 1 Information processing device 11 Control unit 12 Storage unit 13 Communication unit 14 Input unit 15 Operation unit 16 Display unit PG Font setting program TB Font table M Recording medium

Claims (7)

a material acquisition unit that acquires a material image, material text to be superimposed on the material image, and information on an area where the material text is superimposed on the material image;
an image selection unit that selects, from a plurality of teacher images having text superimposed thereon, an image similar to the material image or a combination of the material image and the material text;
a typeface specification unit that specifies a typeface to be applied to the source text based on a typeface of the text superimposed on the selected similar image;
and a superimposing unit that superimposes the material text on the region of the material image in the specified typeface. A division unit that divides the material text into a plurality of words,
The typeface setting device according to claim 1 , wherein the image selection unit extracts candidate images on which one or more divided words are superimposed from the plurality of teacher images, and selects the similar image from the extracted candidate images. a detection unit that detects one or more in-image objects of the candidate image and one or more in-image objects of the material image;
The typeface setting device according to claim 2 , wherein the image selection unit selects the similar image from the candidate images based on a relationship between a pixel value of each intra-image object of the candidate images and a pixel value of each intra-image object of the element image. A storage unit for storing a plurality of font sample groups,
4. The typeface setting device according to claim 2, wherein the typeface specification unit specifies a typeface that is most similar to a typeface of the text superimposed on the similar image selected by the image selection unit, based on the group of the plurality of typeface samples. a calculation unit that calculates a ratio of the region to the material image,
The typeface setting device according to claim 1 , wherein the image selection unit selects the similar image from the plurality of teacher images based on the calculated ratio. Acquiring a material image, material text to be superimposed on the material image, and information on an area in which the material text is superimposed on the material image;
selecting a similar image that is similar to the material image or a combination of the material image and the material text from a plurality of teacher images having text superimposed thereon;
Identifying a typeface to be applied to the source text based on the typeface of the text superimposed on the selected similar image;
The font setting method causes a computer to execute a process of superimposing the material text on the region of the material image using the specified font. Acquiring a material image, material text to be superimposed on the material image, and information on an area in which the material text is superimposed on the material image;
selecting a similar image that is similar to the material image or a combination of the material image and the material text from a plurality of teacher images having text superimposed thereon;
Identifying a typeface to be applied to the source text based on the typeface of the text superimposed on the selected similar image;
A computer program for causing a computer to execute a process of superimposing the material text in the specified font on the region of the material image.

Images