JP5664346B2 - Image processing apparatus, information providing system, and program - Google Patents

Description

  The present invention relates to an image processing apparatus, an information providing system, and a program.

  As a system for searching for an object, for example, in Patent Document 1, a database is obtained by extracting color feature values related to image colors in an image block included in an object and text feature values related to a character string of text in a text block. It is accumulated in 118. Then, when performing a search, a color feature amount and a text feature amount are extracted from the captured image, and each extracted feature amount is compared with each feature amount stored in the database 118, thereby resembling the extracted image. Techniques have been proposed for creating a list of active objects.

JP-A-2005-352783 JP 2002-007432 A

  In the present invention, when embedding a boundary that includes a number of feature points necessary for identifying an image element associated with information provided to a user, the center of the boundary is compared with the case where the center of the boundary is not provided. Then, it aims at arrange | positioning near the position specified by the positional information which specifies the position of the said image element.

  An image processing apparatus according to the present invention includes a unit that acquires an image including an image element associated with information provided to a user, a unit that acquires position information that specifies a position of the image element on the image, and the position Means for acquiring an embedding range of an embedding element whose position to be embedded on the image is specified by information; means for obtaining a distribution of feature points on the image; and a predetermined number necessary for identifying the image element Embedding means for embedding a boundary having a predetermined shape including the above feature points and the embedding element in the image so that the entire embedding element is included in the boundary on the image, and the boundary And at least one image in the boundary among the images in which the embedding element is embedded, and registration for registering in the storage means in association with information specifying information to be provided to the user associated with the image element The embedding means includes the boundary on the acquired image according to a distance relationship between a position of a center of the image surrounded by the boundary and a position specified by the position information. The position and size of embedding are determined.

  In addition, the embedding unit may be configured so that the sum of the index indicating the size of the boundary and the distance between the position of the center of the boundary and the position specified by the position information is minimized. The position and size of the boundary are determined while adjusting the position and size.

  Further, the embedding unit further determines a position and a size of embedding the boundary according to a distance between the boundary arranged on the image and a peripheral edge of the image by adjustment.

  An information providing system according to the present invention includes a user terminal device, an image processing device, and a storage device, and the user terminal device includes information specifying an image associated with information to be provided to a user. A means for transmitting to the apparatus; and a position of an image element associated with information provided to the user on the image is specified by the user, and position information specifying the position of the specified image element is transmitted to the image processing apparatus. And means for acquiring an image including an image element associated with information to be provided to a user, and acquiring position information for specifying a position of the image element on the image. Means for acquiring an embedding range of an embedding element whose position to be embedded on the image is specified by the position information, and means for obtaining a distribution of feature points on the image A boundary having a predetermined shape including a predetermined number of feature points or more necessary for identification of the image element and the embedded element so that the entire embedded element is included in the boundary on the image. And embedding means for embedding in the image, associating the boundary and the image in which the embedding element is embedded with at least the image within the boundary, and information specifying information to be provided to the user associated with the image element Registration means for registering in the storage device, the embedding means according to a distance relationship between the position of the center of the image surrounded by the boundary and the position specified by the position information, The position and size of the boundary embedded on the acquired image are determined.

  The program according to the present invention includes a computer for acquiring an image including an image element associated with information to be provided to a user, a means for acquiring position information for specifying a position of the image element on the image, and the position information. Means for acquiring an embedding range of an embedding element whose position to be embedded on the image is specified, means for obtaining a distribution of feature points on the image, and a predetermined number of features necessary for identifying the image element Embedding means for embedding a boundary having a predetermined shape including a point and the embedding element in the image so that the entire embedding element is included in the boundary on the image, the boundary and the embedding Among the images in which the element is embedded, at least the image in the boundary and the information specifying the information to be provided to the user associated with the image element are associated and registered in the storage unit. Recording means, and the embedding means includes the boundary on the acquired image according to a distance relationship between a center position of the image surrounded by the boundary and a position specified by the position information. The position and size of embedding are determined.

  According to the first aspect of the present invention, when a boundary including the number of feature points necessary for identifying an image element associated with information to be provided to a user is embedded in an image, the center of the boundary is Compared to the case where the image element is not included, the image element can be arranged near the position specified by the position information specifying the position of the image element.

  According to the second aspect of the present invention, when a boundary including the number of feature points necessary for identifying an image element associated with information to be provided to the user is embedded in the image, the center of the boundary is Compared to the case where the image element is not included, the image element can be arranged near the position specified by the position information specifying the position of the image element.

  According to the third aspect of the present invention, the boundary can be arranged at a position where the boundary is not too close to the periphery of the image.

  According to the fourth aspect of the present invention, when embedding a boundary that includes the number of feature points necessary for identifying an image element associated with information provided to the user, the center of the boundary is Compared to the case where the image element is not included, the image element can be arranged near the position specified by the position information specifying the position of the image element.

  According to the fifth aspect of the present invention, when embedding a boundary that includes a number of feature points necessary for identifying an image element associated with information to be provided to the user, the center of the boundary is Compared to the case where the image element is not included, the image element can be arranged near the position specified by the position information specifying the position of the image element.

1 is an overall configuration diagram showing an embodiment of a content providing system according to the present invention. In Embodiment 1, it is the conceptual diagram which showed the principal part of the printed matter produced by printing the image in which the marker was embedded with the printer. 2 is a diagram illustrating an example of a hardware configuration of a computer that forms the image processing apparatus according to Embodiment 1. FIG. 1 is a diagram illustrating an example of a block configuration of an image processing apparatus according to Embodiment 1. FIG. 3 is a flowchart illustrating processing in the image processing apparatus according to the first embodiment. It is the flowchart which showed the detail of the marker formation process in FIG. 6 is a diagram showing a positional relationship between a feature boundary and an anchor used for explaining a range in which a score is calculated in the marker forming process in Embodiment 1. FIG. FIG. 6 is a diagram showing a positional relationship between a feature boundary and an anchor used to explain a score calculation formula obtained in the marker forming process in the first embodiment. FIG. 10 is a diagram illustrating a positional relationship among a feature boundary, an anchor, and a printed material frame used for explaining a range in which a score is calculated in the marker forming process according to the second embodiment.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is an overall configuration diagram showing an embodiment of a content providing system according to the present invention. FIG. 1 shows a configuration in which a user terminal device 1, an image processing device 30, a database server 2, and a printer 3 are connected to a network 4, respectively. The user terminal device 1 is a computer used by a user who wants to embed a marker in an image to be printed. The image processing device 30 embeds a marker in the image in response to a request from the user, generates a partial image at a position associated with the marker, and content information in which content is associated with the image. The printer 3 prints an image with embedded markers on a paper medium in response to a request from the user. The database server 2 is a storage device in which content information generated by the image processing device 30 is stored, and is realized by a server computer in the present embodiment. Further, FIG. 1 shows a printed matter 10 printed by the printer 3 and a mobile phone 5 equipped with a camera.

  FIG. 2 is a conceptual diagram showing a main part of a printed material 10 created by printing an image with a marker embedded by the printer 3. In the printed material 10 illustrated in FIG. 2, an image of a multifunction peripheral and a woman holding a booklet behind the multifunction peripheral is printed. And the marker 6 is superimposed and printed on the image of a woman. The marker 6 is a mark that associates digital content with an image printed on the printed material 10 or an image element included in the image, and includes an anchor 7, a media icon 8, and a feature boundary 9. The anchor 7 is position information that specifies the position of an image element associated with digital content. In FIG. 2, since the anchor 7 is arranged on the booklet, it can be easily assumed that digital contents related to the description contents of the booklet are acquired. Alternatively, when it is desired to associate digital content with the entire printed image, the anchor 7 may be arranged on the booklet as an arbitrary position on the image. The position of the anchor is represented by coordinate data. The anchor 7 is also position information for specifying the display position of the media icon 8 on the printed material 10. In other words, it is position information for specifying a position where the media icon 8 as an embedding element is embedded in an image to be printed on the printed matter 10. The media icon 8 is information that displays the digital content, that is, the type of information provided to the user in an identifiable manner. The media icon 8 illustrated in FIG. 2 includes an image in the shape of a video camera in order to notify that the digital content is a moving image. In addition, information such as sound and still images is included. You may make it include the image of the shape of the speaker or camera which notifies. In addition to the above-described image that conveys the type of digital content, the media icon 8 is formed by an image that represents the boundary for specifying the range of the media icon 8, that is, the shape and size, around this image. In the present embodiment, the media icon 8 has a circular shape as a whole, and a shape blown out from the anchor 7, but may be formed in other shapes. The feature boundary 9 is a boundary having a predetermined shape for specifying an area including a predetermined number or more of feature points, and is a line image for showing the photographer an approximate range to be captured by the camera as will be described later. . The feature points of a predetermined number or more are feature points composed of a minimum number or more necessary for identifying the image element associated with the content. That is, if the range surrounded by the feature boundary 9, in other words, the entire marker 6 is photographed, at least feature points sufficient to identify the image element associated with the content are captured. In the present embodiment, the shape of the region specified by the feature boundary 9 is circular, but it may be formed in other shapes.

  The marker 6 is generated to be transparent or translucent so that the superimposed image can be visually recognized. Even in the case of transparency, at least a line image representing the boundary between the feature boundary 9 and the media icon 8 and a media shape such as a video camera included in the media icon 8 are printed so as to be visible.

  Here, using FIG. 1 and FIG. 2, a flow of processing in which the user acquires digital content associated with the printed matter 10 using the content providing system of the present embodiment will be described.

  First, as advance preparation for providing a service for providing content by the system, content information generated by associating an image including the marker 6 with digital content associated with the marker 6 is registered in the database server 2. Note that the image element associated with the digital content may be a part of the image element of the printed material 10 as in the booklet of the present embodiment, or the entire printed image. This registration process will be described later.

  The user who has acquired the printed material 10 photographs the marker 6 formed on the printed material 10 using a camera mounted on the mobile phone 5. At this time, the user images the entire marker 6 including the feature boundary 9 on the printed material 10 as at least an imaging range. In addition, when the some marker 6 exists in the printed matter 10, a user image | photographs the whole marker 6 linked | related with the digital content to acquire. When the marker 6 is photographed, the mobile phone 5 transmits the imaging data to the database server 2.

  When the image data captured by the mobile phone 5 is transmitted, the database server 2 searches the image data of content information stored therein based on the image data. Specifically, the feature points of the received image data and each image data included in the content information are compared and collated. Then, the digital content associated with the image data most similar to the received image data is taken out and returned to the mobile phone 5. When information specifying the storage location of digital content such as a URL is registered in association with the image data as information specifying information to be provided to the user, the digital content is acquired based on the corresponding URL. May be transmitted. Alternatively, a URL may be transmitted.

  When the digital content is transmitted, the mobile phone 5 refers to the digital content by displaying it on the screen. When the URL is transmitted, the digital content is acquired from the storage location specified by the URL and displayed on the screen. In this manner, the user uses the mobile phone 5 to acquire and refer to digital content associated with the image element of the printed matter 10.

  As described above, the content providing system shown in FIG. 1 provides digital content associated with the printed matter 10 to the user according to the photographing action using the mobile phone 5. The embodiment is characterized in the process of forming the marker 6 embedded in the printed material 10. Here, the configuration of the image processing apparatus 30 that performs the process of forming the marker 6 will be described below.

  FIG. 3 is a diagram illustrating an example of a hardware configuration of a computer forming the image processing apparatus 30 according to the present embodiment. The computer forming the image processing apparatus 30 in the present embodiment can be realized with a general-purpose hardware configuration that has existed in the past. That is, as shown in FIG. 3, the computer is provided as a CPU 11, ROM 12, RAM 13, HDD controller 15 connected to a hard disk drive (HDD) 14, a mouse 16 and keyboard 17 provided as input means, and a display device. An input / output controller 19 for connecting each display 18 and a network controller 20 provided as a communication means are connected to an internal bus 21. Since the user terminal device 1 and the database server 2 are the same computer, they may be configured in the same manner as the image processing device 30.

  FIG. 4 is a diagram illustrating an example of a block configuration of the image processing apparatus 30 according to the present embodiment. The image processing apparatus 30 according to the present embodiment includes an image transmission / reception unit 31, a feature point distribution calculation unit 32, an anchor position reception unit 33, a marker embedding unit 34, a content specifying information acquisition unit 35, a content information generation unit 36, and content information. A registration unit 37 is included. The image transmission / reception unit 31 acquires an image including an image element associated with digital content. Further, the image in which the marker 6 is embedded is transmitted to the user terminal device 1. The feature point distribution calculation unit 32 calculates the distribution of feature points of the acquired image by calculation. The anchor position receiving unit 33 acquires coordinate data indicating the position of the anchor on the image designated by the user. The marker embedding unit 34 embeds the marker 6 in the image. The content specifying information acquisition unit 35 acquires information for specifying the content associated with the image element. In this embodiment, a URL (Uniform Resource Locator) is used as information for specifying content. The content information generation unit 36 generates content information by associating at least the image in the feature boundary 9 among the images in which the marker 6 is embedded and information specifying the digital content associated with the image element. The content information registration unit 37 registers the generated content information by transmitting it to the database server 2.

  The components 31 to 37 in the image processing apparatus 30 are realized by a cooperative operation between a computer that forms the image processing apparatus 30 and a program that operates on the CPU 11 mounted on the computer.

  Further, the program used in this embodiment can be provided not only by communication means but also by storing it in a computer-readable recording medium such as a CD-ROM or DVD-ROM. The program provided from the communication means or the recording medium is installed in the computer, and various processes are realized by the CPU of the computer sequentially executing the installation program.

  Next, as a preparation for providing a service for providing content by the content providing system according to the present embodiment, processing until the marker 6 is embedded in the image associated with the digital content and the content information is registered in the database server 2 is shown in FIG. This will be described with reference to the flowchart shown in FIG.

  In the user terminal device 1, the user selects an image including an image element that the user wants to associate with the digital content, confirms the image by displaying it on a display, and transmits the image to the image processing device 30.

  In the image processing device 30, the image transmission / reception unit 31 acquires the image transmitted from the user terminal device 1 by receiving it (step 100). When information specifying the storage location of an image is transmitted from the user terminal device 1 as information specifying the image instead of the image data itself, the image is acquired by accessing the storage location. The feature point distribution calculation unit 32 calculates the feature point distribution of the received image, that is, the position information of all the feature points extracted from the image (step 200). A well-known method may be used as the calculation method.

  On the other hand, in the user terminal device 1, the user designates the display position of the image element to which the content is to be associated on the image displayed on the display. This operation may be referred to as an operation for designating the position of the anchor 7 on the image. This position designation operation is performed using a pointing device such as a mouse. For example, when it is desired to associate the digital contents with the position of the booklet on the image, the user designates the position of the anchor 7 by moving the mouse cursor over the booklet and performing a predetermined confirming operation.

  Note that the image element associated with the digital content is identified by the feature point included in the feature boundary 9, and the feature boundary 9 is determined in accordance with the position of the marker 7, so that the user can Then, it is preferable to designate the anchor 7 in the vicinity where many feature points are considered to be distributed because the possibility that the size of the marker 6 can be further reduced is increased.

  When the anchor 7 is designated in the user terminal device 1, the anchor position accepting unit 33 in the image processing device 30 accepts coordinate data specifying the designated anchor 7 position (step 300). The position of the anchor 7 is specified by two-dimensional coordinate data expressed when a predetermined position such as an upper left vertex of an image including an image element associated with digital content is set as an origin (0, 0) of the XY coordinate system. The

  As described above, when the coordinate data specifying the position of the anchor 7 and the image to which the digital content is associated is acquired, the marker embedding unit 34 determines the size of the marker 6 and the position to be embedded, which will be described in detail later. (Step 400). Then, the marker embedding unit 34 generates an embedded image of the marker 6 by embedding the formed marker 6 at a position on the determined image (step 500).

  When the image transmission / reception unit 31 transmits the embedded image of the marker 6 generated by the above processing to the user terminal device 1 (step 600), the user displays and confirms the embedded image of the marker 6. Then, the URL is transmitted to the image processing apparatus 30 as the storage location information of the digital content associated with the image.

  When the content specifying information acquisition unit 35 receives the URL transmitted from the user terminal device 1 (step 700), the content information generation unit 36 detects the vicinity of the marker 6 from the image generated by the marker embedding unit 34. Content information is generated by extracting an image and associating the extracted image data with the acquired URL (step 800). The image to be extracted only needs to include at least the marker 6, and does not need to be an image of only the marker 6. Since the feature boundary 9 of the marker 6 includes a number of feature points that can identify image data, if the image near the marker 6 is also extracted, the number of feature points included in the extracted image is calculated. Increase further. Further, for example, extracting a rectangular image including the marker 6 is easier than extracting only the marker 6 having an arbitrary shape. Then, the content information registering unit 37 registers the content information by transmitting the generated content information to the database server 2 (step 900).

  In this way, by storing the content information in the database server 2, as described above, when the imaging data of the printed matter 10 including the image of the marker 6 is transmitted from the mobile phone 5, the database server 2 A URL associated with image data similar to the image data is extracted, and the URL is transmitted to the mobile phone 5. Alternatively, digital content may be acquired from the storage location specified from the URL and transmitted to the mobile phone 5. Accordingly, the user of the mobile phone 5 refers to the digital content associated with the printed material 10 or a specific image element included in the printed material 10.

  In addition, when setting the some marker 6 to an image, what is necessary is just to perform the process of step 300-900 mentioned above repeatedly.

  As described above, in the present embodiment, digital content is provided to the user. However, the above processing procedure is an example, and the present invention is not limited to the procedure illustrated in FIG. For example, the image processing apparatus 30 may first receive the image, the position of the anchor 7 and the URL from the user terminal apparatus 1 together. That is, the position of the image and the anchor 7 may be acquired before the marker forming process, and the URL may be acquired before the process of generating the content information.

  Next, the marker 6 forming process by the marker embedding unit 34 which is characteristic in the present embodiment will be described with reference to the flowchart shown in FIG. This marker forming process is a process of determining the radius of the feature boundary 9 as the size of the marker 6 and the position coordinate of the center point of the circular feature boundary 9 as the position where the marker 6 is embedded in the image. This corresponds to the detailed process description.

  First, the marker embedding unit 34 acquires the size of the media icon 8 (step 401). In the present embodiment, for simplification of description, it is assumed that the size of the media icon 8 is known and the direction displayed with respect to the anchor 7 is fixed.

  The marker embedding unit 34 first initializes each value of the maximum radius ur and the minimum radius lr that can be taken by the feature boundary (step 402). That is, assuming that the radius of the feature boundary 9 to be obtained exists between the maximum radius and the minimum radius, the maximum radius and the minimum radius are set as processing ranges. Basically, a large numerical value that cannot be set as the radius of the feature boundary 9 is set as the maximum radius, and a small numerical value that cannot be set as the radius of the feature boundary 9 is set as the minimum radius. However, from the viewpoint of processing efficiency and reality, the maximum radius that the feature boundary 9 can take is half of the length of the short side of the printed material 10, and therefore the maximum radius is half of the length of the short side of the printed material 10. It is preferable to set such a value as an initial value. Since it is appropriate from the appearance of the marker 6 that the media icon 8 is included in the feature boundary 9, the minimum radius is a radius specified from the embedding range of the media icon 8 acquired in step 401. Is set as the initial value. In the case of this embodiment, the radius specified from the embedding range of the media icon 8 is, strictly speaking, the length of the balloon from the anchor 7 added to the radius of the circular media icon 8 itself. Value. In any case, for the minimum radius, a value that will cause the entire media icon 8 to be surrounded by the feature boundary 9 is set as an initial value.

Further, the marker embedding unit 34 is initialized with an invalid value for the score best when the radius br of the feature boundary 9 to be determined in this processing and the coordinates (x _b , y _b ) of the center point of the feature boundary 9 are obtained. (Step 403). An invalid value is a value that is not possible. For example, a negative number may be set. The score will be described later. Then, the following processing is repeatedly executed until the size and position of the feature boundary 9 considered to be optimal are obtained.

  First, the difference between the current maximum radius ur and the minimum radius lr is compared with a predetermined threshold value. If the difference is larger than the threshold (Y in Step 404), a value obtained by dividing the sum of the maximum radius ur and the minimum radius lr by 2 is set to the radius cr temporarily set as the radius of the feature boundary 9 (Step 405). ). The predetermined threshold is a value serving as a reference for determining that an appropriate feature boundary size and position has been obtained. Details of this threshold will be described later.

Then, the position (x _a , y _a ) of the anchor 7 designated by the user is included in the feature boundary 9 and includes a sufficient number of feature points necessary for identifying the image element associated with the digital content. The coordinates of the center point of the feature boundary having the highest score among the feature boundaries having a radius cr are obtained (step 406). This process will be described in detail with reference to FIGS.

FIG. 7 shows an anchor 7 designated by the user on the image, a feature boundary 9 having a temporarily set radius cr, and a center point 41 of the feature boundary 9. Further, FIG. 8 shows that the distance between the center point 41 of the feature boundary 9 and the anchor 7 is represented by d. Here, although the score s is calculated in step 406, in the present embodiment, the score s is obtained by an inverse number obtained by adding the radius cr and the distance d of the feature boundary 9. That is, the score is calculated by the formula s = 1 / (cr + d). According to this expression, the value of the score s increases as the radius cr of the feature boundary 9 decreases. That is, if at least a predetermined number of feature points set as one of the conditions for obtaining the center coordinates (x _c , y _c ) of the feature boundary 9 in step 406 are included, the temporarily set feature boundary The smaller the radius cr of 9, the higher the possibility that it will be finally selected as the radius of the feature boundary 9. Also, the shorter the distance d, the larger the value of the score s. That is, the closer the distance between the center point 41 of the feature boundary 9 and the anchor 7 is, the higher the possibility that the feature boundary 9 centered on the center point 41 will be finally selected. In the present embodiment, the feature boundary 9 is finally determined based on the relationship between the size of the feature boundary 9 and the distance between the feature boundary 9 and the anchor 7 as described above.

Further, the position (x _a , y _a ) of the anchor 7 set as one of the conditions for obtaining the center coordinates (x _c , y _c ) of the feature boundary 9 in step 406 is included in the feature boundary 9. In order to satisfy the above condition, as shown in FIG. 7, a radius 42 in a region 42 represented by a square having a length of one side is obtained by multiplying the parallel root of 2 (root 2) and the radius cr. The score s is calculated each time while the center point 41 of the feature boundary 9 of cr is moved. Since the score s is calculated as many times as the feature boundary 9 is moved, the score s having the highest value among the calculated values is the score at the feature boundary 9 with the radius cr temporarily set in step 405. Let s. Then, the coordinates of the center point 41 of the feature boundary 9 when the highest score s is obtained are obtained as (x _c , y _c ).

The score s may be obtained by squeezing the center point 41 of the feature boundary 9 in the region 42 one pixel at a time, but may be obtained by skipping several pixels from the viewpoint of improving processing efficiency and speed. Good. Further, the condition that the position (x _a , y _a ) of the anchor 7 in step 406 is included in the feature boundary 9 strictly satisfies the condition indicated by a circle having a radius cr centered on the anchor 7. For this reason, the score s may be obtained in the region indicated by the circle having the radius cr. However, in this embodiment, the score s is calculated in the area of the square 42 having vertices in the direction of 45 degrees from the position of the anchor 7 from the viewpoint of efficiency and speedup of processing.

In step 406, the score s and the center coordinates (x _c , y _c ) at the feature boundary 9 of the temporarily set radius cr are obtained as described above. Here, there is a feature boundary 9 that satisfies the condition shown in step 406 because the temporarily set radius cr is too small and a predetermined number or more of feature points do not exist in the region indicated by the feature boundary 9. If not (N in Step 407), the temporarily set radius cr is substituted for the minimum radius lr (Step 411), and the process returns to Step 404. That is, since the radius cr temporarily set in step 405 may be too small, the radius cr is set to the minimum radius lr so that the radius cr newly set in step 405 is larger than the immediately preceding radius cr. Step 406 will be retried.

  Even if the feature boundary 9 satisfying the condition shown in step 406 exists (Y in step 407), the score s obtained in step 406 is not the current value because the radius cr of the feature boundary 9 is large. If the value is not greater than the effective score best (N in Step 408), the temporarily set radius cr is substituted for the maximum radius ur (Step 412), and the process returns to Step 404. That is, since the radius cr temporarily set in step 405 may be large, the radius cr is set to the maximum radius ur so that the radius cr temporarily set in step 405 is smaller than the immediately preceding radius cr. Step 406 will be tried again.

If the score s obtained in step 406 is greater than the currently valid score best (Y in step 408), the score best is updated with the score s (step 409). Also, by updating (x _b , y _b ) with the coordinates (x _c , y _c ) of the center point of the feature boundary 9 when the score s reaches the highest value, and br with the radius cr, the highest is obtained. The coordinate data and radius at the time of the score are temporarily stored (step 410). Further, in order to find the feature boundary 9 having the highest score, it is desirable to try to calculate the score by reducing the radius cr of the feature boundary 9, so that the temporarily set radius cr is substituted for the maximum radius ur. (Step 412), the process returns to Step 404.

  The processes in steps 404 to 412 described above are repeatedly executed until the difference between the current maximum radius ur and the minimum radius lr is equal to or smaller than the threshold (N in step 404).

  In the present embodiment, the difference between the maximum radius ur and the minimum radius lr is reduced by replacing the value of the maximum radius ur or the minimum radius lr with the temporarily set radius cr of the feature boundary 9 as described above. In other words, when the interval between the maximum radius ur and the minimum radius lr is narrowed and the interval between the maximum radius ur and the minimum radius lr falls within a predetermined threshold, the temporarily held radius br is changed to the feature boundary. It is determined as a radius br of 9.

  As can be seen from the equation for calculating the radius cr shown in step 405, in the present embodiment, the preferred radius br of the feature boundary 9 is determined while narrowing down the difference between the maximum radius ur and the minimum radius lr by half. doing. It seems to be optimal to determine the radius br of the feature boundary 9 by gradually reducing the interval between the maximum radius ur and the minimum radius lr, or by repeating the process until the maximum radius ur and the minimum radius lr substantially coincide. It can also be considered. However, the feature boundary 9 is intended to designate a minimum range for the user to shoot in order to acquire digital content in the first place. Even if the radius of the feature boundary 9 is narrowed more than necessary in order to obtain the optimum solution, the narrowed range is included in the shooting range. Therefore, the optimum solution is obtained from the viewpoint of processing efficiency and speedup. I don't think it makes much sense. That is, the predetermined threshold value used in step 404 can be said to be a value in which an allowable range of the size of the feature boundary 9 is set.

When the difference between the current maximum radius ur and the minimum radius lr is less than or equal to the threshold (N in Step 404), the marker embedding unit 34 temporarily stores the radius br and the coordinates of the center point (x _b , Y _b ), the size of the feature boundary 9 and the position on the image are determined (step 413).

When the size of the feature boundary 9 and the position on the image are determined as described above, the feature boundary 9 and the media icon on the image are determined based on the feature boundary 9 and the known positions of the anchor 7 and the media icon 8. The position of the marker 6 and the embedding position are determined simultaneously. As described above, in step 500, the marker embedding unit 34 refers to the coordinate data (x _a , y _a ) of the anchor 7 and the coordinate data (x _b , y _b ) of the center point of the feature boundary 9. Then, by placing the image of the media icon 8 and the image of the feature boundary 9 at the position determined by the anchor 7 on the image associated with the digital content, the marker 6 is embedded in the image.

  In the marker 6 embedded in the image, since the center of the feature boundary 9 and the anchor 7 are close to each other, the media icon 8 whose embedding range is specified by the anchor 7 is included in the feature boundary 9. It is embedded in a good position near the center.

Embodiment 2. FIG.
In the first embodiment, by setting the size of the media icon 8 to the initial value of the minimum radius lr in the marker forming process, the feature boundary 9 is formed with a size larger than the media icon 8, and the media icon 8 The marker 6 is arranged at a position close to the center of the feature boundary 9, and a good-looking marker 6 in which the media icon 8 does not protrude from the feature boundary 9 is formed. However, even if the positional relationship between the feature boundary 9 and the media icon 8 is good, there is a possibility that the marker 6 itself is embedded in the immediate vicinity of the periphery of the printed matter 10 and the printed matter 10 that is not good in appearance as the whole image is output. Will arise. Therefore, the present embodiment is characterized in that the marker 6 is formed in consideration of the embedded position of the marker 6 itself.

  FIG. 9 is a diagram corresponding to FIG. 8 used for explaining the calculation of the score s in the first embodiment. In the present embodiment, when calculating the score, the radius cr of the feature boundary 9 and In addition to the distance d between the center point of the feature boundary 9 and the position of the anchor 7, the shortest distance e between the periphery of the printed matter 10 and the outer periphery of the feature boundary 9 is added to the calculation formula to calculate the score s.

  The calculation formula of the score s in the present embodiment is s = 1 / {cr + d + (T−e)}. Here, e is the shortest distance, and T is a constant. Then, the above formula is applied when e <T, and otherwise s = 1 / (cr + d) is used as in the first embodiment.

  The use of the above equation when e <T means that the constant T means how much the outer periphery of the feature boundary 9 approaches the periphery of the printed material 10 and the variable e is considered in calculating the score s. Means. The larger the value of the constant T, the smaller the value of the calculated score s. Therefore, the feature boundary 9 at a position close to the periphery of the printed matter 10 is less likely to be selected. On the other hand, the smaller the value of the constant T, the larger the value of the calculated score s. Therefore, the feature boundary 9 at a position closer to the periphery of the printed matter 10 is more easily selected than when the constant T is large. Of course, when e ≧ T, the distance between the periphery of the printed matter 10 and the outer periphery of the feature boundary 9 is not considered in the same manner as in the first embodiment, and thus the feature is independent of the distance between the periphery of the printed matter 10 and the outer periphery of the feature boundary 9. The position of the feature boundary 9 is determined by the radius cr of the boundary 9 and the distance d between the center point of the feature boundary 9 and the anchor 7.

  The present embodiment is different from the first embodiment only in the calculation method of the score s in step 406, and other processing contents, configurations, and the like may be the same as those in the first embodiment, and thus description thereof is omitted.

  In each of the above embodiments, the size of the media icon 8 and the position with respect to the anchor 7 are fixed. However, for example, after determining the feature boundary 9, the position of the media icon 8 with respect to the anchor 7 is adjusted. The icon 8 may be arranged at a position further away from the feature boundary 9 to further improve the appearance of the marker 65.

  In the present embodiment, the media icon 8 representing the type of content is displayed in the feature boundary 9 as an embedded element whose display position is determined by the anchor 7. This is not limited to the media icon 8 that identifies the type of media, but may be another icon or an embedded element that provides different information.

  In the present embodiment, since the feature boundary 9 is circular, the radius is used as an index representing the size of the feature boundary 9. For example, when the feature boundary 9 is rectangular, the length of one side or the length of the side is used. The size of the feature boundary 9 may be expressed using another index such as a total value. Further, the center of the feature boundary 9 when the shape is not a symmetric shape such as a circle or a square is obtained by setting the center of a rectangular shape surrounding the feature boundary 9 as the center of the feature boundary 9, for example. May be.

  In the present embodiment, digital content has been described as an example of information provided to a user. However, other types of information may be associated with an image. Further, in the present embodiment, the printed matter 10 created by printing an image is photographed using the camera function of the mobile phone 5, but the photographed object is not necessarily printed on the printing paper by the printer 3. It is not necessary to limit to the printed matter 10 made, and an image on a booklet or an image displayed on a display may be used.

  1 user terminal device, 2 database server, 3 printer, 4 network, 5 mobile phone, 11 CPU, 12 ROM, 13 RAM, 14 hard disk drive (HDD), 15 HDD controller, 16 mouse, 17 keyboard, 18 display, 19 input Output controller, 20 Network controller, 21 Internal bus, 30 Image processing device, 31 Image transmission / reception unit, 32 Feature point distribution calculation unit, 33 Anchor position reception unit, 34 Marker embedding unit, 35 Content specific information acquisition unit, 36 Content information Generation unit, 37 Content information registration unit.

Claims (5)

Means for obtaining an image including an image element associated with information to be provided to a user;
Means for acquiring position information for specifying a position of the image element on the image;
Means for acquiring an embedding range of an embedding element in which a position to be embedded on the image is specified by the position information;
Means for obtaining a distribution of feature points on the image;
A boundary having a predetermined shape including a predetermined number of feature points or more necessary for identification of the image element and the embedded element are arranged such that the entire embedded element is included in the boundary on the image. Embedding means for embedding in the image;
Registration means for associating at least the image in the boundary and the image in which the embedding element is embedded with information specifying information to be provided to the user associated with the image element in the storage means,
Have
The embedding means determines the position and size of the embedding of the boundary on the acquired image according to the relationship between the position of the center of the image surrounded by the boundary and the position specified by the position information. An image processing apparatus characterized by determining.   The embedding means includes the center position of the boundary and the center position of the boundary so that the sum of the index representing the size of the boundary and the distance between the position of the center of the boundary and the position specified by the position information is minimized. The image processing apparatus according to claim 1, wherein the position and size of the boundary are determined while adjusting the size.   3. The image according to claim 2, wherein the embedding unit further determines an embedding position and a size of the boundary according to a distance between the boundary arranged on the image and a peripheral edge of the image by adjustment. Processing equipment. A user terminal device;
An image processing device;
A storage device;
Have
The user terminal device
Means for transmitting information specifying an image associated with information to be provided to a user to the image processing apparatus;
Means for causing the user to specify a position of an image element associated with information to be provided to the user on the image, and transmitting position information specifying the position of the specified image element to the image processing apparatus;
Have
The image processing apparatus includes:
Means for obtaining an image including an image element associated with information to be provided to a user;
Means for acquiring position information for specifying a position of the image element on the image;
Means for acquiring an embedding range of an embedding element in which a position to be embedded on the image is specified by the position information;
Means for obtaining a distribution of feature points on the image;
A boundary having a predetermined shape including a predetermined number of feature points or more necessary for identification of the image element and the embedded element are arranged such that the entire embedded element is included in the boundary on the image. Embedding means for embedding in the image;
Registration means for associating at least the image in the boundary among the images in which the boundary and the embedding element are embedded and information for specifying information to be provided to the user associated with the image element in association with the storage device; ,
Have
The embedding means determines the position and size of the embedding of the boundary on the acquired image according to the relationship between the position of the center of the image surrounded by the boundary and the position specified by the position information. An information providing system characterized by deciding. Computer
Means for obtaining an image including an image element associated with information to be provided to a user;
Means for acquiring position information for specifying a position of the image element on the image;
Means for acquiring an embedding range of an embedding element in which a position to be embedded on the image is specified by the position information;
Means for obtaining a distribution of feature points on the image;
A boundary having a predetermined shape including a predetermined number of feature points or more necessary for identification of the image element and the embedded element are arranged such that the entire embedded element is included in the boundary on the image. Embedding means for embedding in the image,
Registration means for associating at least the image in the boundary of the boundary and the image in which the embedding element is embedded with information specifying information to be provided to the user associated with the image element in the storage means;
Function as
The embedding means determines the position and size of the embedding of the boundary on the acquired image according to the relationship between the position of the center of the image surrounded by the boundary and the position specified by the position information. A program characterized by deciding.