JP7803757B2 - Content providing device, content providing method, program, and recording medium - Google Patents

Description

The present invention relates to a content providing device, content providing method, program, and recording medium that provide content when an image printed on a medium is read.

With the spread of network communication technology, users can now send images to each other, and images provided by one user can be printed by another user (see, for example, Patent Document 1).

In addition, content may be provided when a printed image is scanned by a scanning device, for example, to encourage printing of the image provided (see, for example, Patent Document 2). Patent Document 2 describes a method in which a printout or the like is photographed (captured) with an information terminal such as a smartphone, and predetermined video content is then composited with the image of the printout displayed on the information terminal. In other words, the technology described in Patent Document 2 provides content for augmented reality (AR).

Japanese Patent Application Laid-Open No. 2019-192016 JP 2016-58024 A

In the technology of Patent Document 2, content provided in response to scanning of a printout is displayed at a position determined based on a part of the image (a so-called marker) printed on the printout. In other words, on the display screen of the information terminal that captures the printout, the content is displayed at a position corresponding to the position of the marker.

On the other hand, the position of the marker relative to the print (more specifically, the relative position between the print and the marker) may change depending on, for example, the size of the medium on which the image is printed. Therefore, if the size of the medium used to print the image is not appropriate, the content displayed by photographing (capturing) the print may not be displayed properly on the display screen.

The present invention was made in consideration of the above circumstances, and specifically aims to provide a content providing device, content providing method, program, and recording medium that can appropriately display content provided by reading an image acquired via a network when the image is printed.

To achieve the above object, the content providing device of the present invention is a content providing device equipped with a processor, which performs the following processes: setting at least a portion of a target image provided by a first user as a first image; saving content displayed on the display screen of a second user's terminal in association with the first image; transmitting print data to the terminal for printing the target image and a second image corresponding to the content's save destination on a designated medium designated by the first user; receiving a scanned image of the scanned print from the terminal when a printout generated by printing the target image and the second image on the designated medium in accordance with the print data is scanned by the terminal; identifying a save destination from a portion of the scanned image that corresponds to the second image; and transmitting display data to the terminal for displaying the content saved in the save destination at a position on the display screen corresponding to the portion of the scanned image that corresponds to the first image.

In addition, in the process of transmitting display data, the above processor may transmit to the terminal display data that displays the content in a state in which the content is superimposed on the scanned image displayed on the display screen.

The processor may also execute a process to receive setting information indicating settings related to content display from the first terminal. In this case, the processor may transmit display data to the terminal that displays the content in accordance with the settings.

Furthermore, in the process of transmitting display data, the processor may transmit to the terminal display data that displays content stored in a storage destination and set by the first user.

Furthermore, in the process of transmitting print data, the above processor may transmit to the terminal print data for printing the target image and the second image on a specified medium whose size is specified by the first user.

The number of times display data can be transmitted may also be stored in the storage device for each user. In this case, the processor may execute a process to reduce the number of times the second user can transmit display data depending on the number of times the process of transmitting display data to the terminal has been executed. If the remaining number of times the second user can transmit display data is 0, transmission of display data to the terminal may be restricted.

In addition, whether or not the fee for the display data has been paid may be stored in the storage device. In this case, the processor may transmit the display data to the terminal of the second user who has paid the fee.

The processor may also execute processing to transfer at least a portion of the fee paid by the second user to a specified remittance destination.

The processor may also execute a process for updating content stored in a storage location in association with the first image, and in the process for transmitting display data to the terminal after the content has been updated, transmit display data that displays the updated content at the above-mentioned position.

Alternatively, a first user may use a first terminal, a second user may use a second terminal, and the first terminal may acquire printer information regarding the printer that the second user will use to print an image. In this case, the processor may transmit to the second terminal print data for printing the target image and the second image on a specified medium according to the printer information acquired by the first terminal.

Furthermore, to solve the above-mentioned problem, the content provision method of the present invention is characterized in that the following steps are carried out by a processor: setting at least a portion of a target image provided by a first user as a first image; saving content displayed on the display screen of a second user's terminal in association with the first image; transmitting print data to the terminal for printing the target image and a second image corresponding to the destination of the content on a designated medium designated by the first user; receiving a scanned image from the terminal, which is an image of the scanned print, when the printout generated by printing the target image and the second image on the designated medium in accordance with the print data is scanned by the terminal; identifying a destination from a portion of the scanned image that corresponds to the second image; and transmitting display data to the terminal for displaying the content saved in the destination at a position on the display screen that corresponds to the portion of the scanned image that corresponds to the first image.

Furthermore, in the step of transmitting the display data, the processor may transmit to the terminal display data that displays the content in a state in which the content is superimposed on the scanned image displayed on the display screen.

The processor may also perform the step of acquiring setting information indicating settings related to content display. In this case, in the step of transmitting display data, the processor may transmit, to the terminal, display data that displays the content in accordance with the settings.

The processor may also perform a process of updating the content stored in the storage location in association with the first image. In this case, in the step of transmitting display data to the terminal after updating the content, the processor may transmit display data that displays the updated content at the position described above.

The present invention also makes it possible to realize a program that causes a computer to execute each step included in the content provision method described above. Furthermore, the present invention also makes it possible to realize a computer-readable recording medium on which a program that causes a computer to execute each step included in the content method described above is recorded.

According to the present invention, when an image acquired via a network is printed, the content provided can be displayed appropriately by scanning the printout.

FIG. 10 is a diagram showing a first example of content. FIG. 10 is a diagram showing a second example of content. FIG. 10 is a diagram showing a third example of content. FIG. 10 is a diagram showing a fourth example of content. FIG. 1 is a diagram showing a procedure for providing content (part 1). FIG. 10 is a diagram showing the procedure for providing content (part 2). FIG. FIG. 1 is a diagram illustrating a configuration of a content providing system. 1 is a diagram illustrating a hardware configuration of a content providing device according to an embodiment of the present invention. FIG. 2 is a diagram illustrating an example of management data stored in a storage device. FIG. 2 is an explanatory diagram illustrating functions of a content providing device according to an embodiment of the present invention. FIG. 1 is a diagram showing a content provision flow according to one embodiment of the present invention (part 1). FIG. 2 is a diagram showing a content provision flow according to one embodiment of the present invention (part 2). FIG. 10 is a diagram showing a content update flow.

One embodiment of the present invention (hereinafter referred to as this embodiment) will be described with reference to the drawings. However, the embodiment described below is merely an example provided to facilitate understanding of the present invention and is not intended to limit the present invention. Furthermore, the present invention may be modified or improved from the following embodiment without departing from the spirit of the present invention. Furthermore, the present invention includes equivalents thereof.

In this specification, the concept of "device" includes a single device that performs a specific function, as well as multiple devices that exist independently and in a distributed manner but work together (in cooperation) to perform a specific function.

In addition, in this specification, "image" refers to image data unless otherwise specified. Examples of image data include lossy compressed image data such as JPEG (Joint Photographic Experts Group) format, and lossless compressed image data such as GIF (Graphics Interchange Format) or PNG (Portable Network Graphics) format.

In this specification, a "user" refers to a user who uses the content providing device of the present invention. Using the content providing device means using the functions of the content providing device, and includes not only directly operating the content providing device but also using the functions of the content providing device from other devices (for example, a terminal owned by the user).
Furthermore, a user may be an individual, a group including multiple people such as a family, a legal entity such as a company, or an institution or other group.

<Content provision service>
A service (hereinafter, "content providing service") realized by the content providing device and content providing method according to this embodiment will be described. A user (first user) can provide content to another user (second user) through the content providing service. The second user can receive and use content provided in response to a request from the first user through the content providing service.

The content is digital content, and in particular content that can be displayed on a display device such as a monitor. In this embodiment, content for augmented reality (hereinafter referred to as AR content) is provided. In other words, when a second user to whom the AR content is provided displays an image of the real world captured by a camera or the like (more specifically, a real-time image) on a display, the AR content is composited with the image and displayed.

The AR content may be, for example, content A1 consisting of text (message) as shown in Fig. 1A, or content A2 consisting of video or images as shown in Fig. 1B. Furthermore, the AR content may be content A3 consisting of illustrations drawn by computer graphics as shown in Fig. 1C. Furthermore, the AR content may be content A4, which is composited with other images to provide a background effect (a background effect of falling rain in Fig. 1D), as shown in Fig. 1D.
The content may be displayed in a static state, or in a moving state like a video, and may also be displayed with audio output.

To outline the procedure for providing and using AR content with reference to Figures 2A and 2B, the first user first sends an image required for content provision (hereinafter referred to as the target image). Here, "the first user sends the target image" means that the target image is sent from the information processing terminal used by the first user (hereinafter referred to as the first terminal 11), specifically, the target image is uploaded. The target image is provided by the first user and is one or more images specified from a group of images owned by the first user, such as an image purchased by the second user from the first user.

As shown in Figure 2A, the first user sets the AR content to be provided and also sets the conditions (display conditions) for displaying that AR content on the second user's side. Setting AR content means creating new AR content or selecting one of several types of AR content prepared in advance.

The display conditions for the AR content are settings related to the display of the AR content, such as the display size, display position, and orientation of the AR content when displayed on the second user's side. Here, the display position is determined using the position of the first image (described below) as the reference position.

The content provision service generates print data based on the target image sent by the first user and information about the AR content set by the first user (specifically, the storage location of the AR content, etc.). The print data is data for printing the image for obtaining the content. The generated print data is sent to the second user as shown in Figure 2A. Here, "sending print data to the second user" means sending the data to the information processing terminal (corresponding to a terminal, hereinafter referred to as second terminal 12) used by the second user.

When the second user (more specifically, the second terminal 12) receives the print data, the second user prints the content acquisition image according to the print data. The device used to print the content acquisition image is, for example, a printer owned by the second user (specifically, the portable printer 13 described below). However, this is not limited to this, and a printing device not owned by the second user, such as a device installed in a store, may also be used, which can be accessed by logging in by entering a PIN or account information.

By printing the content acquisition image, the print P shown in Figure 3 is generated. The print P is a medium (e.g., photosensitive film F, described below) on which the target image and identification image are printed.

Furthermore, as shown in FIG. 3, the image forming surface of the print P includes a target image area Pa and a second image area Pb. The target image area Pa contains the printed target image (i.e., the visualized image of the target image) on the image forming surface.

In addition, the target image area Pa contains an area corresponding to the first image set in the target image (hereinafter referred to as the first image area). The first image is a template image used when displaying AR content, and is composed of all or part of the target image. The template image functions as a marker to be tracked when the AR content is displayed. In the case shown in Figure 3, for example, the face of the subject in the target image is set as the template image.

The second image area Pb is an area where an optically readable second image, such as a QR code (registered trademark), is printed. The second image is an image generated according to the save destination of the AR content set by the first user, and more specifically, is an image obtained by encoding (barcoding) the save destination. Note that the second image is not limited to an encoded image such as a QR barcode (registered trademark), and may be any readable image, such as letters, symbols, numbers, or encrypted characters. The second image may also include cryptographic information that encrypts information for identifying the first image in the target image.

The second image area Pb may be positioned so as to overlap the target image area Pa, as shown in Figure 3, or may be positioned away from the target image area Pa. In this case, it is preferable that the second image area Pb be positioned near the target image area Pa.

Then, as shown in Figure 3, the second user reads the printout P using a second terminal 12 equipped with a reading function. Reading the printout P means obtaining a read image of the printout P, and the read image represents the density (more specifically, the gradation value) of the printout P for each pixel, and is obtained by reading the printout P. To put it simply, photographing (capturing) the printout P with a photographing device corresponds to reading the printout P, and the captured image (real-time image) corresponds to the read image.

Once the printout P is scanned, the scanned image is sent from the second user, as shown in Figure 2B. At this stage, the scanned image, i.e., a real-time image of the printout P, is displayed on the display screen of the second terminal 12.

The content provision service identifies the storage destination of the AR content set by the first user based on the portion of the scanned image sent from the second user that shows the second image area Pb (i.e., the portion corresponding to the second image). Then, display data for displaying the AR content stored in the identified storage destination is transmitted (i.e., downloaded) to the second user. The display data is received by the second terminal 12. As a result, the AR content is provided to the second user.

As shown in Figures 1A to 1D, the AR content is superimposed on the scanned image and displayed on the display screen of the second terminal 12 that has received the display data. At this time, the AR content is displayed according to the display conditions set by the first user; specifically, it is displayed at the display size, display position, and orientation specified by the first user. In particular, the display position of the AR content is determined according to the portion of the display screen in which the first image area appears (i.e., the portion of the scanned image that corresponds to the first image). This allows the AR content to be displayed on the display screen while maintaining a constant positional relationship with the portion in which the first image area appears.

With the content provision service described above, for example, a first user sells an image, and a second user purchases an image (target image) from the first user and prints the purchased image to obtain a printout P. When the second user then scans the printout P using the second terminal 12, the AR content set by the first user is superimposed on the scanned image and displayed on the display screen of the second terminal 12. This allows the second user to use the AR content provided through the content provision service.

In addition to the content provision service, use of the AR content provision service may be free or paid. In this embodiment, the service is paid, and the second user can use the AR content only if they pay the fee for the AR content (strictly speaking, the display data of the AR content), i.e., the AR content can be displayed on the second terminal 12.

Note that a part or all of the fee for the AR content paid by the second user may be remitted to the first user who sold the target image, or may be remitted to a predetermined remittance destination other than the first user. The predetermined remittance destination other than the first user may be, for example, a business operator that operates a content provision service.
With the above configuration, the fee for the AR content paid by the second user can be shared between the first user and the service provider.

In addition, certain restrictions may be placed on the provision of AR content to the second user, i.e., the transmission of AR content display data. For example, the period during which AR content can be provided to the second user may be limited to a certain period after the second user purchases an image (target image) from one user. In addition, the number of times the AR content display data can be transmitted may be set, and the AR content may be provided to the second user until the remaining number of transmissions set for the second user reaches 0. Conversely, when the remaining number reaches 0, the provision of AR content to the second user may be restricted (terminated).

Furthermore, the content provided to the second user when the second user scans the printed matter P may be the same content, or may change depending on the time of scanning. In other words, the content provided when the same printed matter P is scanned may be updated depending on the time, etc.

<Contents provision system>
Next, with reference to FIG. 4, the configuration of a communication system constructed to realize a content providing service, that is, a content providing system S, will be described.
As shown in FIG. 4, the content providing system S is made up of a server computer 10, a first terminal 11, a second terminal 12, a portable printer 13, and the like.
Each device will be described below.

(server computer)
The server computer 10 is a computer that forms a platform for a content providing service, and constitutes the content providing device 20 of the present invention. The server computer 10 may be a server for a cloud service, specifically, a server computer for an ASP (Application Service Provider), SaaS (Software as a Service), PaaS (Platform as a Service), or IaaS (Infrastructure as a Service).

The server computer 10 can communicate with the first terminal 11 and the second terminal 12 via an external network N such as the Internet or a mobile communications network, and acquires various information from these devices. The acquired information is stored in the storage device 14, and the server computer 10 can read the information stored in the storage device 14. The storage device 14 is, for example, configured as built-in or external storage to the server computer 10, or as a NAS (Network Attached Storage). However, without being limited to this, the storage device 14 may also be a third computer capable of communicating with the server computer 10, such as so-called online storage.

The server computer 10 then performs various information processing related to the content provision service. Specifically, the server computer 10 generates print data for the content acquisition image and transmits that data to the second user (see Figure 2A). The server computer 10 also generates display data for the AR content and transmits that data to the second user (see Figure 2B).

(First terminal)
The first terminal 11 is a device operated by the first user, and is configured as a personal computer (PC), a workstation, a smartphone, a tablet terminal, or the like. The first terminal 11 accepts operations performed by the first user to use the content providing service. Examples of operations accepted by the first terminal 11 include an image specification operation, a print condition specification operation, an AR content setting operation, and a display condition setting operation.

The image designation operation is an operation of designating a target image to be sent (transferred) to a second user from among a group of images owned by a first user. The image designation operation may also include an editing operation of applying image processing such as trimming or image quality adjustment to the designated target image.
When the first terminal 11 accepts the image designation operation, it transmits the target image designated by that operation to the server computer 10. At this time, the target image or the edited image is transmitted in association with the ID information of the first user (strictly speaking, the first terminal 11).

The print condition specification operation is an operation for setting print conditions when a content acquisition image including a target image is printed by a second user. Specifically, the first user specifies, as print conditions, the second user who will perform the printing and the medium (hereinafter, designated medium) on which the content acquisition image is to be printed. The print conditions may also include, for example, the layout of each of the target image and the second image in the content acquisition image, as well as the print size of each image.
In this embodiment, the first user first confirms the ID information of the second user (more precisely, the second terminal 12) and the type of portable printer 13 used by the second user, and sets the print conditions after understanding this information. When the first terminal 11 accepts a print condition specification operation, it transmits information on the print conditions specified by the first user (hereinafter, "specification information") to the server computer 10. At this time, the specification information is transmitted in association with the ID information of the first user (more precisely, the first terminal 11).
Here, when the first user specifies the printing conditions, this includes not only the case where the first user specifies the printer conditions by operating the first terminal 11, but also the case where the printer conditions are automatically specified by the first terminal 11 used by the first user, as described below.

Note that the method of acquiring information regarding the type of portable printer 13 of the second user, print size, etc. (hereinafter referred to as printer information) is not limited to the above method. For example, the printer information may be stored on the second terminal 12, or may be stored in association with the second user's ID information on the server computer 10. In this case, the first terminal 11 may acquire the printer information by communicating with the second terminal 12 or the server computer 10. The first terminal 11 may also automatically set print conditions, including the specified medium, based on the acquired printer information.

The AR content setting operation is an operation for setting AR content to be provided to the second user, and for example, if the AR content is a message, it is an operation for creating (inputting) the message. Furthermore, if multiple candidates for AR content that can be provided are prepared in advance, the first user may perform the AR content setting operation by selecting one or more candidates.
When the first terminal 11 accepts the AR content setting operation, it transmits the AR content set (more specifically, created or selected) by the first user to the server computer 10. At this time, the AR content is transmitted in association with ID information of the first user (strictly speaking, the first terminal 11).

The display condition setting operation is an operation for setting the display conditions when the AR content is displayed on the second user side, and specifically, the display size, display position, orientation, etc. of the AR content are set.
When the first terminal 11 accepts the display condition setting operation, it transmits information on the display conditions set by the first user (hereinafter, "setting information") to the server computer 10. At this time, the setting information is transmitted in association with ID information of the first user (strictly speaking, the first terminal 11).

(Second terminal)
The second terminal 12 is a device operated by the second user and equipped with a reading function, and is specifically configured as a smartphone, tablet terminal, or the like equipped with a photographing device such as a camera. The second terminal 12 accepts operations performed by the second user to use the content providing service. Examples of operations accepted by the second terminal 12 include a purchase operation, a payment operation, and a reading instruction operation.

A purchase operation is an operation performed by a second user when purchasing an image from a first user. When the second terminal 12 accepts the purchase operation, it transmits information to the server computer 10 to identify the image to be purchased. Note that the information to identify the image to be purchased is associated with the ID information of the second user (strictly speaking, the second terminal 12) and sent from the server computer 10 to the first terminal 11.

The payment operation is an operation performed by the second user when paying for the AR content (more specifically, the fee for the display data of the AR content), i.e., a settlement operation. When the second terminal 12 accepts the purchase operation, it transmits information regarding the payment of the fee to the server computer 10.

The reading instruction operation is an operation performed by the second user when instructing the second user to read the printout P. When the second terminal 12 accepts the reading instruction operation, the reading device (i.e., the photographing device) of the second terminal 12 is activated. Then, with the photographing unit (specifically, the photographing lens) of the reading device facing the printout P, reading (photographing) of the printout P begins. As a result, a live video of the read image is displayed on the display screen of the second terminal 12. The second terminal 12 also transmits the read image to the server computer 10. At this time, the read image is transmitted in association with the ID information of the second user (strictly speaking, the second terminal 12).

Furthermore, when the second terminal 12 receives the printing data from the server computer 10, it controls the portable printer 13 in accordance with the printing data and causes the portable printer 13 to print the image for obtaining the content. When printing the image, the second terminal 12 may inquire of the second user whether or not printing is necessary. In this case, the second terminal 12 may cause the portable printer 13 to print the image for obtaining the content on the condition that the second user replies that printing is necessary.

Furthermore, when the second terminal 12 receives the display data for the AR content from the server computer 10, it expands the display data and displays the AR content. At this time, the AR content is displayed on the display screen of the second terminal 12 according to the display conditions set by the first user. Specifically, the AR content is displayed by being superimposed on the scanned image of the printout P displayed on the display screen. At this time, the AR content is displayed at the display position, display size, and orientation set by the first user. In particular, the display position of the AR content on the display screen is determined according to the portion of the scanned image in which the first image area appears.

(Portable printer)
The portable printer 13 is a device used by the second user to print images for obtaining content, and is wirelessly connected to the second terminal 12 via Wi-Fi (registered trademark) or Bluetooth (registered trademark). However, the present invention is not limited to this, and the portable printer 13 may also be connected to the second terminal 12 via a wired connection.
The portable printer 13 then prints the image under the control of the second terminal 12 .

The portable printer 13 of this embodiment prints images using an instant photography method. That is, the portable printer 13 stores photosensitive film inside and exposes the image-forming surface of the photosensitive film to light to form a latent image. Next, the portable printer 13 breaks a developer pot located at the end of the film, spreading developer on the film surface and making the latent image visible. After this process, the photosensitive film is ejected from the printer, and development proceeds, ultimately forming a complete image on the image-forming surface of the photosensitive film. This results in a print P.
The image printing method is not limited to the instant photography method, but may be an inkjet method, a dye-sublimation thermal transfer method, or an electrophotographic method using toner.

Photosensitive film is the medium on which images are printed, and is film specifically designed for the portable printer 13. There are multiple sizes of photosensitive film that can be used with the portable printer 13, and the type of film to be used is determined depending on the type of portable printer 13. Specifically, in this embodiment, there are multiple types of portable printers 13, and the size of the photosensitive film is determined to be one of the sizes depending on the type of portable printer 13 used by the second user.

In this embodiment, even when printing the same image, the size of the printed image changes if the size of the photosensitive film is different. Therefore, when a content acquisition image is printed on photosensitive film, the size of the target image area Pa, in other words, the position of the first image, changes depending on the size of the photosensitive film. Therefore, when the second user requests that the content acquisition image be printed, the second user must specify the type (size) of photosensitive film to be used. Therefore, in this embodiment, printing data is generated according to the specified type of photosensitive film (corresponding to the specified medium), and the generated data is sent to the second user, or more specifically, to the second terminal 12.

<Configuration example of content providing device>
Next, a configuration example of the content providing device 20 according to this embodiment will be described. As described above, the content providing device 20 according to this embodiment is configured by the server computer 10. The number of computers that make up the content providing device 20 may be one, or two or more. In other words, the content providing device 20 is realized by a processor and a program that can be executed by the processor, and is configured by, for example, a general-purpose computer.

As shown in Figure 5, the server computer 10 that constitutes the content providing device includes a processor 10A, memory 10B, a communications interface 10C, etc.

The processor 10A is configured by, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), or a TPU (Tensor Processing Unit).
The memory 10B is configured by semiconductor memory such as a read only memory (ROM) and a random access memory (RAM), for example.
The communication interface 10C is configured by, for example, a network interface card or a communication interface board.

In addition, a program (hereinafter referred to as the "content provision program") that causes the server computer 10 to function as the content provision device of the present invention is installed on the server computer 10. The content provision program is a program that causes the computer to execute each step included in the content provision method of the present invention. In other words, the processor 10A reads the content provision program and executes a series of processes related to the content provision service described above.

The content provision program may be obtained by reading it from a computer-readable recording medium, or by receiving (downloading) it via a communication line such as the Internet or an intranet.

The content providing device 20 is connected to the storage device 14 and can read information stored in the storage device 14. The storage device 14 stores information necessary for the content providing service, such as information or images acquired by the content providing device 20 from the first terminal 11 and the second terminal 12. The storage device 14 also stores management data for each user to manage usage records of the content providing service, etc.

The management data includes information about users of the content provision service, information about the AR content provided, and information about the content provision record. Specifically, as shown in FIG. 6, the management data includes ID information for the first user's first terminal 11, the storage destination for the AR content set by the first user, and ID information for the second user's second terminal 12 to which the AR content is provided. The management data also includes printer information indicating the type of portable printer 13 used by the second user. Furthermore, the management data includes, for the second user, whether or not the fee for the AR content has been paid, the amount paid, the period for which the AR content can be provided, and the number of times it can be provided.

The configuration of the content providing device 20 will be explained again from a functional perspective, with reference to Figure 7. The content providing device 20 has a first receiving unit 21, a setting unit 22, a second receiving unit 23, a storage unit 24, a third receiving unit 25, a fourth receiving unit 26, a first transmitting unit 27, a fifth receiving unit 28, an identification unit 29, and a second transmitting unit 30. The content providing device 20 also has a determination unit 31, a restriction setting unit 32, an update unit 33, and a remittance processing unit 34. These functional units are realized through cooperation between hardware devices included in the server computer 10 and software including the content providing program described above.

The first receiver 21 receives the target image transmitted from the first terminal 11. The received target image is stored, for example, in the storage device 14.

The setting unit 22 sets a first image for the target image acquired by the first receiving unit 21. At this time, the setting unit 22 sets part or all of the target image as the first image. For example, if a person appears in the target image, the person's face may be set as the first image. The first image may be set based on a location in the target image indicated by the first user, or may be set automatically by the setting unit 22.

The second receiving unit 23 receives AR content set by the first user and provided to the second user (i.e., content displayed on the display screen of the second terminal 12). Note that if the first user selects one or two candidates from multiple candidates to set the AR content, the second receiving unit 23 receives the selection results.

The storage unit 24 stores the AR content received by the second receiving unit 23 in association with the first image set by the setting unit 22. The storage destination of the AR content is not particularly limited, and may be, for example, the storage device 14. From the viewpoint of reducing the data capacity in the content providing device 20, the storage destination of the AR content is preferably the storage device 14 other than the content providing device 20.
Furthermore, the storage unit 24 generates a second image (for example, a code image such as a QR code (registered trademark)) according to the storage destination of the AR content.

The third receiving unit 25 receives and acquires setting information indicating the settings (settings) made by the first user regarding the display conditions of the AR content from the first terminal 11. The display conditions indicated by the setting information include the display size, display position, and orientation of the AR content. The display position is determined based on the aforementioned first image.

The fourth receiving unit 26 receives from the first terminal 11 specification information indicating the print conditions specified by the first user for the content acquisition images, including the target image and the second image. The print conditions indicated by the specification information include the size of the photosensitive film on which the content acquisition images will be printed (in other words, the type of portable printer 13), the layout of each of the target image and the second image, and the print size of each image. The specification information also includes ID information of the second user who will have the content acquisition images printed.

The first transmitting unit 27 generates printing data and transmits it to the second terminal 12. The printing data is data for printing the content acquisition image in accordance with the printing conditions indicated by the specification information received by the fourth receiving unit 26. By transmitting the printing data from the first transmitting unit 27, the content acquisition image is printed on a photosensitive film of the type (size) specified by the first user at the second user side.
The print data cannot be changed (overwritten) on the second terminal 12. In other words, the second user can print the content acquisition image only if he/she uses photosensitive film of the size specified by the first user.

The fifth receiving unit 28 receives a scanned image of the printout P from the second terminal 12. The printout P is generated by printing the content acquisition image in accordance with the above-mentioned printing data, and the scanned image is obtained when the second user scans the printout P using the second terminal 12. When receiving the scanned image from the second terminal 12, the fifth receiving unit 28 receives the scanned image in association with the ID information of the second terminal 12.

The identification unit 29 analyzes the scanned image received by the fifth receiving unit 28 and identifies the storage destination of the AR content from the portion of the scanned image that shows the second image area Pb (the portion corresponding to the second image).

The second transmission unit 30 creates display data of the AR content stored in the storage destination identified by the identification unit 29 and transmits the data to the second terminal 12. At this time, the second terminal 12 displays the scanned image of the printout P on its display screen.
Then, when the display data is received by the second terminal 12, the AR content, i.e., the AR content set by the first user, is displayed on the display screen in a state where it is combined with the read image. Specifically, the AR content is displayed on the display screen at a position corresponding to the portion of the read image in which the first image area appears (the portion corresponding to the first image).
The AR content is displayed according to the display conditions (settings) indicated by the setting information received by the third receiving unit 25 .

The determination unit 31 determines whether the second user to whom the AR content is provided has paid the fee for the AR content (more specifically, the fee for the display data of the AR content). Specifically, the determination unit 31 identifies the ID information of the second terminal 12 associated with the scanned image received by the fifth receiving unit 28, and references the management data corresponding to that ID information. The determination unit 31 then determines whether the second user using the second terminal 12 has paid the fee.

If the determination unit 31 determines that the second user has paid the fee, the second transmission unit 30 transmits display data of the AR content to the second terminal 12 of the second user. Conversely, if the second user has not paid the fee, the AR content is not provided to the second user, that is, the display data of the AR content is not transmitted to the second terminal 12 of the second user. In this way, in this embodiment, the AR content can be displayed only on the second terminal 12 of the second user who purchased the target image.

The restriction setting unit 32 sets restrictions on providing AR content to the second user (i.e., transmitting display data of the AR content to the second terminal 12). Specifically, the restriction setting unit 32 sets a period during which the AR content can be provided to the second user. The second transmission unit 30 transmits display data of the AR content to the second terminal 12 of the second user only during the period during which the AR content can be provided set for the second user.

The restriction setting unit 32 also sets the number of times the AR content can be provided, i.e., the number of times the display data of the AR content can be transmitted, for the second user. Furthermore, the restriction setting unit 32 reduces the number of times the second user can transmit, depending on the number of times the second transmission unit 30 executes the process of transmitting display data to the second terminal 12. When the number of times the second user can transmit reaches 0, the second user's subsequent transmission of display data to the second terminal 12 is restricted.

The number of times that the AR content can be provided (the number of times that the display data can be transmitted) may be any number of times, for example, may be only one. In other words, after the AR content has been provided to the second user once, provision of the AR content (transmission of the display data) may be restricted.
The method for restricting the provision of the AR content is not particularly limited, and the provision of the AR content (transmission of display data) may be prohibited, or the AR content itself may be deleted from the storage location. Furthermore, when a restriction is set, a notification prompting the second user to pay a fee to remove the restriction may be sent to the second user.

When predetermined conditions (hereinafter referred to as update conditions) are met, the update unit 33 updates the AR content that is provided when a second user scans the same printed matter P. In other words, the update unit 33 updates the AR content that is associated with the same first image and stored in a storage location specified by the second image.

The updated AR content may be set by, for example, the first user. That is, when an update condition is satisfied, the first user sets new AR content and display conditions, and the second receiving unit 23 receives and acquires the new AR content, and the third receiving unit 25 receives and acquires setting information indicating the new display conditions. The update unit 33 saves the new AR content as the updated AR content in the storage location of the AR content before the update. At this time, the updated AR content is associated with the first image that was associated with the AR content before the update.
When the AR content is updated, it is preferable to notify the second user of this.

When the second transmission unit 30 transmits display data for the AR content to the second terminal 12 after updating the content, the display data for the updated AR content is transmitted. As a result, the updated AR content is displayed on the display screen of the second terminal 12 at a position corresponding to the portion of the first image area in the scanned image.

The update condition is not particularly limited and can be any condition, but may be set in association with the second user, for example. Specifically, the update condition may be that a predetermined period of time has passed since the second user first scanned the printout P, or that the second user's birthday has arrived.

The remittance processing unit 34 executes processing (hereinafter referred to as the remittance processing) to remit at least a portion of the AR content fee paid by the second user to a specified remittance destination. The remittance processing is, for example, processing to remit money to the account of the first user or the operator of the content provision service, and the processing content and procedures are similar to those of remittance processing in known electronic commerce.

<Information processing flow for providing content according to one embodiment of the present invention>
An information processing flow (hereinafter, "content provision flow") by a content provision device according to one embodiment of the present invention will be described. The content provision flow employs the content provision method of the present invention. In other words, each step in the content provision flow corresponds to a component of the content provision method of the present invention.

The content provision flow proceeds as shown in Figures 8 and 9. Furthermore, each process (step) in the content provision flow is executed through cooperation between the processor 10A of the server computer 10 constituting the content provision device 20 and each of the first terminal 11 and second terminal 12.

The content provision flow is initiated, for example, when a second user purchases an image from a first user. In the content provision flow, first, the first terminal 11 transmits the image purchased by the second user, i.e., the target image, to the server computer 10 (S001). The processor 10A of the server computer 10 receives the target image from the first terminal 11 (S012). At this time, the processor 10A also obtains information about the second terminal 12 of the second user who purchased the target image.

Next, the processor 10A sets at least a portion of the target image as the first image (S003). In S003, the processor 10A may set the first image based on an editing operation on the target image performed by the first user on the first terminal 11, or may set the first image automatically in accordance with predetermined setting criteria.

Next, based on the operation by the first user, the first terminal 11 sets the AR content to be provided to the second user and displayed on the display screen of the second terminal 12, as well as its display conditions (S004). The first terminal 11 then transmits the set AR content and setting information indicating the set display conditions to the server computer 10 (S005). The processor 10A receives and acquires the AR content and setting information from the first terminal 11 (S006).

Next, the processor 10A associates the AR content received in S006 with the first image set in S003 and saves it (S007). At this time, the setting information may also be associated with the first image and saved along with the AR content. Also, in S007, the processor 10A encodes information indicating the save destination of the AR content to generate a second image.

Next, the first terminal 11 specifies print conditions for the content acquisition images, including the target image and the second image, based on operations by the first user (S008). Specifically, in S008, the type (size) of photosensitive film to be used to print the content acquisition images is specified as the print conditions. The first terminal 11 then transmits specification information indicating the specified print conditions to the server computer 10 (S009). The processor 10A receives the specification information from the first terminal 11 (S010).

Next, the processor 10A generates print data for printing the content acquisition image according to the print conditions indicated in the specification information, and transmits the generated data to the second terminal 12 (S011). The print data is received by the second terminal 12 of the second user who purchased the target image (S012).

Next, the second terminal 12 controls the portable printer 13 in accordance with the printing data to cause the portable printer 13 to print the content acquisition image (S013). As a result, the content acquisition image is printed on photosensitive film of the type (size) specified by the first user, and as a result, the second user obtains the printout P.

Next, the second terminal 12 uses its reading function to read the printed matter P, and the read image is sent to the server computer 10 (S014). At this time, the read image is associated with the ID information of the second terminal 12 that sent it. The sent read image is received by the processor 10A (S015).

Next, processor 10A analyzes the scanned image received in S015 and extracts the portion of the scanned image that shows second image area Pb (the portion corresponding to the second image). Processor 10A then identifies, from the extracted portion, the storage destination for the AR content set by the first user and provided to the second user (S016). Processor 10A also reads the AR content from the identified storage destination.

Next, processor 10A determines whether the second user to whom the AR content is to be provided has paid the fee for the AR content (the fee for the display data of the AR content) (S017). If the second user has paid the fee, the process proceeds to S018; if the fee has not been paid, the flow ends.

Next, the processor 10A determines whether the remaining number of times the display data can be sent set for the second user is 0 (S018). If the remaining number is not 0, the process proceeds to S019; if the remaining number is 0, the flow ends.

Next, processor 10A generates display data for the AR content saved in the storage location identified in S016 and transmits the generated display data to second terminal 12 (S019). When second terminal 12 receives the display data (S020), it unpacks the display data and displays the AR content on the display screen (S021). At this time, the read image (i.e., a real-time image of printout P) is displayed on the display screen of second terminal 12, and the AR content is displayed superimposed on the read image. In other words, in S019, display data for displaying the AR content superimposed on the read image displayed on the display screen is transmitted to second terminal 12.

Furthermore, in S021, the AR content is displayed according to the display conditions indicated by the setting information received in S006. To explain in more detail, the AR content is displayed at a position on the display screen that is determined based on the portion of the scanned image in which the first image area appears (the portion corresponding to the first image). In other words, in S019, display data for displaying the AR content according to the display conditions, more specifically, display data for displaying the AR content at a position on the display screen that corresponds to the portion in which the first image area appears, is transmitted to the second terminal 12.

The content provision flow ends when the above series of steps are completed. According to the content provision flow of this embodiment, the content acquisition image obtained via the network is printed, and the provided AR content can be displayed appropriately by scanning the printout P.

Specifically, if the type (size) of photosensitive film used to print the content acquisition image changes, the size of the printed image will change. In that case, the position of the first image (i.e., template image) in the printed image will change. Therefore, if the correct type (size) of photosensitive film is not used, the AR content may not be displayed in the appropriate position. In such a situation, there is a risk that the intention of the first user who set the content will not be properly reflected when the AR content is displayed.

In contrast, in this embodiment, data for printing the content acquisition image onto a type of photosensitive film (designated medium) specified by the first user or first terminal is transmitted to the second terminal 12. This printing data cannot be changed (overwritten) by the second terminal 12, so the content acquisition image is printed only on the type of photosensitive film specified by the first user. This allows the content acquisition image to be properly printed on the photosensitive film, and the first image to be positioned in the correct position in the content acquisition image. As a result, when a printout P of the content acquisition image is scanned, the AR content is displayed properly on the display screen (more specifically, in the appropriate position on the display screen).

The above-mentioned effect is particularly significant when AR content is composited and displayed on a scanned image of the printout P. In other words, in this embodiment, AR content can be displayed appropriately using the above-mentioned procedure, making it easier to popularize the provision of AR content through network printing. As a result, content provision services can be revitalized.

In addition, in this embodiment, restrictions are placed on the provision of AR content (in other words, the transmission of AR content display data) in terms of timing, number of times, etc. This makes it possible to avoid inappropriate use and copying of AR content, and as a result, it is possible to ensure the appropriate use and protection of AR content.

In addition, in this embodiment, the content update flow shown in Figure 10 is executed separately. The content update flow is started when the update conditions are met (S031). When the flow starts, the first user sets new AR content and its display conditions, and the first terminal 11 transmits setting information indicating the new AR content and the new display conditions to the server computer 10 (S032, S033).

When the processor 10A receives new AR content and setting information (S034), it updates the AR content that was associated with the first image and saved in S007 to the new AR content (S035). This completes the content update flow.

Then, in step S019 of the content provision flow, which is performed after the AR content has been updated, processor 10A transmits display data for the updated AR content. This display data causes the updated AR content to be displayed on the display screen of second terminal 12 at a position corresponding to the portion of the scanned image where the first image area appears. By updating the AR content that can be displayed by scanning a single printout P in this way, the second user's interest, specifically their interest in the AR content being provided, can be sustained more than if only a single AR content were displayed.

<Other embodiments>
The embodiment described above is a specific example given to clearly explain the content providing device and content providing method of the present invention, and is merely an example, and other embodiments are also possible.

In the above embodiment, the print data for the content acquisition image is received by the second terminal 12, but this is not limited to this. For example, the portable printer 13 may receive the print data directly.

Furthermore, in the above embodiment, from the perspective of content protection, restrictions are placed on the provision of AR content, specifically by setting a period of time during which the content can be provided and the number of times it can be provided. However, this is not limited to this, and if the number of second users to whom a certain AR content is provided (specifically, the number of second users who have purchased the same target image) reaches a predetermined number or more, the restrictions on that AR content may be lifted.

Furthermore, in the above embodiment, even if the printed matter P is scanned by the second terminal 12, the AR content will not be displayed unless the fee for the AR content has been paid. However, this is not limited to this, and for example, if a user who has not paid the fee for the AR content (for example, a family member of a second user who has paid the fee) scans the printed matter P with a camera-equipped smartphone or the like, the AR content may be displayed on the display screen.

Furthermore, in the above embodiment, the functions of the content providing device of the present invention are performed by the processor 10A provided in the server computer 10, but this is not limited to this. The processor provided in the content providing device of the present invention may also be provided in the terminals used by the user (i.e., the first terminal 11 and the second terminal 12), as well as in the portable printer 13. In other words, some or all of the above-mentioned functional units provided in the content providing device of the present invention may be provided in the first terminal 11, the second terminal 12, or the portable printer 13.

The content providing device of the present invention may also be the first terminal 11; that is, the processor included in the content providing device of the present invention may be a processor included in the first terminal 11. In this case, the processor of the first terminal 11 will perform each step in the content providing flow described above. That is, the first terminal 11 may acquire printer information from the second terminal 12, generate print data based on the acquired printer information, and send it to the second terminal 12. The first terminal 11 may also receive a scanned image from the second terminal 12, identify a storage location for the content from the portion of the scanned image that corresponds to the second image, and send display data for displaying the content to the second terminal 12.

The processor included in the content providing device of the present invention includes various types of processors, such as a CPU, which is a general-purpose processor that executes software (programs) and functions as various processing units.
The various processors also include PLDs (Programmable Logic Devices), which are processors whose circuit configuration can be changed after manufacturing, such as FPGAs (Field Programmable Gate Arrays).
Furthermore, various processors include dedicated electrical circuits such as ASICs (Application Specific Integrated Circuits), which are processors having circuit configurations designed exclusively for performing specific processes.

Furthermore, one processing unit possessed by the content providing device of the present invention may be configured by one of the various processors described above, or may be configured by a combination of two or more processors of the same or different types, for example, a combination of multiple FPGAs, or a combination of an FPGA and a CPU, etc.
Furthermore, the plurality of functional units of the content providing device of the present invention may be configured by one of various processors, or two or more of the plurality of functional units may be configured together by one processor.
Furthermore, as in the above-described embodiment, one processor may be configured by combining one or more CPUs and software, and this processor may function as multiple functional units.

Furthermore, for example, a processor may be used that realizes the functions of the entire system, including multiple functional units, of the content providing device of the present invention on a single IC (Integrated Circuit) chip, as typified by an SoC (System on Chip). Furthermore, the hardware configuration of the various processors described above may be an electric circuit (Circuitry) that combines circuit elements such as semiconductor elements.

10 Server computer 10A Processor 10B Memory 10C Communication interface 11 First terminal 12 Second terminal 13 Portable printer 14 Storage device 20 Content providing device 21 First receiving unit 22 Setting unit 23 Second receiving unit 24 Storage unit 25 Third receiving unit 26 Fourth receiving unit 27 First transmitting unit 28 Fifth receiving unit 29 Identification unit 30 Second transmitting unit 31 Determination unit 32 Restriction setting unit 33 Update unit 34 Remittance processing unit A1, A2, A3, A4 Content N External network P Printed matter Pa Target image area Pb Second image area S Content providing system

Claims (16)

A content providing device including a processor,
The processor:
setting at least a portion of a target image provided by a first user as a first image;
a process of storing content displayed on a display screen of a terminal of a second user in association with the first image;
a process of transmitting, to the terminal, print data for printing the target image and a second image corresponding to the storage destination of the content on a designated medium designated by the first user;
a process of receiving, when a printout generated by printing the target image and the second image on the specified medium in accordance with the print data is read by the terminal, a read image of the printout from the terminal;
a process of identifying the storage destination from a portion of the scanned image that corresponds to the second image;
transmitting, to the terminal, display data for displaying the content stored in the storage location at a position on the display screen corresponding to a portion of the scanned image that corresponds to the first image;
A content providing device that executes the above. The content providing device according to claim 1, wherein, in the process of transmitting the display data, the processor transmits to the terminal the display data that displays the content in a state in which the content is superimposed on the scanned image displayed on the display screen. the processor executes a process of acquiring setting information indicating settings related to display of the content;
The content providing device according to claim 1 , wherein the processor transmits the display data for displaying the content according to the setting to the terminal. A content providing device according to any one of claims 1 to 3, wherein the processor, in the process of transmitting the display data, transmits to the terminal the display data that displays the content stored in the storage destination and set by the first user. A content providing device described in any one of claims 1 to 4, wherein in the process of sending the printing data, the processor sends the printing data to the terminal for printing the target image and the second image on the specified medium whose size is specified by the first user. the number of times the display data can be transmitted is stored in a storage device for each user;
the processor executes a process of reducing the number of times the second user can transmit the display data to the terminal in accordance with the number of times the process of transmitting the display data to the terminal has been executed;
The content providing device according to claim 1 , wherein when the remaining number of transmissions allowed for the second user is 0, transmission of the display data to the terminal is restricted. whether or not the fee for the display data has been paid is stored in a storage device;
The content providing device according to claim 1 , wherein the processor transmits the display data to the terminal of the second user who has paid the fee. The content providing device of claim 7, wherein the processor executes processing to remit at least a portion of the fee paid by the second user to a predetermined remittance destination. The processor:
performing a process of updating the content stored in the storage location in association with the first image;
The content providing device according to claim 1 , wherein the process of transmitting the display data to the terminal after updating the content comprises transmitting the display data for displaying the updated content at the position. The first user uses a first terminal, and the terminal used by the second user is a second terminal;
A content providing device described in any one of claims 1 to 9, wherein when the first terminal acquires printer information regarding a printer to be used by the second user when printing an image, the processor transmits the printing data to the second terminal for printing the target image and the second image on the specified medium according to the printer information acquired by the first terminal. setting at least a portion of a target image provided by a first user as a first image;
storing content displayed on a display screen of a terminal of a second user in association with the first image;
transmitting print data to the terminal for printing the target image and a second image corresponding to the storage destination of the content on a designated medium designated by the first user;
a step of receiving, when a printout generated by printing the target image and the second image on the specified medium in accordance with the print data is read by the terminal, a read image of the printout that has been read from the terminal;
specifying the storage destination from a portion of the scanned image that corresponds to the second image;
transmitting, to the terminal, display data for displaying the content stored in the storage location at a position on the display screen corresponding to a portion of the scanned image that corresponds to the first image;
A content providing method, wherein the above is performed by a processor. The content providing method according to claim 11, wherein in the step of transmitting the display data, the processor transmits to the terminal the display data that displays the content in a state in which the content is superimposed on the scanned image displayed on the display screen. acquiring setting information indicating settings related to display of the content by the processor;
13. The content providing method according to claim 11, wherein in the step of transmitting the display data, the processor transmits the display data for displaying the content in accordance with the setting details to the terminal. updating the content stored in the storage location in association with the first image, performed by the processor;
14. The content providing method according to claim 11, wherein in the step of transmitting the display data to the terminal after updating the content, the processor transmits the display data that causes the updated content to be displayed at the position. A program for causing a computer to execute each step included in the content provision method described in any one of claims 11 to 14. A computer-readable recording medium,
A recording medium on which a program for causing a computer to execute each step included in the content providing method according to any one of claims 11 to 14 is recorded.