JP6520082B2 - Image providing apparatus, image providing method, and computer program - Google Patents

Description

  The present invention relates to the technology of AR (Augmented Reality).

  In recent years, technology of AR (Augmented Reality) has begun to spread. AR is generally translated as "augmented reality".

  According to the AR technology, when a printed matter to which an AR marker is attached is photographed by a smartphone or the like, content corresponding to the AR marker is displayed on the image of the photographed printed matter. This makes it possible for the user to feel as if the real space has expanded. The same effect can be obtained by a glasses-type wearable computer. Such content may be generally referred to as "air tag" or "AR tag".

  Conventionally, the following technologies related to AR have been proposed. In the condition storage unit of the AR content server, for each one marker attached to the product package, a plurality of contents and a provision condition of each of the plurality of contents are stored in association with each other. When content provision is transmitted from the terminal device to the AR content server, one of the plurality of pieces of content is selected based on the provision condition stored in the condition storage unit. The AR content server transmits the selected content to the terminal device, and the terminal device superimposes the content received from the AR content server and the photographed image of the camera and displays the superimposed content on the display (Patent Document 1).

JP, 2012-215989, A

  By printing an AR marker on a printed matter instead of a certain content and displaying the content when the AR marker is photographed, the content can be concealed. In addition, confidentiality can be enhanced by limiting the users who can access this content.

  Also, there is a case where it is desired to temporarily permit a specific user to access this content. In such a case, the administrator may change the setting of the access right to the content. However, the task of setting the access right is troublesome for the administrator.

  An object of the present invention is, in view of such problems, to make it possible to change the access right to content in AR more easily than in the prior art.

An image providing apparatus according to an aspect of the present invention overlaps a registered image stored in a storage unit in association with an identifier visually described on the medium with the medium or a photographed image obtained by photographing the medium. An image providing device provided to the terminal device for display on the terminal device, the transmitting means transmitting the image data of the registered image to the terminal device of the user having the access right of the registered image; in the first terminal device of a first user having an access right, the identifier different specific images are captured and, and, and in the second terminal device of the second user does not have the access right, the An access right giving unit that gives the second user the access right when a specific image is taken at a predetermined position or posture .
According to another aspect of the present invention, there is provided an image providing apparatus, wherein the medium or a photographed image obtained by photographing the medium is stored in association with the identifier visually described on the medium and stored in the storage unit. An image providing device, which is provided to the terminal device to be displayed on the terminal device so as to be superimposed on the image, and which transmits the image data of the registered image to the terminal device of the user having the access right of the registered image; The first terminal device of the first user who has the access right, the second image of the second user whose specific image different from the identifier is taken at a predetermined position or posture and which does not have the access right The second terminal device includes access right giving means for giving the access right to the second user when the specific image is photographed.
According to another aspect of the present invention, there is provided an image providing apparatus, wherein the medium or a photographed image obtained by photographing the medium is stored in association with the identifier visually described on the medium and stored in the storage unit. An image providing device, which is provided to the terminal device to be displayed on the terminal device so as to be superimposed on the image, and which transmits the image data of the registered image to the terminal device of the user having the access right of the registered image; A first terminal device of the first user who has the access right and a second terminal device of the second user who does not have the access right, when a common image is taken; Access right granting means for granting the access right to the second user only while the common image is being shot by the user.
According to another aspect of the present invention, there is provided an image providing apparatus, wherein the medium or a photographed image obtained by photographing the medium is stored in association with the identifier visually described on the medium and stored in the storage unit. An image providing device, which is provided to the terminal device to be displayed on the terminal device so as to be superimposed on the image, and which transmits the image data of the registered image to the terminal device of the user having the access right of the registered image; A case where a common image is captured in the first terminal device of the first user who has the access right and the second terminal device of the second user who does not have the access right, corresponding to the identifier And granting the access right to the second user when the second user has a second access right to a second registered image different from the registered image. Has a scan granting means, the.

  According to the present invention, the access right to content in the AR can be changed more easily than in the past.

It is a figure which shows the example of the whole structure of a document browsing system. It is a figure which shows the example of the hardware constitutions of a document server. It is a figure which shows the example of the hardware constitutions of a tablet-type terminal device. It is a figure which shows the example of the hardware constitutions of a spectacles type terminal device. It is a figure which shows the example of printed matter. It is a figure which shows the example of a functional structure of a document server. It is a figure which shows the example of a functional structure of a terminal device. It is a figure which shows the document database and the example of the record for content reproduction memorize | stored in it. It is a figure which shows the example of an access right database and the access right record memorize | stored in it. It is a figure which shows the example of a document and alternative content. It is a figure which shows the example of a document registration screen and an authority setting screen. 5 is a flowchart illustrating an example of the flow of a document registration process. It is a flowchart explaining the example of the flow of signature image registration processing. It is a figure which shows the example of arrangement | positioning of the signature to printed matter. It is a flowchart explaining the modification of the flow of signature image registration processing. It is a flow chart explaining an example of a flow of contents offer processing. It is a flowchart explaining the example of the flow of change flag update processing. It is a flow chart explaining an example of a flow of upper rank inspection right grant processing. It is a flowchart explaining the example of the flow of the whole process regarding a document.

  FIG. 1 is a diagram showing an example of the overall configuration of a document browsing system 100. As shown in FIG. FIG. 2 is a diagram showing an example of the hardware configuration of the document server 1. FIG. 3 is a diagram illustrating an example of a hardware configuration of the tablet terminal device 2T. FIG. 4 is a diagram showing an example of a hardware configuration of the glasses-type terminal device 2M. FIG. 5 is a view showing an example of the printed matter 5.

  As shown in FIG. 1, the document browsing system 100 includes a document server 1, a plurality of terminal devices 2, an image forming apparatus 3, and a communication line 4. The document browsing system 100 manages data related to a document (document) and provides a user with a document browsing service.

  Hereinafter, the case where the document browsing system 100 is used in a certain company will be described as an example. Each employee or user of this company is previously given a unique user code.

  The document server 1, the terminal device 2, and the image forming apparatus 3 can communicate with each other via the communication line 4. As the communication line 4, the Internet, a so-called LAN (Local Area Network) line, a mobile phone line or the like is used.

  The document server 1 stores data for displaying a document, and provides the data to the terminal device 2 as appropriate. A so-called server machine is used as the document server 1. Alternatively, a so-called cloud server may be used. Hereinafter, a case where a server machine is used as the document server 1 will be described as an example.

  As shown in FIG. 2, the document server 1 includes a central processing unit (CPU) 10a, a random access memory (RAM) 10b, a read only memory (ROM) 10c, a large capacity storage device 10d, and a network interface card (NIC) 10e. Composed of

  The NIC 10 e communicates with the terminal device 2 or the image forming apparatus 3 depending on a protocol such as TCP / IP.

  A document management program 10P is stored in the ROM 10c or the mass storage 10d. The document management program 10P is loaded into the RAM 10b and executed by the CPU 10a. A hard disk, a solid state drive (SSD) or the like is used as the mass storage device 10d.

  The image forming apparatus 3 is an image processing apparatus generally called an MFP (Multi Function Peripherals) or a multifunction peripheral, and is an apparatus in which functions such as copying, PC printing (network printing), faxing, and scanning are integrated.

  The terminal device 2 displays the document based on the data provided from the document server 1. As the terminal device 2, a personal computer, a tablet computer, a smartphone, a glasses-type computer, or the like is used.

  The “glass-type computer” is a type of wearable computer, and is a type of computer worn on the face like glasses, such as Google Glass of Google Inc.

  Hereinafter, a case where a tablet computer or a glasses-type computer is used as the terminal device 2 will be described as an example. The former and the latter may be described separately as “tablet type terminal device 2T” and “glasses type terminal device 2M”, respectively.

  As shown in FIG. 3, the tablet terminal device 2T uses a CPU 20a, a RAM 20b, a ROM 20c, a flash memory 20d, a touch panel display 20e, an operation button group 20f, a digital camera 20g, and a wireless communication device 20h.

  The touch panel display 20 e is configured of a flat display (for example, a liquid crystal display) and a touch panel. The flat display displays various images. It is used specifically to display documents. The touch panel detects the position touched by the user, and notifies the CPU 20a of the position.

  The operation button group 20 f includes a button for returning to the home screen, a button for determining something, a button for adjusting the volume, and a button for switching on / off the power.

  The digital camera 20g captures various images. In particular, it is used to capture a form for reading a document.

  The wireless communication device 20 h communicates with the image forming device 3 or the document server 1 according to a protocol such as TCP / IP via a base station configuring the communication line 4. As a communication method of the wireless communication device 20h, for example, Wi-Fi, Long Term Evolution (LTE), Wideband-Code Division Multiple Access (W-CDMA), Personal Handy-phone System (PHS), or the like is used.

  The client program 20P is stored in the ROM 20c or the flash memory 20d. The client program 20P is loaded into the RAM 20b and executed by the CPU 20a.

  As shown in FIG. 4, the eyeglass-type terminal device 2M uses a CPU 21a, a RAM 21b, a ROM 21c, a flash memory 21d, a display 21e, an operation button group 21f, a digital camera 21g, a wireless communication device 21h, and the like.

  The functions of the CPU 21a, the RAM 21b, the ROM 21c, the flash memory 21d, the display 21e, the operation button group 21f, the digital camera 21g, and the wireless communication device 21h are the CPU 20a, the RAM 20b, the ROM 20c, the flash memory 20d, and the touch panel of the tablet type terminal device 2T, respectively. It is basically the same as the display 20e, the operation button group 20f, the digital camera 20g, and the wireless communication device 20h.

  However, the method of reproducing an image is different between the touch panel display 20 e and the display 21 e. As described above, the touch panel display 20 e is a flat display such as a liquid crystal display. On the other hand, the display 21e reproduces an image by emitting light from the projector provided at the root of the handle to the prism provided near the end of the lens.

  The client program 20P is stored in the ROM 21c or the flash memory 21d, loaded into the RAM 21b, and executed by the CPU 21a.

  By the way, the document browsing system 100 can reproduce not only documents such as PDF (Portable Document Format) but also documents by AR (Augmented Reality) technology. At this time, the printed matter 5 is used. The printed matter 5 is distributed to a plurality of users.

  As shown in FIG. 5A, in addition to a part of a plurality of contents (objects) constituting a document, a calling marker 51 is printed on the printed matter 5. The call marker 51 is for calling from the document server 1 the content not printed on the printed matter 5.

  When the user wears the glasses-type terminal device 2 M and looks at the printed matter 5, the image data of the content is downloaded from the document server 1 based on the calling marker 51. Then, as shown in FIG. 5 (B), it looks as if this content is printed on the printed matter 5. The same appearance is also seen when the printed matter 5 is viewed while being photographed by the tablet type terminal device 2T.

  Thus, according to the document browsing system 100, AR is realized. An AR marker is used as the calling marker 51. Hereinafter, a document in which part of the content can be reproduced by the AR will be referred to as a “document 50”. Among the contents constituting the document 50, the one printed on the printed matter 5 is described as "printed content 52", and the one visible to the user by the terminal device 2 is described as "AR content 53".

  The mechanism that makes the AR content 53 appear to be printed on the printed matter 5 is realized by the document management program 10P and the client program 20P. Furthermore, the AR content 53 can be changed according to the user who views the printed matter 5. The mechanism will be described below.

  FIG. 6 is a diagram showing an example of a functional configuration of the document server 1. FIG. 7 is a diagram showing an example of a functional configuration of the terminal device 2.

  According to the document management program 10P, the document database 101, the access right database 102, the document registration unit 103, the access right registration unit 104, the print data transmission unit 105, the signature registration unit 106, the content providing unit 107, and the change flag update shown in FIG. The functions such as the unit 108 and the access right change unit 109 are implemented in the document server 1.

  According to the client program 20P, the document upload unit 201, the message display control unit 202, the signature image upload unit 203, the content download unit 204, the content display control unit 205, the temporary release request unit 206, the access right change request unit 207 shown in FIG. The terminal device 2 is provided with functions such as the change permission / non-permission notification unit 208 and the like.

[Preparation for reading document 50]
FIG. 8 is a diagram showing an example of the document database 101 and the content reproduction record 6R stored therein. FIG. 9 is a diagram showing an example of the access right database 102 and the access right record 6K stored therein. FIG. 10 is a diagram showing an example of the document 50 and the alternative content 53d. FIG. 11 is a diagram showing an example of the document registration screen 71 and the authority setting screen 72. As shown in FIG. FIG. 12 is a flowchart illustrating an example of the flow of the document registration process. FIG. 13 is a flowchart illustrating an example of the flow of the signature image registration process. FIG. 14 is a diagram showing an example of the arrangement of the signature 54 on the printed matter 5. FIG. 15 is a flowchart for explaining a modification of the flow of the signature image registration process.

  In the document server 1 of the document server 1, as shown in FIG. 8, a content reproduction record 6R is stored for each document. The contents indicated by the content reproduction record 6R will be described later.

  In the access right database 102, as shown in FIG. 9, an access right record 6K is stored for each combination of the document 50 and the user. The contents indicated by the access right record 6K will be described later.

  The content reproduction record 6R and the access right record 6K are generated by the document registration unit 103, the access right registration unit 104, and the document upload unit 201 of the terminal device 2, and registered in the document database 101 or the access right database 102.

  Here, for example, when a user Ua registers a record of a document 50 as shown in FIG. 10A using the tablet type terminal device 2T, the process of each unit at the time of registering these records and the user Describe the operation.

  The user Ua prepares the image data of the document 50 using an application such as spreadsheet or drawing installed in the tablet type terminal device 2T. Image data may be prepared by causing the image forming apparatus 3 to read a sheet on which the document 50 is written. Furthermore, the image data of the content to be replaced with the content to be classified in the document 50 is prepared by using the application or the image forming apparatus 3 or the like. For example, image data of content as shown in FIG. 10 (B) is prepared. Hereinafter, the content to be classified is referred to as "confidential content 53k", and the alternative content is referred to as "alternative content 53d". As described later, the confidential content 53k and the alternative content 53d are both treated as one of the AR content 53.

  The alternative content 53d may not be prepared, or a plurality of alternative contents may be prepared. Hereinafter, the case where two contents are prepared as the alternative content 53d will be described as an example.

  Furthermore, the user Ua causes the tablet terminal device 2T to start the client program 20P. Then, a command to switch to the registration mode is input.

  Then, in the tablet type terminal device 2T, the document upload unit 201 executes the process of uploading the document 50 to the document server 1 as follows.

  The document upload unit 201 arranges the document registration screen 71 and the authority setting screen 72 as shown in FIG. 11 and causes the touch panel display 20 e to display the document registration screen 71 and the authority setting screen 72.

  In addition to the document 50, the document registration screen 71 is provided with a registration button 71a and a security designation button 71b.

  The user Ua specifies, from among the contents (objects) of the document 50, one to be the confidential content 53k by tracing the range in which the content is located with a finger or the like. Furthermore, if it is desired to treat the document 50 as confidential information, tap the confidential designation button 71b.

  The authority setting screen 72 shows the user code of each user. Further, the upper level browsing right button 72a and the lower level browsing right button 72b for each alternative content 53d are arranged on the right side to correspond to each of them. In the present example, as described above, two alternative contents 53d are prepared, so two lower level browsing right buttons 72b are arranged two by two.

  The user Ua designates the user who gives the upper level browsing right of the document 50 by tapping the upper level browsing right button 72a (right side) corresponding to the user code of the user. It is desirable to specify the user code of the user Ua himself who is the registrant of the document 50.

  The “upper level browsing right” is a right (access right) capable of browsing the confidential content 53k.

  Furthermore, the user Ua determines, among the users who do not give the upper level browsing right, the user who gives the lower level browsing right of the document 50. Then, it designates by, for example, tapping the lower level browsing right button 72b corresponding to the user code of the user who has decided to give.

  The “lower-level browsing right” is a right (access right) that can not browse the confidential content 53k, but can instead browse the alternative content 53d.

  In this example, as described above, two alternative contents 53d are prepared. Therefore, for each user, which alternative content 53 d can be browsed is designated by tapping the corresponding lower browsing right button 72 b.

  Then, when the user Ua finishes specifying the content to be the confidential content 53k, the user who gives the upper browsing right, and the user who gives the lower browsing right, the user Ua taps the registration button 71a.

  Then, the document upload unit 201 generates designated data 6S indicating the content of the designation described above.

  Specifically, the designated data 6S indicates a user code for each user and a given access right. The access right is represented by a predetermined code. In this example, “2” represents the upper browsing right, and “1” represents the lower browsing right. When there are a plurality of alternative contents 53d, alphabets are added after “1” like “1A”, “1B”,... From the first alternative contents 53d. Further, the designated data 6S indicates the designated range (the position of the confidential content 53k) and whether or not the document 50 is treated as confidential information.

  Then, the document upload unit 201 transmits the image data of the document 50, the image data of the alternative content 53d, and the designation data 6S to the document server 1. Thereby, data on the document 50 is uploaded to the document server 1.

  In the document server 1, when the document registration unit 103, the access right registration unit 104, the print data transmission unit 105, and the signature registration unit 106 receive these data, the process for registering the document 50 is performed according to the procedure shown in FIG. Do.

  The document registration unit 103 issues a unique document code and a unique AR marker (# 801). One content reproduction record 6R is generated (# 802) and stored in the document database 101 (# 803).

  As shown in FIG. 8, the content reproduction record 6R is configured by a document code, document data, a call marker, content position, confidential content data, alternative content data, a confidential flag, and the like.

  The document code and the calling marker are respectively the document code and the AR marker issued in step # 801.

  Document data is received image data of the document 50. The confidential content data is image data of the confidential content 53k. This is obtained by extracting an image of the range (the position of the confidential content 53k) indicated by the designated data 6S from the document 50.

  The substitute content data is the received image data of the substitute content 53 d. In the present example, since two image data are received, the content reproduction record 6R includes two image data. Then, the image data of each alternative content 53d is associated with the codes representing the lower browsing right such as "1A", "1B", ... in order from the top.

  The content position is the position of the AR content 53 (the confidential content 53 k or the alternative content 53 d) in the printed material 5 with reference to the calling marker 51. This is determined in the process of step # 804 described later.

  The security flag is a flag indicating whether or not the document 50 is to be treated as confidential information. "1" represents that it is handled as confidential information, and "0" represents that it is not handled as confidential information. The value of the security flag is based on the designated data 6S.

  Before or after the processing by the document registration unit 103, the print data transmission unit 105 arranges the issued AR code at a position where the content is not arranged in the document 50 as a calling marker 51, By deleting the content 53k from the document 50, a confidential document 50S is generated (# 804). Then, print data for printing the classified document 50S is sent to the image forming apparatus 3 together with a print command (# 805). The image forming apparatus 3 prints the confidential document 50S on a sheet based on the print data. Thereby, the printed matter 5 as shown in FIG. 5A is obtained.

  The print data may be transmitted to the tablet terminal 2T of the user Ua. Then, the user Ua may operate the tablet terminal device 2T to give a print command to the image forming apparatus 3.

  Before or after or in parallel with the processing of each of the document registration unit 103 and the print data transmission unit 105, the access right registration unit 104 generates an access right record 6K for each user code indicated in the designation data 6S ( # 806), stored in the access right database 102 (# 807).

  As shown in FIG. 9, the access right record 6K is configured by a document code, a user code, an access right, an expiration date, signature image data, a signature position, a change flag, and the like.

  The document code is a document code issued upon receiving image data or the like. The user code and the access right are the user code and the access right indicated in the designated data 6S.

  The change flag is a flag indicating whether or not the access right may be temporarily changed. The initial value is “0” indicating that change is prohibited.

  The expiration date, the signature image data, and the signature position are not yet included in the access right record 6K when the access right record 6K is generated. These will be sequentially described later.

  Then, the document registration unit 103 notifies the tablet device 2T of the user Ua of a message indicating that the registration process for the document 50 is completed and a message asking whether the registration of the signature 54 is necessary (# 808, # 809). .

  The signature 54 is an image for certifying the user having the upper browsing right, and is used to temporarily permit another user to browse the confidential content 53k.

  In the tablet type terminal device 2T, the message display control unit 202 displays these messages on the touch panel display 20e. In addition, the message display control unit 202 appropriately causes the touch panel display 20 e to display another message described later.

  When the user Ua wants to register the signature 54, the user Ua inputs a predetermined command. Then, processing of registration of the signature 54 is performed by the signature image upload unit 203 and the signature registration unit 106.

  The signature image upload unit 203 causes the touch panel display 20 e to display a message prompting the digital camera 20 g to capture the signature 54. Here, the user Ua shoots the signature 54. The signature 54 may be written by the user Ua on a sheet of paper with a pen, but an image of an object can be used as the signature 54. Hereinafter, the case where an image of a pen having a specific shape is used as the signature 54 will be described as an example.

  When the signature 54 is photographed, the signature image upload unit 203 displays a message prompting to designate the signature position. The “signature position” is the position of the signature 54 with respect to the calling marker 51 in the printed matter 5. How the signature location is used will be described later. In the present embodiment, any one of eight of upper, lower, right, left, upper right, lower right, upper left, and lower left can be designated as the signature position.

  When the signature position is designated, the signature image upload unit 203 transmits the image data of the signature 54 and the designated signature position to the document server 1. Thereby, the image data of the signature 54 is uploaded.

  In the document server 1, when the signature registration unit 106 receives the image data of the signature 54 and the designated signature position (Yes in # 810 of FIG. 12), the processing is performed according to the procedure shown in FIG.

  The signature registration unit 106 stores the signature position in the access right record 6K (see FIG. 9) indicating the user code of the user Ua and the document code of the document 50, and stores the image data as signature image data (see FIG. 10 # 821). Then, a message indicating that the registration of the signature 54 has been completed is transmitted to the tablet type terminal device 2T (# 822). The message display control unit 202 of the tablet terminal device 2T displays this message on the touch panel display 20e.

  Alternatively, registration of the signature 54 may be performed by the following method. The user Ua places the signature 54 by placing the pen at the desired signature position on the printed matter 5, as shown in FIG. Then, the printed matter 5 is photographed such that at least both of the calling marker 51 and the pen fall within the photographing range.

  Then, the signature image upload unit 203 transmits the image data of the captured image to the document server 1.

  In the document server 1, when receiving the image data (Yes in # 810 of FIG. 12), the signature registration unit 106 performs the process according to the procedure shown in FIG.

  Based on the received image data, the signature registration unit 106 detects the signature 54 (pen image) and the call marker 51 from the captured image (# 831 in FIG. 15). The position of the signature 54 with respect to the calling marker 51 is determined (# 832). This position is the signature position.

  Then, the signature registration unit 106 stores the determined signature position in the access right record 6K indicating the user code of the user Ua and the document code of the document 50, and stores the image data of the detected signature 54 as signature image data. (# 833). Thereby, registration of the signature 54 is completed. Thereafter, as in the case of FIG. 13, a message to the effect that the registration of the signature 54 is completed is transmitted to the tablet type terminal device 2T (# 834). The message display control unit 202 of the tablet terminal device 2T displays this message on the touch panel display 20e.

  It is not necessary to register the signature 54 when registering the document 50. Thereafter, it may be performed when it is necessary to allow the user who does not have the upper browsing right to browse the confidential content 53k.

  Also, registration work and processing may be performed by the personal computer or the image forming apparatus 3 instead of the tablet type terminal device 2T.

[Reading of document 50]
FIG. 16 is a flowchart for explaining an example of the flow of the content provision processing.

  Next, the operation of the user and the processing of each part when the user browses the document 50 will be described.

  The user inputs his / her user code and logs in to the terminal device 2. The client program 20P is activated on the terminal device 2 and a command to switch to the browsing mode is input. Then, the digital camera 20 g or 21 g of the terminal device 2 is directed to the printing surface of the printed matter 5. Then, an image of the printing surface is taken by the digital camera 20g or 21g. In the tablet terminal device 2T, the photographed image is displayed on the touch panel display 20e.

  If the call marker 51 is included in the captured image, the content download unit 204 of the terminal device 2 detects the call marker 51. Then, the content corresponding to the calling marker 51 is downloaded as follows. The AR content 53 is requested to the document server 1. At this time, the image data of the photographed image and the user code of the user are transmitted to the document server 1.

  Then, in the document server 1, the content providing unit 107 executes processing for distributing content not printed on the printed material 5, that is, AR content 53 in the procedure shown in FIG. 16.

  The content providing unit 107 detects the call marker 51 from the captured image (# 841). The document 50 corresponding to the calling marker 51 is specified (# 842).

  If it can be specified (Yes in # 843), the content providing unit 107 checks the access right of the user (# 844). Specifically, the access right record 6K corresponding to the document code of the specified document 50 and the transmitted user code is read from the access right database 102, and the access right stored in the access right record 6K is checked.

  Then, if there is the access right (Yes in # 845), the content providing unit 107 transmits the image data of the AR content 53 to the terminal device 2 according to the access right (# 846). For example, if the access right is "2", the image data of the confidential content 53k is transmitted. Alternatively, if the access right is "1A", the image data of the first alternative content 53d is transmitted. Alternatively, if the access right is "1B", the image data of the second alternative content 53d is transmitted. Furthermore, the content position is transmitted.

  Then, in the terminal device 2, the content download unit 204 receives the image data and content position of the AR content 53 (the confidential content 53 k or the alternative content 53 d) from the document server 1.

  When the image data and the content position are received (downloaded) by the content download unit 204, the content display control unit 205 causes the touch panel display 20e or the display 21e to display the AR content 53 as follows.

  In the case of the tablet type terminal device 2T, the content display control unit 205 calls out of an image (an image of the printed matter 5 as shown in FIG. 5A) captured by the digital camera 20g and displayed by the touch panel display 20e. The position of the marker 51 is detected. Then, with reference to this position, AR content 53 represented by this image data is superimposed and displayed on this content position on the captured image.

  Thus, for example, when the entire printed matter 5 shown in FIG. 5 (A) is photographed and the user has the upper level browsing authority, as shown in FIG. 5 (B), Confidential content 53 k is displayed as AR content 53 on the image. When the user has the lower level browsing right, the alternative content 53d as shown in FIG. 10B is displayed instead of the confidential content 53k.

  On the other hand, in the case of the glasses-type terminal device 2M, the content display control unit 205 detects the position of the calling marker 51 in the image captured by the digital camera 21g. Then, the AR content 53 represented by the image data is displayed on the display 21 e so as to be superimposed on the content position on the basis of the position.

  Thereby, for example, when the user having the upper browsing right views the whole of the printed material 5 shown in FIG. 5 (A) with the glasses-type terminal 2M, the state shown in FIG. 5 (B) is substantially the same as that shown in FIG. Similarly, the printed matter 5 is visible. If the user has the lower level browsing right, the alternative content 53d as shown in FIG. 10B is displayed instead of the confidential content 53k.

[Change of access right]
FIG. 17 is a flowchart illustrating an example of the flow of the change flag update process. FIG. 18 is a flow chart for explaining an example of the flow of upper-level browsing right assignment processing.

  It may be desirable for multiple users to work while viewing the same document 50. However, if the access right differs for each user, the AR content 53 seen for each user may differ. If so, work may be hindered.

  Therefore, the user having the upper browsing right can temporarily give the upper browsing right to the user having the lower browsing right. Hereinafter, the operation of each user and the process of each part will be described by taking an example where the user Ua having the upper browsing right temporarily gives the upper browsing right to the user Ub having the lower browsing right.

  The user Ua activates the client program 20P on his / her terminal device 2 (hereinafter referred to as "terminal device 2A") and inputs a command to switch to the access right change mode. Then, a pen for signature is placed on the printed matter 5 and imaging is performed so that both the calling marker 51 and the pen fall within the imaging range. The position of the pen is a signature position registered for the document 50 in advance.

  Then, in the terminal device 2A, the temporary release request unit 206 requests the document server 1 so that the access right of the document 50 can be temporarily changed. At this time, the image data of the photographed image and the user code of the user Ua are transmitted to the document server 1.

  In the document server 1, in response to a request from the terminal device 2A, the change flag updating unit 108 prepares to temporarily give the user having the lower browsing right of the document 50 the upper browsing right according to the procedure shown in FIG. Run.

  The change flag updating unit 108 detects the signature 54 (pen image) and the calling marker 51 from the captured image based on the received image data (# 851 in FIG. 17). Then, it is checked whether the access right of the user Ua is the upper browsing right (# 852). The access right is shown in the access right record 6K (see FIG. 9) indicating the document code of the document 50 corresponding to the detected call marker 51 and the received user code.

  If the access right is the upper level browsing right (Yes in # 853), the change flag updating unit 108 determines whether the detected signature 54 matches the image represented by the signature image data of the access right record 6K. (# 854).

  If they match (Yes in # 855), the change flag updating unit 108 determines whether the position of the detected signature 54 with respect to the detected call marker 51 matches the signature position of the access right record 6K. (# 856).

  If it matches (Yes in # 857), the change flag updating unit 108 indicates this document code and updates the change flag of the access right record 6K of the user who has the lower level browsing authority to “1” (# 858). As a result, it is possible to temporarily change the access right of the document 50 of the user having the lower level browsing right.

  The process of step # 852 and the process of steps # 854 to # 856 may be performed simultaneously, or the order may be changed.

  Then, the change flag updating unit 108 notifies the terminal device 2A of a message that the access right can be temporarily changed (# 859).

  In the terminal device 2A, the message display control unit 202 displays this message on the touch panel display 20e or the display 21e.

  Here, the user Ua urges the user Ub to take a picture while keeping the pen on the printed matter 5.

  The user Ub activates the client program 20P on his / her terminal device 2 (hereinafter referred to as "terminal device 2B") and inputs a command to switch to the access right request mode. Then, the printed matter 5 is photographed so that both the calling marker 51 and the pen fall within the photographing range. Note that the user Ua may borrow a pen, place the pen at the signature position of the printed matter 5 distributed to the user Ub itself, and shoot.

  Then, in the terminal device 2B, the access right change request unit 207 requests the document server 1 to give the upper level browsing right. At this time, the image data of the photographed image and the user code of the user Ub are transmitted to the document server 1.

  In the document server 1, in response to a request from the terminal device 2B, the access right changing unit 109 executes processing for temporarily giving the upper level browsing right of the document 50 to the terminal device 2B according to the procedure shown in FIG.

  The access right changing unit 109 detects the signature 54 (pen image) and the call marker 51 in the photographed image based on the received image data (# 861). Whether the detected signature 54 matches the image represented by the signature image data of the access right record 6K of the user (in this example, the user Ua in this example) who has the upper browsing right of the document 50 corresponding to the detected call marker 51 (# 862).

  If they match (Yes in # 863), the access right changing unit 109 determines whether the position of the detected signature 54 with respect to the detected call marker 51 matches the signature position of the access right record 6K. (# 864).

  If it matches (Yes in # 865), the access right changing unit 109 checks whether the change code of the access right record 6K indicating the document code of the document 50 and the user code of the user Ub is "1". (# 866).

  In the case of “1” (Yes in # 867), the access right changing unit 109 requests the terminal device 2A to temporarily give the upper level browsing right (# 868). At this time, the user code of the user Ub is notified to the terminal device 2A.

  Then, in the terminal device 2A, the message display control unit 202 displays a message prompting to temporarily give the upper level browsing right together with the notified user code.

  Here, the user Ua inputs whether or not to give the upper level browsing right. If you want to give it, further, enter the expiration date of the upper browsing right arbitrarily.

  The change permission notification unit 208 notifies the document server 1 of the content input by the user Ua.

  In the document server 1, when the access right changing unit 109 is notified from the terminal device 2A that the access right changing unit 109 gives the upper access right (Yes in # 869), the access right of the access right record 6K is set to "upper access right". Along with updating, the notified expiration date is reflected in the access right record 6K (# 870).

  By the above processing, the upper browsing right of the document 50 is temporarily given to the user Ub.

  Thereafter, the terminal device 2B attempts to download the AR content 53, as described in the section "Viewing the Document 50". Then, since the access right given to the user Ub is the upper level browsing right, the confidential content 53k is transmitted from the document server 1 to the terminal device 2B as the AR content 53.

  Note that the access right changing unit 109 constantly monitors whether there is an access right record 6K for which the upper browsing right has expired. And when such an access right record 6K is found, it returns to the original access right.

  FIG. 19 is a flow chart for explaining an example of the overall processing flow relating to the document 50.

  Next, the overall flow of processing in the document server 1 will be described with reference to the flowchart shown in FIG.

  Each time the document server 1 receives data and the like from the terminal device 2, the document server 1 executes processing according to that as follows.

  When the image data of the document 50, the image data of the alternative content 53d, and the designation data 6S are received (# 11 in FIG. 19), the document server 1 records the content reproduction record 6R (see FIG. 8) and the document 50. A process of registering the access right record 6K (see FIG. 9) is executed (# 12). The procedure of this process is as described above in FIG.

  Alternatively, when the image data of the signature 54 or the like is received independently of the registration process of the content reproduction record 6R (Yes in # 13), the document server 1 is a terminal that is a transmission source of the image data. The image data of the signature 54 is added to the access right record 6K of the user of the device 2 (# 14). The procedure of this process is as described above in FIG. You may carry out in the procedure demonstrated in FIG.

  Alternatively, when the request for the AR content 53 is received (Yes in # 15), the document server 1 performs processing for providing the AR content 53 to the request source (# 16). The procedure of this process is as described above in FIG.

  Alternatively, if it is requested that the access right of the document 50 can be temporarily changed from the terminal device 2 of the user having the upper level browsing right (Yes in # 17), the document server 1 records the access right record of the document 50. Among 6K, the change flag of the access right record 6K of the user having the lower level browsing right is updated to "1" (# 18). The procedure of this process is as described above in FIG.

  Alternatively, if the terminal device 2 of the user who has the lower browsing right of the document 50 is requested to give the upper browsing right (Yes in # 19), the document server 1 requests the user of the document 50. A process is performed to give the upper browsing right (# 20). The procedure of this process is as described above in FIG.

  While the power is on (No in # 21), the document server 1 appropriately executes the processes of steps # 11 to # 20.

  According to this embodiment, the access right to the content in the AR can be changed more easily than in the past.

  In the present embodiment, the lower-level view is that the signature 54 is placed and photographed on a specific one of the upper, lower, right, left, upper right, lower right, upper left, and lower left of the call marker 51. It is one of the conditions for giving the upper browsing right to the user who has the right. However, the image may be taken by being placed in any one of upper right, lower right, upper left, and lower left. Alternatively, the condition may be that the subject is placed and photographed in any one of top, bottom, right, and left.

  Alternatively, photographing with the orientation of the signature 54 in a predetermined direction may be set as one of the conditions for providing the upper browsing right to the user having the lower browsing right. For example, the condition that the longitudinal direction of the pen is placed from the upper right end to the left lower end of the printed product 5 may be taken as a condition.

  In the present embodiment, a sheet is used as a medium on which the call-out marker 51 is written, but a wall, a white board, or a top of a desk may be used.

  In the present embodiment, in the terminal device 2 of the user having the lower level browsing right, the browsing mode and the access right request mode are switched by the operation of the user. However, the image data of the image taken by the digital camera 20g or the digital camera 21g and the user code of the user are transmitted to the document server 1 both in the viewing mode and in the access right request mode. Therefore, in the terminal device 2, the browsing mode may be unified. Then, in the document server 1, it may be determined whether the processing in the browsing mode should be performed or the access right request mode should be performed.

  In this case, the document server 1 checks whether the captured image includes the signature 54 based on the transmitted image data. Then, when it is included, it is determined that it is the access right request mode. However, after that, it is determined that the browsing mode is set until the expiration date is reached. If it is not included, it is determined that the browsing mode is in effect.

  When the signature 54 disappears from the image taken by the terminal device 2 of the user having the upper browsing right, the upper browsing right given to the user having the lower browsing right is invalidated regardless of whether the expiration date is reached or not. May be That is, only while the user having the upper browsing right places the pen on the printed matter 5 and browsing the document 50, the upper browsing right may be given to the user having the lower browsing right.

  If the user who has the lower browsing right has not been given the upper browsing right even if a predetermined time (for example, 3 minutes) has passed since the change flag of the user who has the lower browsing right is updated to “1” The change flag may be set back to "0".

  In the present embodiment, various conditions for temporarily giving the upper browsing right to the user are imposed, but it is not necessary to impose all the conditions.

  For example, the signature 54 may be arranged and photographed only for the user who is given the upper level browsing right. Alternatively, it may be limited to the user who gives the upper browsing right.

  Alternatively, the upper browsing right may be given temporarily to the latter on condition that the user having the upper browsing right and the user not having the upper browsing right photographed the same call marker 51 at the same time.

  Alternatively, if other conditions are satisfied, the upper browsing right may be given without asking for the permission of the user having the upper browsing right.

  In addition, the configuration of the entire or each part of the document browsing system 100, the document server 1, and the terminal device 2, the processing content, the processing order, the configuration of data, and the like can be appropriately changed according to the purpose of the present invention.

1 Document server (image providing device)
107 Content distribution unit (transmission means)
109 Access Right Changer (Access Right Granting Means)
2 terminal device 2A terminal device (first terminal device)
2B Terminal (second terminal)
50S classified document (photographed image)
51 Call marker (identifier, common image, specific image)
53d Alternative content (second registered image)
53k Confidential Content (Registered Image)
54 signatures (common image, specific image)

Claims (18)

The terminal device for displaying the registered image stored in the storage means in association with the identifier visually described on the medium on the terminal device so as to overlap the medium or the photographed image obtained by photographing the medium Providing an image providing apparatus,
Transmission means for transmitting image data of the registered image to a terminal device of a user who has an access right to the registered image;
In the first terminal device of a first user having the access right, the identifier different specific images are captured and, and, in the second terminal device of the second user does not have the access right, the An access right giving unit that gives the second user the access right when a specific image is taken at a predetermined position or posture ;
An image providing apparatus characterized by having: The terminal device for displaying the registered image stored in the storage means in association with the identifier visually described on the medium on the terminal device so as to overlap the medium or the photographed image obtained by photographing the medium Providing an image providing apparatus,
Transmission means for transmitting image data of the registered image to a terminal device of a user who has an access right to the registered image;
In the first terminal device of the first user who has the access right, the second image of the second user whose specific image different from the identifier is taken at a predetermined position or posture and which does not have the access right An access right granting means for granting the access right to the second user when the specific image is photographed in the terminal device of
An image providing apparatus characterized by having: The predetermined position is a predetermined position based on the identifier.
An image providing apparatus according to claim 1 or 2 . The access right granting means grants the access right to the second user only while the specific image is captured by the first terminal device.
The image providing apparatus according to any one of claims 1 to 3 . The access right granting means grants the access right when the second user has a second access right of a second registered image corresponding to the identifier and different from the registered image.
The image providing apparatus according to any one of claims 1 to 4 . The terminal device for displaying the registered image stored in the storage means in association with the identifier visually described on the medium on the terminal device so as to overlap the medium or the photographed image obtained by photographing the medium Providing an image providing apparatus,
Transmission means for transmitting image data of the registered image to a terminal device of a user who has an access right to the registered image;
When a common image is captured by the first terminal device of the first user who has the access right and the second terminal device of the second user who does not have the access right, the first terminal device Access right granting means for granting the access right to the second user only while the common image is being taken ;
An image providing apparatus characterized by having: The terminal device for displaying the registered image stored in the storage means in association with the identifier visually described on the medium on the terminal device so as to overlap the medium or the photographed image obtained by photographing the medium Providing an image providing apparatus,
Transmission means for transmitting image data of the registered image to a terminal device of a user who has an access right to the registered image;
A case where a common image is captured in the first terminal device of the first user who has the access right and the second terminal device of the second user who does not have the access right , corresponding to the identifier And access right granting means for granting the access right to the second user when the second user has a second access right of a second registered image different from the registered image ;
An image providing apparatus characterized by having: The common image is a specific image different from the identifier.
The image provision apparatus of Claim 6 or Claim 7 . The access right granting means grants the access right when the first user permits.
An image providing apparatus according to any one of claims 1 to 8 . The access right granting means grants the access right only during a period designated by the first user.
The image providing apparatus according to any one of claims 1 to 9 . The terminal device for displaying the registered image stored in the storage means in association with the identifier visually described on the medium on the terminal device so as to overlap the medium or the photographed image obtained by photographing the medium Providing an image providing method,
In the first terminal device of the first user who has the access right of the registered image, the second terminal device of the second user whose specific image different from the identifier is taken and which does not have the access right And granting the access right to the second user when the specific image is captured at a predetermined position or posture ,
After granting the access right, the image data of the registered image is transmitted to the second terminal device.
An image providing method characterized by The terminal device for displaying the registered image stored in the storage means in association with the identifier visually described on the medium on the terminal device so as to overlap the medium or the photographed image obtained by photographing the medium Providing an image providing method,
In the first terminal device of the first user who has the access right of the registered image, the second user whose specific image different from the identifier is taken at a predetermined position or posture and which does not have the access right In the second terminal device, when the specific image is captured, the access right is granted to the second user,
After granting the access right, the image data of the registered image is transmitted to the second terminal device.
An image providing method characterized by The terminal device for displaying the registered image stored in the storage means in association with the identifier visually described on the medium on the terminal device so as to overlap the medium or the photographed image obtained by photographing the medium Providing an image providing method,
The first terminal device of the first user who has the access right to the registered image and the second terminal device of the second user who does not have the access right, when the common image is photographed , The access right is granted to the second user only while the common image is captured by the terminal device ,
After granting the access right, the image data of the registered image is transmitted to the second terminal device.
An image providing method characterized by The terminal device for displaying the registered image stored in the storage means in association with the identifier visually described on the medium on the terminal device so as to overlap the medium or the photographed image obtained by photographing the medium Providing an image providing method,
The case where a common image is photographed in the first terminal device of the first user who has the access right of the registered image and the second terminal device of the second user who does not have the access right, and the identifier When the second user has a second access right to a second registered image corresponding to the second image and different from the registered image, the second user is given the access right to the second user,
After granting the access right, the image data of the registered image is transmitted to the second terminal device.
An image providing method characterized by The terminal device for displaying the registered image stored in the storage means in association with the identifier visually described on the medium on the terminal device so as to overlap the medium or the photographed image obtained by photographing the medium A computer program for use in providing a computer
On the computer
In the first terminal device of the first user who has the access right of the registered image, the second terminal device of the second user whose specific image different from the identifier is taken and which does not have the access right And causing the second user to execute the process of giving the access right when the specific image is captured at a predetermined position or posture .
After granting the access right, processing of transmitting image data of the registered image to the second terminal device is executed.
A computer program characterized by The terminal device for displaying the registered image stored in the storage means in association with the identifier visually described on the medium on the terminal device so as to overlap the medium or the photographed image obtained by photographing the medium A computer program for use in providing a computer
On the computer
In the first terminal device of the first user who has the access right of the registered image, the second user whose specific image different from the identifier is taken at a predetermined position or posture and which does not have the access right Causing the second user to execute the process of giving the access right to the second user, when the specific image is captured;
After granting the access right, processing of transmitting image data of the registered image to the second terminal device is executed.
A computer program characterized by The terminal device for displaying the registered image stored in the storage means in association with the identifier visually described on the medium on the terminal device so as to overlap the medium or the photographed image obtained by photographing the medium A computer program for use in providing a computer
On the computer
The first terminal device of the first user who has the access right to the registered image and the second terminal device of the second user who does not have the access right, when the common image is photographed , Only while the common image is taken by the terminal device , execute the process of giving the access right to the second user,
After granting the access right, processing of transmitting image data of the registered image to the second terminal device is executed.
A computer program characterized by The terminal device for displaying the registered image stored in the storage means in association with the identifier visually described on the medium on the terminal device so as to overlap the medium or the photographed image obtained by photographing the medium A computer program for use in providing a computer
On the computer
The case where a common image is photographed in the first terminal device of the first user who has the access right of the registered image and the second terminal device of the second user who does not have the access right, and the identifier When the second user has a second access right to a second registered image corresponding to the second image and different from the registered image, the second user is allowed to execute the process of giving the access right to the second user ,
After granting the access right, processing of transmitting image data of the registered image to the second terminal device is executed.
A computer program characterized by

Images