JP7809443B2 - Information Display System - Google Patents

Description

The present invention relates to an information providing device, an information processing device, an information display system, and a program.

In recent years, systems have been developed that use augmented reality (AR) to provide users with product-related information in the form of images and audio.

Patent Document 1 discloses an AR-based customer value-added information provision system. In this AR-based customer value-added information provision system, a user such as a product customer reads first identification information attached to the product, product packaging, or packaging box using an information terminal device, etc., and an information acquisition mechanism reads second information via the Internet, etc., and provides the product customer with additional product information such as how to assemble the product or how to use it safely as the second information.

JP 2016-12349 A

Conventionally, identification information attached to a product is read by a terminal device or the like to identify the product and determine the information to be provided to the user.
However, when using a photographing device such as a camera to read the identification information attached to the side of a can, the side of the can is curved, so unless the depth of field of the photographing device is deep, the entire identification information may not be in focus, and as a result, the identification information may not be read.
The present invention aims to provide an information processing device, etc. that can identify identification information attached to the side of a can body and provide provided information even if the depth of field of the imaging device is not deep.

Thus, according to the present invention, an information display system is provided which comprises: an imaging means for capturing an image of a can body; an information providing means for providing provided information including an image to be displayed in augmented reality together with the can body based on the captured image, which is an image of the can body captured by the imaging means; a display means for displaying a composite image obtained by combining the image included in the provided information with the captured image; and a display control means for controlling the image to be displayed on the display means, wherein the information providing means comprises: an image acquisition unit for acquiring image information which is a part of the captured image of the can body, an area focused by the imaging device that captured the can body, and which is longer in the axial direction than in the circumferential direction of the can body; an identification unit for identifying identification information which includes an image of the part of the image acquired by the image acquisition unit; and a determination unit for determining provided information to be provided to a user, which corresponds to the identification information identified by the identification unit, and wherein the identification unit identifies the identification information by template matching between the image acquired by the image acquisition unit and an image of the identification information .

Here, the identification unit can perform template matching by moving the image acquired by the image acquisition unit in the circumferential direction of the can body, rather than in the axial direction, relative to the image of the identification information, thereby reducing the processing load required for template matching.

The present invention provides an information processing device that can identify the identification information attached to the side of a can body and provide the provided information, even when the depth of field of the imaging device is not deep.

1 is a diagram illustrating an example of the configuration of an AR display system according to an embodiment of the present invention. FIG. 1 is a diagram illustrating an outline of the operation of an AR display system. 1A to 1C are explanatory views of a can body according to this embodiment. FIG. 1 is a block diagram showing an example of the functional configuration of an AR display system according to an embodiment of the present invention. 10A and 10B are diagrams showing an example of an extracted image. (A) is a diagram showing the area in focus when the side of a can body is photographed with a camera having a deep depth of field, and (B) is a diagram showing the area in focus when the side of a can body is photographed with a camera having a shallow depth of field. 10A and 10B are diagrams illustrating template matching performed in this embodiment. FIG. 10 is a diagram showing an example of an AR image displayed on a terminal device based on provided information. FIG. 10 is a diagram illustrating a management table. 10 is a flowchart illustrating the operation of the AR display system.

The following describes in detail the embodiments of the present invention. Note that the present invention is not limited to the following embodiments, and various modifications can be made within the scope of its gist. The drawings used are for the purpose of explaining the present embodiments, and do not represent the actual size.

<Explanation of the Overall AR Display System 1>
FIG. 1 is a diagram showing an example of the configuration of an AR display system 1 according to the present embodiment.
The AR display system 1 of this embodiment is an example of an information display system, and as shown in the figure, includes a terminal device 10 owned by a user and a server device 30 that provides services to the user via the terminal device 10. In the AR display system 1 of this embodiment, the terminal device 10 and the server device 30 are capable of mutual information communication via a network 40. Also shown in the figure is a can body 50, although it does not constitute the AR display system 1 of this embodiment.

Terminal device 10 is an example of an information processing device, and when a can body 50 is photographed, it displays AR based on provided information provided by server device 30. Here, "provided information" includes at least image information and is information provided to the user. This image information displays, for example, an image to be displayed in AR together with can body 50.
The terminal device 10 is, for example, a mobile terminal such as a mobile computer, a mobile phone, a smartphone, a tablet, etc. The terminal device 10 performs wireless communication and connects to a network 40.

The server device 30 is an example of an information providing device or information providing means that provides information to the user, and is a server computer that manages the entire AR display system 1. When the server device 30 is provided on the cloud, it can also be said to be a cloud server. The server device 30 creates information to be provided to the terminal device 10, including image information for displaying AR.

The terminal device 10 and the server device 30 each include a central processing unit (CPU) as a computing means, a main memory as a storage means, and storage such as an HDD or solid state drive (SSD). The CPU executes various software such as an operating system (OS) and application software. The main memory is a storage area for storing various software programs and data used to run them, while the storage is a storage area for storing input data for the various software programs and output data from the various software programs.
Furthermore, the terminal device 10 and the server device 30 are equipped with a communication interface (hereinafter referred to as "communication I/F") for communicating with the outside world, a display device consisting of a video memory, a display, etc., and input devices such as a keyboard and a mouse.
The terminal device 10 also includes a camera, which is an example of a photographing means or photographing device, for photographing an image of the can body 50, and a speaker for outputting sound such as voice.

The CPU, main memory, and storage provided in the terminal device 10 function as display control means that control the images displayed on the display device. The display device provided in the terminal device 10 also functions as display means that displays the provided information.

The network 40 is a communication means used for information communication between the terminal device 10 and the server device 30, and may be, for example, a LAN (Local Area Network), a WAN (Wide Area Network), or the Internet. The communication lines used for data communication may be wired or wireless, or a combination of these may be used. In addition, the terminal device 10 and the server device 30 may be connected via multiple networks or communication lines using relay devices such as gateway devices or routers.

The AR display system 1 of this embodiment realizes a service in which identification information (described below) provided on the surface of a can 50 filled with a beverage or the like is read by a terminal device 10, and information to be provided, including images and audio, is transmitted to the terminal device 10 in accordance with the read identification information. The AR display system 1 of this embodiment performs AR (augmented reality) display, such as superimposing a predetermined image as information to be provided on an image showing the can 50, thereby creating a new customer experience for users who purchase a can 50 filled with a beverage.

A web application (hereinafter simply referred to as the web application) for the AR (augmented reality) display system 1 is installed on the terminal device 10, for example, downloaded via a network. The web application of this embodiment operates resources of the terminal device 10, such as hardware installed in the terminal device 10 and software pre-installed on the terminal device 10, in order to provide provided information in accordance with identification information. Specifically, by operating the resources of the terminal device 10, the web application of this embodiment realizes various functions, such as displaying an image when reading an identification image, photographing and displaying on the screen an object including the can body 50, reading the identification image, transmitting information such as identification information to the server device 30, etc., receiving information such as provided information from the server device 30, etc., displaying images such as provided information on the screen, and accepting user operations and instructions via the screen.

Meanwhile, identification information and provided information are registered in association with each other in the server device 30. In this embodiment, the identification information is a predetermined image that is displayed on the surface of the can body 50, as described below. Any image can be registered as identification information. Examples of provided information include text information, music, audio, video, two-dimensional images, three-dimensional images, e-books, and links to access predetermined websites. Registration of identification information and provided information can be performed by service providers that provide services using the AR display system 1, or by beverage manufacturers that sell can bodies 50 filled with beverages, for example.

Then, a viewer based on the web application is launched on the terminal device 10, and the identification information is read. The terminal device 10 transmits the read identification information to the server device 30. The server device 30 manages the identification information and the provided information. When the server device 30 receives the identification information from the terminal device 10, it transmits the provided information corresponding to the identification information to the terminal device 10. Furthermore, the terminal device 10 displays the received provided information using a viewer.

<Outline of Operation of AR Display System 1>
FIG. 2 is a diagram showing an outline of the operation of the AR display system 1. As shown in FIG.
First, a user operating the terminal device 10 photographs the can body 50 (1A). The user photographs the can body 50 using a camera provided in the terminal device 10. The photographed image of the can body 50 is then sent to the server device 30 as photographic information of the can body 50 (1B).
The server device 30, which has received the photographic information of the can body 50, creates information to be provided to the user based on the photographed image (1C). As described above, this information to be provided includes image information for displaying AR.
The information to be provided created by the server device 30 is sent to the terminal device 10 (1D).
Then, the terminal device 10 displays the AR based on the provided information (1E).

<Can body 50>
Next, the can body 50 of this embodiment will be described in detail.
3A to 3C are explanatory diagrams of the can body 50 of this embodiment.
Fig. 3(A) is a perspective view of the can body 50 when viewed from a predetermined direction. Fig. 3(B) is a perspective view of the can body 50 when viewed from a direction different from the predetermined direction of Fig. 3(A). Fig. 3(C) is an explanatory diagram of an image printed on the can body 50.

3(A) and 3(B), the can body 50 has a cylindrical tubular portion 510, a lid portion 520 that closes an opening located at one end of the can body 50 in the axial direction, and a bottom portion 530 that is provided on the opposite side of the tubular portion 510 from the lid portion 520 in the axial direction. The can body 50 of this embodiment can be made of a metal such as aluminum. A beverage or other contents are sealed in the can body 50.
Although the tubular portion 510 in this embodiment is a circular cylinder, it may be formed in a polygonal shape.

As shown in Figure 3(C), various information is printed on the cylindrical portion 510 of the can body 50, including the beverage manufacturer's name 51m, product graphic 51g, product name 51p, alcohol mark 51a, recycle identification mark 51r, barcode 51b, can manufacturer logo 51s, product text 51t, nutritional information 51n, warning text 51c, and product description section 51e.

The beverage manufacturer name 51m is information indicating the name of the company that fills the can 50 with a beverage and sells the can 50 filled with a beverage (hereinafter referred to as the product). The product graphic 51g is information indicating the product logo. The product name 51p is information indicating the product name. The alcohol mark 51a is information indicating that the product is an alcoholic beverage. The recycle identification mark 51r is information that serves as a mark when reusing the can 50 as a resource. The barcode 51b is encoded information such as the name of the company that manufactures and sells the product, and the type of product.

The can manufacturer logo 51s is information showing the logo of the company that manufactured the can body 50 itself. The product text 51t is information showing various information about the product in text form. The nutritional information display 51n is information showing the nutritional content of the product. The warning text 51c is information showing precautions regarding handling of the product.
The product description column 51e includes text 51e1 that explains the product and explanatory figures 51e2 that graphically represent the image of the product.

In the AR display system 1 of this embodiment, of the various types of information printed on the cylindrical portion 510, both the product graphic 51g and the explanatory graphic 51e2 are set as identification information. Here, the product graphic 51g and the explanatory graphic 51e2 are provided at different, separate positions in the circumferential direction of the cylindrical portion 510.

In the AR display system 1 of this embodiment, the product graphic 51g and explanatory graphic 51e2 are set as identification information, but this is not limited to this example. Other information from the various types of information (images and text) printed on the tube portion 510 described above may also be used as identification information.

Furthermore, in the AR display system 1 of this embodiment, the images used as identification information are not code images such as barcodes that encode information, but rather images such as company logos or product logos that directly convey information to the user's visual sense. In addition, in the AR display system 1 of this embodiment, a single image provided on the can body 50 serves both the function of directly conveying information to the user's visual sense and the function of identifying information. This makes it unnecessary to provide a code image, which is likely to detract from the aesthetic appeal of the design.

<Description of the Configuration of AR Display System 1>
Next, the configuration of the AR display system 1 according to this embodiment will be specifically described.
FIG. 4 is a block diagram showing an example of the functional configuration of the AR display system 1 according to this embodiment.
Here, functions related to this embodiment are selected from the various functions of the AR display system 1 and illustrated.

The terminal device 10 of the AR display system 1 includes a transmitter/receiver unit 110 that transmits and receives information to and from external devices, a photographing unit 120 that takes photographs, a display unit 130 that displays images, an audio output unit 140 that outputs audio, and a control unit 150 that controls the entire terminal device 10.

The transmitting/receiving unit 110 transmits and receives photographing information of the can body 50 and provided information for displaying AR to and from the server device 30. The transmitting/receiving unit 110 functions as a provided information acquisition unit that acquires the provided information. The transmitting/receiving unit 110 corresponds to, for example, a communication I/F.
The photographing unit 120 photographs the can body 50. The photographing unit 120 corresponds to a camera and includes, for example, an optical system such as a lens that converges light, and an image sensor such as a CCD (Charge Coupled Device) that detects the light converged by the optical system.

The display unit 130 displays a user operation screen and AR. The display unit 130 corresponds to a display device such as a monitor. The display unit 130 may also be a touch panel that functions as both a display device and an input device.
The audio output unit 140 outputs audio accompanying the AR display. The audio output unit 140 corresponds to a speaker, for example.

The control unit 150 controls the transmission of the captured image of the can body 50 to the server device 30 via the transmission/reception unit 110. In this case, the control unit 150 functions as a captured image acquisition unit that acquires the image of the can body 50 from the imaging unit 120.
Furthermore, the control unit 150 creates an AR image to be displayed on the display unit 130. That is, the control unit 150 creates a composite image by combining image information included in the provided information sent from the server device 30 with the captured image captured by the image capturing unit 120. The control unit 150 then controls the display unit 130 to display this composite image as AR. In this case, the control unit 150 functions as a display control unit that controls the display of the provided information.

The server device 30 of the AR display system 1 includes a transmitter/receiver 310 that transmits and receives information to and from external devices, an extractor 320 that extracts a portion of the captured image of the can body 50, an identifier 330 that identifies the identification information, a determiner 340 that determines the information to be provided, and a memory unit 350 that stores the information to be provided.

The transmitter/receiver 310 transmits and receives photographic information of the can body 50 and information to be provided for displaying AR to and from the terminal device 10. The transmitter/receiver 310 corresponds to, for example, a communication I/F.

The extraction unit 320 is an example of an image acquisition unit, and extracts a portion from the captured image of the can body 50 acquired by the transmission/reception unit 310. In this case, the extraction unit 320 acquires image information of an image that is a portion of the captured image of the can body 50 and is longer in the axial direction than in the circumferential direction of the can body 50. The extraction unit 320 receives the entire image captured by the terminal device 10, extracts from it an image that is longer in the axial direction than in the circumferential direction of the can body 50, and creates image information of the extracted image, which is the cropped image, thereby achieving acquisition. Alternatively, such an extracted image may be created in advance by the terminal device 10, and then acquired by the extraction unit 320.

5A and 5B are diagrams showing an example of an extracted image.
Of these, the extracted image C1 shown in FIG. 5(A) is a portion of the photographed image G of the can body 50 that was in focus with the camera that photographed the can body 50.
That is, a portion that happens to be in focus is cut out from the image captured by the terminal device 10 and used as an extracted image. Since the can body 50 has a cylindrical shape, such a portion becomes a rectangular region in the captured image of the can body 50, with the long side extending along the axial direction of the can body 50.

Fig. 6(A) is a diagram showing the area in focus when the side of the can body 50 is photographed with a camera having a deep depth of field. Fig. 6(B) is a diagram showing the area in focus when the side of the can body 50 is photographed with a camera having a shallow depth of field. Figs. 6(A) and 6(B) are diagrams of the can body 50 as viewed from above, as viewed from the direction VI in Fig. 3.
As shown in Figure 6(A), when the side of the can body 50 is photographed with a camera having a deep depth of field, the entire area where the identification information is printed is in focus. In contrast, as shown in Figure 6(B), when the side of the can body 50 is photographed with a camera having a shallow depth of field, some of the area where the identification information is printed is in focus, but other areas are out of focus, resulting in a blurred image.

In other words, extracted image C1 is strip-shaped or rectangular. If the depth of field of the camera of terminal device 10 is not deep, as in extracted image C1 in Figure 5(A), the distance between the camera and can body 50 is approximately the same in the direction along the axial direction of the can body 50, making it easy to focus. In contrast, as you move circumferentially from the in-focus point, the distance between the camera and can body 50 changes, making it difficult to focus. As a result, the point that happens to be in focus tends to be like extracted image C1 shown in the figure, which is a rectangular area with its long sides aligned along the axial direction of the can body 50.

The extracted image C2 shown in FIG. 5B is of the central portion of the captured image of the can body 50.
In other words, some cameras of the terminal device 10 are designed to focus near the center rather than the edges. In this case, it is easy to achieve focus near the center of the captured image, but it is difficult to achieve focus away from the center. Typically, when a user photographs the can body 50, they position the can body 50 near the center. As a result, if the depth of field of the camera of the terminal device 10 is not deep, it is easy to achieve focus in the direction along the axial direction of the can body 50. In contrast, it is difficult to achieve focus as you move from the center toward the circumferential direction. As a result, the focused area is likely to be a central area in the captured image, a rectangular area with its long sides along the axial direction of the can body 50, as shown in the extracted image C2.

4 , the identification unit 330 identifies identification information including a partial image of the captured image acquired by the extraction unit 320. At this time, the identification unit 330 identifies the identification information by template matching between the image acquired by the extraction unit 320 and the image of the identification information.
That is, the identifying unit 330 searches for the identification information of the partial image acquired by the extracting unit 320 by template matching, thereby identifying the identification information photographed by the user.

7A and 7B are diagrams showing the template matching performed in this embodiment.
FIG. 7A is a diagram conceptually showing the relationship between the extracted image C and the image S of the identification information.
In FIG. 7A, the pixel values of each pixel in extraction image C are (H _C , S _C , V _C ). Furthermore, the pixel values of each pixel in identification information image S are (H _S , S _S , V _S ). Note that, here, the image information of extraction image C and identification information image S is usually composed of RGB data defined in the RGB color space, but this is converted to HSV data defined in the HSV color space. From this, the brightness components V _C and V _S are extracted, and the value of (V _C - V _S ) ² is considered. Furthermore, the difference e, defined by the following equation 1, is calculated as the sum total for all pixels in extraction image C. Note that V _C (i, j) is the brightness component value of the pixel at coordinates (i, j) in extraction image C. Furthermore, V _S (i, j) is the brightness component value of the pixel at coordinates (i, j) in identification information image S.

In the above example, RGB data is converted into HSV data, but this is not limiting. For example, the data may be converted into L ^* a ^* b ^* data, IPT data, Lαβ data, CAMO2 data, XYZ data, etc.

The identification unit 330 shifts the extracted image C left, right, up, and down within the identification information image S, and determines the case where the difference e is smallest. In this case, it can also be said that the identification unit 330 shifts the extracted image C in both the axial and circumferential directions of the can body 50 within the identification information image S, and determines the case where the difference e is smallest.

If the minimum difference e is smaller than a predetermined threshold, the identification unit 330 determines that the extracted image C is a part of the identification information image S. In other words, if the extracted image C is not a part of the identification information image S, the images do not match, so the difference e will be a large value even when it is minimum. On the other hand, if the extracted image C is a part of the identification information image S, there will be a portion where the two images almost match. The difference e will be minimum at this portion. The difference e at this time will be a small value. Therefore, if a threshold is set and the difference e is minimum, if it is greater than this threshold, it can be determined that the images do not match and the extracted image C is not a part of the identification information image S. On the other hand, if it is smaller than the threshold, the images match and it can be determined that the extracted image C is a part of the identification information image S.
Then, by performing this process sequentially on a plurality of pieces of identification information, it is possible to find matching identification information.

7(B) shows a case in which the identification unit 330 performs template matching by moving the extracted image C acquired by the extraction unit 320 in the circumferential direction, but not in the axial direction, of the can body 50 relative to the image of the identification information. In other words, in this case, the minimum difference e is determined in the same manner as in FIG. 7(A), but the extracted image C is moved left and right, but not up and down. If the user captures an image of the entire can body 50, the positions of the top and bottom ends of the can body 50, i.e., the lid portion 520 and the bottom portion 530, can be identified. Furthermore, the axial position at which the identification information is printed is predetermined. Therefore, the axial position of the identification information can be identified based on the positions of the top and bottom ends of the can body 50 in the axial direction of the can body 50. Therefore, the axial position of the extracted image C can be first determined, and then the extracted image C can be moved in the circumferential direction to perform template matching.

Note that Figure 7(B) shows a case where the up-down direction of the captured image corresponds to the axial direction of the can body 50, but depending on the orientation of the camera used by the user, the left-right direction of the captured image may also correspond to the axial direction of the can body 50. In this case, template matching is performed by moving the extracted image C in the up-down direction but not in the left-right direction. In this case, too, template matching can be said to be performed by moving the extracted image C in the circumferential direction of the can body 50, rather than in the axial direction.

Returning to FIG. 4 again, the determination unit 340 determines the provided information to be provided to the user, which is information corresponding to the identification information identified by the identification unit 330. As described above, the provided information includes image information for displaying AR. The provided information may also include audio information about audio that accompanies the display of AR. Furthermore, the provided information may include information other than the information for displaying AR.

FIG. 8 is a diagram showing an example of an AR image displayed on the terminal device 10 based on the provided information.
In this example, the control unit 150 of the terminal device 10 controls the display of a provided image for displaying an AR image in accordance with the position of a can body image 500, which is an image of a can body 50 displayed on the screen 100. At this time, the control unit 150 identifies the outline of the can body 50 displayed as the can body image 500. The control unit 150 then identifies the positions of the tubular portion 510, the lid portion 520, and the bottom portion 530 of the can body 50 in the can body image 500. The control unit 150 then displays the provided image at a predetermined position in the can body image 500.

The control unit 150 of this embodiment integrates the camera function and motion function of the terminal device 10 to perform AR display. Furthermore, the control unit 150 of this embodiment performs AR display by recognizing spatial structures on a software basis based on resources installed in the terminal device 10 and image analysis data, without using special hardware. For example, the control unit 150 can use VIO (Visual Inertial Odometry) technology, which combines the device's motion sensing (accelerometer and gyroscope) with analysis of images acquired from the terminal device 10's camera. Furthermore, the control unit 150 may use environmental mapping to reproduce a simulated reflection of the surrounding environment on the surface of a three-dimensional object.

Then, as shown in FIG. 8, the control unit 150 superimposes the character image 75 as a provided image on the lid portion 520 of the can body image 500. As a result, the control unit 150 displays an image in which the character image 75 appears to be sitting on the lid portion 520 of the can body 50. This can also be said to mean that the control unit 150 controls the display of the character so that it is positioned above the image of the can body 50. Furthermore, the control unit 150 displays the character image 75 so that it appears to be elongated in a direction perpendicular to the lid portion 520.

The storage unit 350 stores the provided information in association with the contents of the can body 50. Here, this information is stored in a management table T.
FIG. 9 is a diagram showing the management table T.
As shown in the figure, in the management table T, "identification information" and "provided information" are associated with each management number.
For example, for management number 001, two pieces of identification information are set: first identification information (001A.jpg) and second identification information (001B.jpg).
The provided information includes either provided images, provided audio, or provided images and provided audio. The provided images are composed of still images or moving images such as animation. The provided audio includes sound configured to be synchronized with the still images or moving images. In other words, in the case of moving images, the provided audio is played in accordance with the timing of the moving images.

<Explanation of Operation of AR Display System 1>
Next, the operation of the AR display system 1 will be described.
FIG. 10 is a flowchart illustrating the operation of the AR display system 1.
First, the photographing unit 120 of the terminal device 10 photographs the can body 50 in response to a user operation (step 101).
Next, the transmitting/receiving unit 110 transmits the photographic information of the can body 50 photographed by the photographing unit 120 to the server device 30, and the photographic information is received by the transmitting/receiving unit 310 of the server device 30 (step 102).

Then, the extraction unit 320 extracts a portion from the captured image of the can body 50 (step 103). As described above, the extracted image, extracted image C, is an image that is longer in the axial direction of the can body 50 than in the circumferential direction. Extracted image C is, for example, an extracted image C1 described in Figure 5(A), which is a portion of the captured image of the can body 50 that was in focus with the camera that captured the image of the can body 50. Alternatively, extracted image C is, for example, an extracted image C2 described in Figure 5(B), which is a central portion of the captured image of the can body 50.

Then, the identifying unit 330 acquires one piece of identification information from the storage unit 350 (step 104).
Next, the identifying unit 330 determines whether the extracted image C matches the image of the acquired identification information (step 105). This process can be performed by template matching, as described with reference to FIG.
If there is no match (No in step 105), the identifying unit 330 acquires the next identification information from the storage unit 350 (step 106).

On the other hand, if there is a match (Yes in step 105), the identification information has been identified. In this case, the determination unit 340 refers to the storage unit 350 and determines the information to be provided from the identified identification information (step 107).

Furthermore, the transmitting/receiving unit 310 transmits the determined information to be provided to the terminal device 10, and the transmitting/receiving unit 110 of the terminal device 10 receives it (step 108).
In the terminal device 10, the control unit 150 combines the can body 50 with the image information included in the provided information to create an AR image (step 109).
Then, the display unit 130 displays the AR image (step 110). At this time, if the provided information includes audio information, the audio output unit 140 outputs audio together with the AR image.

The AR display system 1 described above in detail makes it possible to provide an information processing device that can identify the identification information attached to the side of the can body 50 and provide the information to be provided, even when the camera's depth of field is not deep.

In the embodiment described above, the server device 30 creates the information to be provided based on the captured image, but this may be done by the terminal device 10. In this case, the functions of the above-described AR display system 1 can be provided by the terminal device 10 alone.
In the above-described embodiment, the terminal device 10 is provided with a camera, but the camera may be separate from the terminal device 10. Similarly, the terminal device 10 is provided with a display device, but the display device may be a liquid crystal display or the like and separate from the terminal device 10.

<Program Description>
In this embodiment, the processes performed by the server device 30 are realized by the cooperation of software and hardware resources. That is, a CPU (not shown) provided inside the server device 30 executes a program that realizes each function of the server device 30, thereby realizing each function.

Therefore, the processing performed by the server device 30 can also be considered as a program that causes a computer to implement an image acquisition function that acquires image information of an image that is part of a captured image of the can body 50 and is longer in the axial direction than in the circumferential direction of the can body 50, a specification function that identifies identification information that includes the image acquired by the image acquisition function, and a determination function that determines the information to be provided to the user, which is information that corresponds to the identification information identified by the specification function.

The present embodiment has been described above, but the technical scope of the present invention is not limited to the scope described in the above embodiment. It is clear from the claims that various modifications and improvements to the above embodiment are also included within the technical scope of the present invention.

1...AR display system, 10...terminal device, 30...server device, 110, 310...transmitter/receiver unit, 120...photographer unit, 130...display unit, 320...extraction unit, 330...identification unit, 340...determination unit

Claims (2)

an imaging means for capturing an image of the can body;
an information providing means for providing information including an image to be displayed in augmented reality together with the can body based on a captured image that is an image of the can body captured by the imaging means;
a display means for displaying a composite image obtained by combining the image included in the provided information with the photographed image;
a display control means for controlling an image to be displayed on the display means;
Equipped with
The information providing means
an image acquisition unit that acquires image information of a part of the captured image of the can body, the image being a region in focus of the photographing device that photographed the can body and longer in the axial direction than in the circumferential direction of the can body;
an identification unit that identifies identification information including a part of the image acquired by the image acquisition unit;
a determination unit that determines information to be provided to a user, the information corresponding to the identification information identified by the identification unit;
Equipped with
The identification unit identifies the identification information by template matching between the image acquired by the image acquisition unit and an image of the identification information . The information display system according to claim 1, characterized in that the identification unit performs template matching by moving the image acquired by the image acquisition unit in a circumferential direction of the can body rather than in an axial direction of the can body with respect to the image of the identification information.