JP7765030B2 - Information linking device, image processing device, information linking method, image processing method, and program - Google Patents

Description

The present invention relates to an information linking device, an image processing device, an information linking method, an image processing method, and a program.

Technology has been reported that uses AR (Augmented Reality) technology to provide users with information about products they desire in stores such as supermarkets (see, for example, Patent Document 1).

Japanese Patent Application Laid-Open No. 2019-192091

However, there are issues with registering the correspondence between product information and display location for products on store shelves, due to factors such as the rapid update cycle of location information and the vast variety of products. Furthermore, these issues are not limited to products in retail stores such as supermarkets, electronics retailers, and convenience stores; they also apply to, for example, goods in warehouses, packages at shipping company terminals, and office documents.

The present invention aims to provide an information linking device for search objects that can easily link the identification information of search objects with the location information of the search objects.

In order to achieve the above object, the information linking device for search objects of the present invention includes a starting position information acquisition unit, a movement information acquisition unit, an image information acquisition unit, an identification information extraction unit, a position information extraction unit, and an output unit,
the origin position information acquisition unit acquires first origin position information;
the first origin location information includes information on a first origin location that is a origin of location information of a first user;
The movement information acquisition unit acquires first user movement information,
the first user movement information includes information on a movement amount of the first user from the first origin position;
the image information acquisition unit acquires image information of a search target;
the identification information extraction unit extracts identification information of the search object from image information of the search object;
the location information extraction unit extracts location information of the first user when image information of the search target object is acquired based on the first origin location information and the first user movement information;
The output unit is a device that links the identification information of the search object with the location information of the first user when image information of the search object was acquired, and outputs the linked information as location information of the search object.

The image processing device of the present invention includes a search object information linking unit, an imaging information acquisition unit, an object position information acquisition unit, a virtual information generation unit, a map information management unit, a terminal information acquisition unit, an identification unit, and an image information output unit,
the search object information linking unit is the search object information linking device of the present invention,
the imaging information acquisition unit acquires a captured image captured in a specific area by a second user, the captured image including second origin position information;
the second origin location information includes information on a second origin location that is a origin of location information of a second user;
the object position information acquisition unit acquires the position information of the search object output by the search object information linking unit,
the virtual information generation unit generates a virtual image relating to the specific area;
the map information management unit manages location information of the search object by linking it to a map of the specific area;
the terminal information acquisition unit acquires location information and orientation information of the terminal based on the second origin location information;
the image information output unit outputs a composite image in which the virtual image is superimposed on the captured image;
the virtual image generated by the virtual information generation unit includes a virtual image for each of the search objects in the specific area,
the virtual image for each of the search objects includes a position image of the search object, and the position images are linked to each other for each of the search objects;
The image information output unit is a device that outputs to the terminal an image in which the position image is superimposed at a position corresponding to the search target object present in the captured image.

The method for linking information on search objects of the present invention includes a starting position information acquisition step, a movement information acquisition step, an image information acquisition step, an identification information extraction step, a position information extraction step, and an output step,
The origin position information acquisition step acquires first origin position information,
the first origin location information includes information on a first origin location that is a origin of location information of a first user;
The movement information acquisition step acquires first user movement information,
the first user movement information includes information on a movement amount of the first user from the first origin position;
the image information acquisition step acquires image information of a search target;
the identification information extraction step extracts identification information of the search object from image information of the search object;
the position information extraction step extracts position information of the first user when image information of the search object is acquired based on the first origin position information and the first user movement information;
The output step is a method of linking identification information of the search object with location information of the first user when image information of the search object was acquired, and outputting the linked information as location information of the search object.

The image processing method of the present invention includes a search object information linking step, an image capture information acquisition step, an object position information acquisition step, a virtual information generation step, a map information management step, a terminal information acquisition step, a specification step, and an image information output step,
The information linking step of the search object is the information linking method of the search object of the present invention,
The image information acquisition step acquires a captured image captured in a specific area by a second user, the captured image including second origin position information;
the second origin location information includes information on a second origin location that is a origin of location information of a second user;
the object position information acquisition step acquires the position information of the search object output by the search object information linking step,
the virtual information generating step generates a virtual image relating to the specific area;
the map information management step links and manages position information of the search object on a map of the specific area;
the terminal information acquisition step acquires location information and orientation information of the terminal based on the second origin location information;
the image information output step outputs a composite image in which the virtual image is superimposed on the captured image,
the virtual image generated in the virtual information generating step includes a virtual image for each of the search objects in the specific area,
the virtual image for each of the search objects includes a position image of the search object, and the position images are linked to each other for each of the search objects;
The image information output step is a method of outputting to the terminal an image in which the position image is superimposed at a position corresponding to the search target in the captured image, for the search target that exists in the captured image.

A first program of the present invention is a program for causing a computer to execute a starting point position information acquisition procedure, a movement information acquisition procedure, an image information acquisition procedure, an identification information extraction procedure, a position information extraction procedure, and an output procedure,
The origin position information acquisition step includes acquiring first origin position information;
the first origin location information includes information on a first origin location that is a origin of location information of a first user;
The movement information acquisition step includes acquiring first user movement information;
the first user movement information includes information on a movement amount of the first user from the first origin position;
The image information acquisition step acquires image information of a search target,
the identification information extraction step extracts identification information of the search object from image information of the search object;
the location information extraction step extracts location information of the first user when image information of the search target object is acquired based on the first origin location information and the first user movement information;
The output step links the identification information of the search object with location information of the first user when image information of the search object was acquired, and outputs the linked information as location information of the search object.

A second program of the present invention is a program for causing a computer to execute a search object information linking procedure, an image capture information acquisition procedure, an object position information acquisition procedure, a virtual information generation procedure, a map information management procedure, a terminal information acquisition procedure, a specification procedure, and an image information output procedure,
the information linking procedure for the search object is executed by the first program of the present invention,
The image information acquisition step includes acquiring a captured image captured in a specific area by a second user and including second origin position information;
the second origin location information includes information on a second origin location that is a origin of location information of a second user;
the object position information acquisition step acquires position information of the search object output by the search object information linking step,
The virtual information generation step generates a virtual image relating to the specific area;
the map information management step includes managing location information of the object to be searched for by linking it to a map of the specific area;
the terminal information acquisition step includes acquiring location information and orientation information of the terminal based on the second origin location information;
the image information output step includes outputting a composite image in which the virtual image is superimposed on the captured image;
the virtual image generated in the virtual information generation step includes a virtual image for each of the search objects in the specific area;
the virtual image for each of the search objects includes a position image of the search object, and the position images are linked to each other for each of the search objects;
The image information output step includes outputting to the terminal an image in which the position image is superimposed at a position corresponding to the search target object present in the captured image.

According to the present invention, it is possible to easily link the identification information of a search target with the location information of the search target.

FIG. 1 is a block diagram showing an example of the configuration of an information associating device according to the first embodiment. FIG. 2 is a block diagram illustrating an example of the hardware configuration of the information associating device according to the first embodiment. FIG. 3 is a flowchart illustrating an example of processing in the information associating device according to the first embodiment. FIG. 4 is a block diagram illustrating an example of the configuration of an information associating device according to the second embodiment. FIG. 5 is a flowchart illustrating an example of processing in the information associating device according to the second embodiment. FIG. 6 is a block diagram showing an example of the configuration of an image processing system including an image processing apparatus according to the third embodiment. FIG. 7 is a block diagram showing an example of the hardware configuration of an image processing apparatus according to the third embodiment. FIG. 8 is a block diagram showing an example of the hardware configuration of the image display terminal according to the third embodiment. FIG. 9 is a flowchart showing an example of processing in the image processing apparatus according to the third embodiment. FIG. 10 is a block diagram showing an example of the configuration of an image processing system including an image processing apparatus according to the fourth embodiment. FIG. 11 is a flowchart showing an example of processing in the image processing apparatus according to the fourth embodiment. FIG. 12 is a schematic diagram showing an example of an image output from the image processing apparatus according to the fourth embodiment to a user terminal, displayed on the display of the user terminal.

In the present invention, the term "search object" is not particularly limited as long as it is possible to link location information with identification information. Specific examples of search objects include merchandise in a retail store, goods in a warehouse, goods transported by a shipping company, documents in an office, servers in a server room, library materials, etc.

In the present invention, a "first user" refers to a user who links the location information and identification information of a search object using the information linking device of the present invention. Specific examples of the first user include those who manage search objects, such as employees of a retail store, warehouse, or shipping company; server room managers; library staff; etc.

In the present invention, a "second user" refers to, for example, a user who displays an image processed by the image processing device of the present invention on a terminal. Specific examples of the second user include a customer visiting a retail store; an employee of a warehouse or shipping company; a server room manager; a user visiting a library; or anyone who accesses a search object to which information is linked.

Next, embodiments of the present invention will be described using the drawings. The present invention is not limited to the following embodiments. In each of the following drawings, identical parts are designated by the same reference numerals. Furthermore, unless otherwise specified, the descriptions of each embodiment can be used interchangeably, and the configurations of each embodiment can be combined unless otherwise specified.

[Embodiment 1]
Fig. 1 is a block diagram showing an example of the configuration of an information matching device 10 according to this embodiment. As shown in Fig. 1, the device 10 includes a starting point position information acquisition unit 11, a movement information acquisition unit 12, an image information acquisition unit 13, an identification information extraction unit 14, a position information extraction unit 15, and an output unit 16. The device 10 may also include a storage unit (not shown).

The device 10 may be, for example, a single device including each of the above-mentioned components, or a device in which each of the above-mentioned components can be connected via a communication network. Furthermore, the device 10 can be connected to an external device (described later) via the communication network. The communication network is not particularly limited and any known network can be used, and may be wired or wireless, for example. Examples of the communication network include the Internet, WWW (World Wide Web), telephone lines, LAN (Local Area Network), SAN (Storage Area Network), DTN (Delay Tolerant Networking), LPWA (Low Power Wide Area), and L5G (Local 5G). Examples of wireless communication include Wi-Fi (registered trademark), Bluetooth (registered trademark), local 5G, and LPWA. The wireless communication may be in the form of direct communication between devices (ad hoc communication), infrastructure communication, or indirect communication via an access point. The device 10 may be incorporated into a server as a system. The device 10 may also be a personal computer (e.g., desktop or laptop PC), smartphone, tablet, or other device with the program of the present invention installed. The device 10 may be, for example, a terminal-type device that performs all processing within the device owned by the first user, or a server-type device that acquires and processes information acquired by the device owned by the first user. Furthermore, the device 10 may be, for example, a cloud computing or edge computing system in which at least one of the components is located on a server and the other components are located on the terminal.

Figure 2 shows an example block diagram of the hardware configuration of the device 10. The device 10 includes, for example, a central processing unit (CPU, GPU, etc.) 101, memory 102, bus 103, storage device 104, input device 105, display device 106, and communication device 107. Each part of the device 10 is connected to each other via the bus 103 and their respective interfaces (I/F).

The central processing unit 101 is responsible for overall control of the device 10. In the device 10, the central processing unit 101 executes, for example, the program of the present invention and other programs, and also reads and writes various types of information. Specifically, the central processing unit 101 functions as, for example, a starting point position information acquisition unit 11, a movement information acquisition unit 12, an image information acquisition unit 13, an identification information extraction unit 14, a position information extraction unit 15, and an output unit 16.

The bus 103 can also be connected to, for example, external devices. Examples of such external devices include external storage devices (external databases, etc.), printers, external input devices, external display devices, and external imaging devices. The device 10 can be connected to an external network (the communication line network) via, for example, a communication device 107 connected to the bus 103, and can also be connected to other devices via the external network.

The memory 102 may be, for example, a main memory (primary storage device). When the central processing unit 101 performs processing, the memory 102 reads various operating programs, such as the program of the present invention, stored in the storage device 104 (described below), and the central processing unit 101 receives data from the memory 102 and executes the programs. The main memory may be, for example, a RAM (random access memory). The memory 102 may also be, for example, a ROM (read-only memory).

The storage device 104 is also referred to as an auxiliary storage device, in contrast to the main memory (primary storage device). As mentioned above, the storage device 104 stores operating programs, including the program of the present invention. The storage device 104 may be, for example, a combination of a recording medium and a drive that reads and writes from and to the recording medium. The recording medium is not particularly limited and may be internal or external, and examples include HD (hard disk), CD-ROM, CD-R, CD-RW, MO, DVD, flash memory, and memory card. The storage device 104 may be, for example, a hard disk drive (HDD) or solid state drive (SSD) that integrates the recording medium and drive. When the device 10 includes the storage unit, the storage device 104 functions as the storage unit, for example.

In the device 10, the memory 102 and storage device 104 can also store various information such as log information, information obtained from an external database (not shown) or external device, information generated by the device 10, and information used by the device 10 when executing processing. Note that at least some of the information may be stored, for example, on an external server other than the memory 102 and storage device 104, or may be stored in a distributed manner across multiple terminals using blockchain technology, etc.

The device 10 further includes, for example, an input device 105 and a display 106. Examples of the input device 105 include pointing devices such as a touch panel, trackpad, or mouse; a keyboard; imaging means such as a camera or scanner; a card reader such as an IC card reader or magnetic card reader; and audio input means such as a microphone. The display 106 includes, for example, a display device such as an LED display or LCD display. In the first embodiment, the input device 105 and the display 106 are configured separately, but the input device 105 and the display 106 may also be configured as an integrated unit, such as a touch panel display. Furthermore, the device 10 may include, for example, an imaging device such as a camera as the input device 105, and the central processing unit 101 may control the imaging device to acquire images. In this case, for example, the imaging device functions as the image acquisition unit 13.

Next, an example of the image processing method of this embodiment will be described based on the flowchart in Figure 3. The image processing method of this embodiment is implemented, for example, using the information linking device 10 of Figure 1 or Figure 2 as follows. Note that the image processing method of this embodiment is not limited to use with the information linking device 10 of Figure 1 or Figure 2. Furthermore, in the following example, an example will be described in which this device 10 acquires information from a terminal held by a first user via a communications network and performs processing based on the acquired information. However, the present invention is not limited to this. For example, the terminal held by the first user may be this device 10, and this terminal (this device 10) may process information acquired directly. The terminal is the first user's terminal and is not particularly limited, and may be, for example, a personal computer, smartphone, mobile phone, or wearable terminal such as eyeglasses or wristwatch.

First, the origin position information acquisition unit 11 acquires first origin position information (S1, origin position information acquisition step). The first origin position information includes information on the first origin position, which is the origin of the first user position information. The first origin position information is, for example, information that identifies the location and orientation (direction) of the first user's terminal using a location code or direction mark placed within a specific area. The location code may also serve as the direction mark, for example, and the location code and direction mark are also referred to as an origin marker image. The origin marker image is, for example, a symbol image that can identify the origin position. Specific examples include a QR code (registered trademark), an AR marker, a barcode, and a chameleon code (registered trademark). In this case, for example, an origin marker image is captured by an image acquisition unit of the terminal held by the first user, the image information acquisition unit 13 acquires the origin marker image from the terminal, and the origin position information acquisition unit 11 acquires the first origin position information from the origin marker image.

Next, the movement information acquisition unit 12 acquires first user movement information (S2, movement information acquisition step). The first user movement information includes information on the amount of movement of the first user from the first starting position. The information on the amount of movement of the first user can be calculated using information acquired by a position sensor, such as a gyro sensor, acceleration sensor, or geomagnetic sensor, on the first user's terminal.

Next, the image information acquisition unit 13 acquires image information of the search object (S3, image information acquisition step). Specifically, an image of the search object is captured by the image acquisition means of the first user's terminal, and the image information acquisition unit 13 acquires the captured image of the search object.

Next, the identification information extraction unit 14 extracts identification information for the search object from the image information for the search object (S4, identification information extraction step). The identification information is not particularly limited as long as it is information that can identify the search object, and examples include product identification codes such as JAN code, EAN code, UPC code, ITF (Interleaved Two of Five), CODE39, Code128, and NW-7; image information for the product packaging; etc.

Next, the location information extraction unit 15 extracts the location information of the first user when the image information of the search object was acquired based on the first origin location information and the first user movement information (S5, location information extraction step). Specifically, for example, the location information extraction unit 15 can extract the location information of the first user by calculating, based on the first user movement information, the location at which the terminal acquired the first origin location image, i.e., the amount of movement from the first origin location to when the image information of the search object was acquired.

Then, the output unit 16 links the identification information of the search object with the location information of the first user at the time the image information of the search object was acquired, and outputs this as location information of the search object (S6, output step), thereby ending the process. For example, if the device 10 includes the storage unit, the location information of the search object can be stored in the storage unit. Alternatively, for example, the output unit 16 may output the location information of the search object to an external database via the communication network, and the external database may store the location information of the search object.

In the information linking method of this embodiment, steps S4 and S5 are performed in this order, but the information linking method of the present invention is not limited to this order; steps S4 and S5 may be performed in the order of, for example, S5 and S4, or steps S4 and S5 may be performed in parallel.

The information linking device 10 of this embodiment can, for example, link the location information of the first user when image information of the search object was acquired with identification information of the search object based on the first origin location information and the first user movement information, and output the location information of the search object. Therefore, the information linking device 10 of this embodiment can easily link, for example, the identification information of the search object with the location information of the search object.

[Embodiment 2]
This embodiment is similar to the information linking device 10 of embodiment 1, except that it further includes an attribute information acquisition unit in addition to the configuration of the information linking device 10 of embodiment 1, and the description therefor can be used. The information linking device of this embodiment further includes, for example, an attribute information acquisition unit, which acquires attribute information of the search object based on identification information of the search object, and the output unit links the attribute information of the search object to location information of the search object and outputs the linked attribute information. Therefore, according to this embodiment, in addition to the identification information of the search object, the attribute information of the search object can be linked. Therefore, according to the information linking device of this embodiment, more detailed information about the search object can be linked to the location information of the search object.

Figure 4 is a block diagram showing an example configuration of an information matching device 10A according to this embodiment. As shown in Figure 4, the information matching device 10A further includes an attribute information acquisition unit 17 in addition to the configuration of the report matching device 10 according to embodiment 1. The hardware configuration of the information matching device 10A is the same as that of the report matching device 10 in Figure 2, except that the central processing unit 101 includes the configuration of the information matching device 10A in Figure 4 instead of the configuration of the report matching device 10 in Figure 1.

An example of processing in a support system including the information matching device 10A and the support terminal 2 of this embodiment will be described based on the flowchart in Figure 5. Figure 5 is a flowchart showing an example of processing (S1 to S5, S11, S12) of the information matching device 10A.

First, steps S1 to S5 are carried out in the same manner as steps S1 to S5 in the processing of the information linking device 10 in embodiment 1.

Next, the attribute information acquisition unit 17 acquires attribute information of the search object based on the identification information of the search object (S17, attribute information acquisition step). The attribute information is not particularly limited, and may include, for example, detailed information about the item to which the search object belongs. Specific examples of the attribute information include, for example, if the search object is a supermarket product, information such as production date, expiration date, best-before date, ingredients, and price.

Then, the output unit 16 links the attribute information to the location information of the search target and outputs it (S12, output step), and the process ends.

In the information linking method of this embodiment, S11 is performed after S5, but the information linking method of the present invention is not limited to this order, and S11 may be performed, for example, downstream of S4 and upstream of S12. Therefore, in the information linking method of this embodiment, S11 and S5 may be performed in the order of S5 and S11, or S11 and S5 may be performed in parallel.

[Embodiment 3]
This embodiment is an example of an image processing system including an image processing device of the present invention and an image display terminal.

The image processing system of this embodiment will be described using FIG. 6. FIG. 6 is a block diagram showing an example configuration of an image processing system 200 including an image processing device 20 and an image display terminal 30 of this embodiment. Although not shown, the image processing system 200 may include, for example, multiple image display terminals 30. As shown in FIG. 6, the image processing device 20 can be connected to the image display terminal 30 via a communication network 40. The image processing system 200 may also include, for example, the first user's terminal. In this case, the image processing device 20 can be connected to the first user's terminal via, for example, the communication network 40. Although not shown, the image processing device 20 can also be connected to an external terminal of a system administrator via, for example, the communication network 40, and the system administrator may manage the image processing device 20 from the external terminal.

The image processing device of this embodiment will be described using FIG. 6. As shown in FIG. 6, this device 20 includes a search object information linking unit 10, an image information acquisition unit 21, an object position information acquisition unit 22, a virtual information generation unit 23, a map information management unit 24, a terminal information acquisition unit 25, and an image information output unit 26. Although not shown, this device 20 may also include, for example, a memory unit.

The device 20 may be, for example, a single device including the above-mentioned components, or a device in which the above-mentioned components can be connected via a communication network. The device 20 can also be connected to an external device (described later) via the communication network 40. The communication network 40 is not particularly limited and can be any known network, and may be wired or wireless, for example. Examples of the communication network 40 include the Internet, the World Wide Web (WWW), a telephone line, a LAN (Local Area Network), a SAN (Storage Area Network), a DTN (Delay Tolerant Networking), a LPWA (Low Power Wide Area), and a L5G (Local 5G). Examples of wireless communication include Wi-Fi (registered trademark), Bluetooth (registered trademark), local 5G, and LPWA. The wireless communication may be a form in which each device communicates directly (ad hoc communication), infrastructure communication, indirect communication via an access point, or the like. The device 20 may be incorporated into a server as a system. The device 20 may also be, for example, a personal computer (PC, e.g., desktop or laptop) on which the program of the present invention is installed, a smartphone, a tablet terminal, or the like. Furthermore, the device 20 may be, for example, in the form of cloud computing or edge computing, in which at least one of the components is located on a server and the other components are located on a terminal.

Examples of the image display terminal 30 include PCs; mobile devices such as mobile phones, smartphones, and tablet devices; smart watches, smart glasses, and wearable devices. The image display terminal 30 may also include imaging means such as a camera or scanner, and audio input means such as an IC (integrated circuit) card reader or microphone. The image display terminal 30 is, for example, the terminal of the second user.

Figure 7 shows an example block diagram of the hardware configuration of device 20. Device 20 includes, for example, a central processing unit (CPU, GPU, etc.) 201, memory 202, bus 203, storage device 204, input device 205, display device 206, and communication device 207. Each component of device 20 is interconnected via bus 203 and its respective interface (I/F).

The central processing unit 201 is responsible for overall control of the device 20. In the device 20, the central processing unit 201 executes, for example, the program of the present invention and other programs, and also reads and writes various types of information. Specifically, the central processing unit 201 functions as, for example, a search object information linking unit 10, an image information acquisition unit 21, an object position information acquisition unit 22, a virtual information generation unit 23, a map information management unit 24, a terminal information acquisition unit 25, and an image information output unit 26.

The bus 203 can also be connected to, for example, external devices. Examples of such external devices include external storage devices (external databases, etc.), printers, external input devices, external display devices, and external imaging devices. The device 20 can be connected to an external network (the communication line network) via, for example, a communication device 207 connected to the bus 203, and can also be connected to other devices via the external network.

The memory 202 may be, for example, a main memory (primary storage device). When the central processing unit 201 performs processing, the memory 202 reads various operating programs, such as the program of the present invention, stored in the storage device 204 (described below), and the central processing unit 201 receives data from the memory 202 and executes the programs. The main memory may be, for example, a RAM (random access memory). The memory 202 may also be, for example, a ROM (read-only memory).

The storage device 204 is also referred to as an auxiliary storage device, in contrast to the main memory (primary storage device). As mentioned above, the storage device 204 stores operating programs, including the program of the present invention. The storage device 204 may be, for example, a combination of a recording medium and a drive that reads and writes from and to the recording medium. The recording medium is not particularly limited and may be internal or external, and examples include HD (hard disk), CD-ROM, CD-R, CD-RW, MO, DVD, flash memory, and memory card. The storage device 204 may be, for example, a hard disk drive (HDD) or solid state drive (SSD) that integrates the recording medium and drive. If the device 20 includes, for example, the storage unit, the storage device 204 functions as the storage unit.

In the device 20, the memory 202 and storage device 204 can also store various information such as log information, information obtained from an external database (not shown) or external device, information generated by the device 20, and information used by the device 20 when executing processing. Note that at least some of the information may be stored, for example, on an external server other than the memory 202 and storage device 204, or may be stored in a distributed manner across multiple terminals using blockchain technology, etc.

The device 20 further includes, for example, an input device 205 and a display 206. Examples of the input device 205 include pointing devices such as a touch panel, trackpad, or mouse; a keyboard; imaging means such as a camera or scanner; card readers such as IC card readers or magnetic card readers; and audio input means such as a microphone. Examples of the display 206 include display devices such as an LED display or LCD display. In this embodiment 1, the input device 205 and display 206 are configured separately, but the input device 205 and display 206 may also be configured as an integrated unit, such as a touch panel display.

Figure 8 shows an example block diagram of the hardware configuration of the image display terminal 30. As shown in Figure 8, the image display terminal 30 includes, for example, a CPU 301, memory 302, a bus 303, a storage device 304, an input device (input section) 305, an imaging section 31, a communication device (communication section) 32, and a display (display section) 33. Each section of the image display terminal 30 is connected via the bus 303 and its respective interface (I/F). The description of each section of the image display terminal 30 can be made using the description of each section of the image processing device 20.

Next, an example of the image processing method of this embodiment will be described based on the flowchart in Figure 9. The image processing method of this embodiment is implemented as follows, for example, using the image processing system device 200 of Figure 6. Note that the image processing method of this embodiment is not limited to use with the image processing system device 200 of Figure 6. Figure 9 is a flowchart showing an example of the processing (S21 to S27) of the image processing system device 200.

First, the search object information linking unit 10 links the identification information of the search object with the location information of the search object (S21, search object information linking step). S21 can be implemented, for example, by performing S1 to S6, or S1 to S5, S11, and S12, in the same manner as S1 to S6 in embodiment 1 or S1 to S5, S11, and S12 in embodiment 2.

Next, the imaging information acquisition unit 21 acquires an image captured in a specific area, the image including second origin location information (S22, imaging information acquisition process). The specific area is not particularly limited and may be indoors or outdoors. Examples of indoor locations include retail stores such as supermarkets and electronics retailers; warehouses; shipping company terminals; server rooms; offices; libraries; etc. The captured image may be, for example, a still image or a video. The second origin location information includes information on the second origin location that serves as the origin of the second user's location information. The second origin location information is similar to, for example, the first origin location information, except that it is information on the second origin location that serves as the origin of the second user's location information, and the description therefor can be used. The captured image can be acquired, for example, by acquiring the captured image captured by the imaging unit 31 of the second user's terminal (image display terminal 30) via the communication network 40.

Next, the object location information acquisition unit 22 acquires the location information of the search object output by the search object information linking unit 10 (S23, object location information acquisition step).

Next, the virtual information generation unit 23 generates a virtual image related to the specific area (S24, virtual information generation step). The generated virtual image includes, for example, a virtual image of the search object in the specific area. The virtual image of the search object is, for example, a virtual image in which the location information of the search object and the location image of the search object are linked to each other. Generating the virtual image may mean, for example, reading out the virtual image stored in advance in memory 202 or storage device 204, or may mean obtaining the virtual image from an external database via the communication network. Furthermore, the virtual information generation unit 23 may, for example, obtain information related to the specific area (also referred to as specific area information) and generate the virtual image based on the specific area information. The virtual image of the search object is also referred to as, for example, a "pin" or "pin image" of the search object.

Next, the map information management unit 24 links and manages the location information of the search object on a map of the specific area (S25, map information management step). The map may be, for example, a map that represents the specific area in two dimensions, or a map that represents the specific area in three dimensions.

Next, the terminal information acquisition unit 25 acquires the terminal's position information and orientation information based on the second origin position information (S26, terminal information acquisition step). The terminal information acquisition unit 26 may, for example, acquire the terminal's position information and orientation information in association with the captured image. The orientation information may also be, for example, the imaging direction of the imaging device of the terminal.

Then, the image information output unit 26 outputs a composite image in which the virtual image is superimposed on the captured image (S27, image information output step), and the process ends (END). The output may be performed, for example, via the communication network. Specifically, the image information output unit 26 superimposes the position image on the search object present in the captured image at a position corresponding to the search object in the captured image.

Furthermore, the virtual image for each object generated by the virtual information generation unit 23 may further include a detailed information image of the search object. The detailed information image includes, for example, attribute information of the search object. In this case, the position image of the search object and the detailed information image are linked to each other for each search object. Then, for example, in the step (S27), the image information output unit 26 further provides a window section within the captured image and superimposes the detailed information image linked to the position image of the search object present in the captured image on the window section. The window section can, for example, display multiple detailed information images on a screen that allows transitions between them. The detailed information image is, for example, an image with a hierarchical structure of one or more layers. If the detailed information image is an image with a hierarchical structure of one or more layers, the window section can, for example, display the detailed information image on a screen that allows transitions between images for each layer. Specifically, the detailed information image is, for example, an image based on detailed information about the search object (for example, attribute information about the search object), and one detailed information image may be linked to one position image of the object, or two detailed information images may be linked to one position image of the object.

According to this embodiment, a virtual image of the search object can be displayed using the location information of the search object linked by the information linking device of the present invention.

[Embodiment 4]
This embodiment is similar to the image display system device 200 of Embodiment 3, except that the image processing device 20 further includes a selection information acquisition unit and the image display terminal 30 further includes an input unit, and the description thereof can be used. The image processing device of this embodiment includes, for example, a selection information acquisition unit, which acquires selection information for selecting the search objects, and the image information output unit selects the search objects present in the captured image based on the selection information and outputs an image with the position image superimposed at a position corresponding to the selected search object to the terminal. Furthermore, the image display terminal of this embodiment includes, for example, an input unit, which is capable of inputting the selection information, and the communication unit is capable of transmitting the selection information to the image processing device of the present invention and receiving an image with the position image superimposed at a position corresponding to the selected search object, and the display unit displays the image with the position image superimposed at a position corresponding to the selected search object. Therefore, according to this embodiment, search objects can be selected based on the selection information, and an image can be output in which the position image is superimposed at a position corresponding to the selected search object. Therefore, according to this embodiment, for example, the second user can select only any search object on the virtual image, so that the desired search object can be easily found. The image processing system device of this embodiment can also be called, for example, an object search system device.

Figure 10 is a block diagram showing an example configuration of an image processing system device 200A including an image processing device 20A and an image display terminal 30A of this embodiment. As shown in Figure 10, the image processing device 20A further includes a selection information acquisition unit 27 in addition to the configuration of the image processing device 20 of embodiment 3. The hardware configuration of the image processing device 20A is similar to that of the image processing device 20 of Figure 7, except that the central processing unit 201 has the configuration of the image processing device 20A of Figure 10 instead of the configuration of the image processing device 20 of Figure 6. Also, as shown in Figure 10, the image display terminal 30A further includes an input unit 34 in addition to the configuration of the image display terminal 30 of embodiment 3. The hardware configuration of the image display terminal 30A is similar to that of the image display terminal 30 of Figure 8, except that the input device 305 functions as the input unit 34.

Next, an example of the image processing method of this embodiment will be described based on the flowchart in Figure 11. The image processing method of this embodiment is implemented as follows, for example, using the image processing system device 200A of Figure 10. Note that the image processing method of this embodiment is not limited to use with the image processing system device 200A of Figure 10. Figure 11 is a flowchart showing an example of the processing (S21 to S27, S31) of the image processing system device 200A.

First, steps S21 to S26 are performed in the same manner as steps S21 to S26 in the processing of the image processing system apparatus 200 in embodiment 3.

Next, for example, the second user inputs selection information using the input unit 34 of the image display terminal 30A, and the communication unit 32 of the image display terminal 30A transmits the selection information to the image processing system device 200A via the communication network 40. The selection information is, for example, information for selecting the search object, and any information can be set. A specific example of the selection information is, for example, attribute information of the search object. Specifically, for example, if the search object is a supermarket product, information such as the production date, expiration date, best-before date, ingredients, and price can be input as the selection information. Specific examples of the selection information include information on ingredients of the search objects (e.g., grains and sesame such as peanuts, wheat, buckwheat, soybeans, sesame, barley, etc.; meat, eggs, and seafood such as eggs, milk, beef, pork, chicken, salmon, mackerel, shrimp, crab, squid, octopus, fish eggs, abalone, salmon roe, gelatin, etc.); and price information (e.g., ¥0-100, ¥101-500, ¥500 and above). The selection information may be, for example, information for excluding search objects that include the attribute information from display, or information for displaying search objects that include the attribute information. Furthermore, the selection information may include a plurality of pieces of information. In this case, for example, selection information that compiles multiple attribute information, such as excluding foods containing specific allergens, selecting only Halal or vegan products, or selecting only products in a specific price range, may be displayed on the display unit 33 of the image display terminal 30A, and the second user may select these to input the selection information.

Then, the selection information acquisition unit 27 acquires the selection information (S31, selection information acquisition step).

Next, the image information output unit 26 selects the search objects present in the captured image based on the selection information, and outputs to the terminal an image in which the position image is superimposed at a position corresponding to the selected search object (S27, image information output step), thereby ending processing by the image processing device 200A (END).

Then, the communication unit 32 of the image display terminal 30A receives an image with the position image superimposed at a position corresponding to the selected search object, and the display unit 33 displays the image with the position image superimposed at a position corresponding to the selected search object. Figure 12 shows an example of an image displayed on the image display terminal 30A.

As shown in FIG. 12, the display unit 33 of the image display terminal 30A displays a virtual image in which a position image 331 is superimposed at a position corresponding to a search object (supermarket product) included in an image captured of the interior of a supermarket as a specific area. Furthermore, when a position image 331 is selected, a detailed information image about the selected product is superimposed and displayed in a window portion 332 displayed on the display unit 33. By inputting the selection information, the second user can display on the display unit 33 only position images 331 corresponding to search objects having the desired attributes, thereby easily finding search objects having the desired attributes.

[Embodiment 5]
The first program of this embodiment is a program for causing a computer to execute each step of the information linking method described above. Specifically, the program of this embodiment is a program for causing a computer to execute a starting point location information acquisition procedure, a movement information acquisition procedure, an image information acquisition procedure, an identification information extraction procedure, a location information extraction procedure, and an output procedure.

The origin position information acquisition step includes acquiring first origin position information;
the first origin location information includes information on a first origin location that is a origin of location information of a first user;
The movement information acquisition step includes acquiring first user movement information;
the first user movement information includes information on a movement amount of the first user from the first origin position;
The image information acquisition step acquires image information of a search target,
the identification information extraction step extracts identification information of the search object from image information of the search object;
the location information extraction step extracts location information of the first user when image information of the search target object is acquired based on the first origin location information and the first user movement information;
The output step links the identification information of the search object with location information of the first user when image information of the search object was acquired, and outputs the linked information as location information of the search object.

The program of this embodiment can also be said to be a program that causes a computer to function as a starting point location information acquisition procedure, a movement information acquisition procedure, an image information acquisition procedure, an identification information extraction procedure, a location information extraction procedure, and an output procedure.

The program of this embodiment can use the descriptions of the information linking device and information linking method of the present invention. For example, the "procedure" in each of the steps can be read as "processing." The program of this embodiment may also be recorded on a computer-readable recording medium. The recording medium is, for example, a non-transitory computer-readable storage medium. The recording medium is not particularly limited, and examples include random access memory (RAM), read-only memory (ROM), hard disk (HD), optical disk, and floppy disk (FD).

[Embodiment 6]
The second program of this embodiment is a program for causing a computer to execute each step of the image processing method described above. Specifically, the program of this embodiment causes a computer to execute a search object information linking procedure, an image information acquisition procedure, an object position information acquisition procedure, a virtual information generation procedure, a map information management procedure, a terminal information acquisition procedure, an identification procedure, and an image information output procedure.

the information linking procedure for the search object is executed by the first program of the present invention,
The image information acquisition step includes acquiring a captured image captured in a specific area by a second user and including second origin position information;
the second origin location information includes information on a second origin location that is a origin of location information of a second user;
The virtual information generation step generates a virtual image relating to the specific area;
the object position information acquisition step acquires position information of the search object output by the search object information linking step,
the map information management step includes managing location information of the object to be searched for by linking it to a map of the specific area;
the terminal information acquisition step includes acquiring location information and orientation information of the terminal based on the second origin location information;
the image information output step includes outputting a composite image in which the virtual image is superimposed on the captured image;
the virtual image generated in the virtual information generation step includes a virtual image for each of the search objects in the specific area;
the virtual image for each of the search objects includes a position image of the search object, and the position images are linked to each other for each of the search objects;
The image information output step includes outputting to the terminal an image in which the position image is superimposed at a position corresponding to the search target object present in the captured image.

The second program of this embodiment can also be described as a program that causes a computer to function as a search object information linking procedure, an image information acquisition procedure, an object location information acquisition procedure, a virtual information generation procedure, a map information management procedure, a terminal information acquisition procedure, an identification procedure, and an image information output procedure.

The program of this embodiment can be implemented by reference to the descriptions of the image processing device and image processing method of the present invention. For example, the "procedure" in each of the steps can be read as "processing." The program of this embodiment may also be recorded on a computer-readable recording medium. The recording medium is, for example, a non-transitory computer-readable storage medium. The recording medium is not particularly limited, and examples include random access memory (RAM), read-only memory (ROM), hard disk (HD), optical disk, and floppy disk (FD).

The present invention has been described above with reference to embodiments, but the present invention is not limited to the above embodiments. Various modifications that would be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

<Additional Notes>
Some or all of the above embodiments can be described as, but not limited to, the following supplementary notes.
(Appendix 1)
The system includes a starting point position information acquisition unit, a movement information acquisition unit, an image information acquisition unit, an identification information extraction unit, a position information extraction unit, and an output unit,
the origin position information acquisition unit acquires first origin position information;
the first origin location information includes information on a first origin location that is a origin of location information of a first user;
The movement information acquisition unit acquires first user movement information,
the first user movement information includes information on a movement amount of the first user from the first origin position;
the image information acquisition unit acquires image information of a search target;
the identification information extraction unit extracts identification information of the search object from image information of the search object;
the location information extraction unit extracts location information of the first user when image information of the search target object is acquired based on the first origin location information and the first user movement information;
The output unit links the identification information of the search object with the location information of the first user when image information of the search object was acquired, and outputs it as location information of the search object.
(Appendix 2)
the image information acquisition unit acquires an origin marker image;
2. The information linking device according to claim 1, wherein the origin position information acquisition unit acquires the first origin position information from the origin marker image.
(Appendix 3)
An attribute information acquisition unit is included,
the attribute information acquisition unit acquires attribute information of the search object based on identification information of the search object;
3. The information linking device according to claim 1, wherein the output unit links attribute information of the search object to the location information of the search object and outputs the linked information.
(Appendix 4)
The system includes a search object information linking unit, an imaging information acquisition unit, an object position information acquisition unit, a virtual information generation unit, a map information management unit, a terminal information acquisition unit, an identification unit, and an image information output unit,
the information linking unit for the search object is the information linking device for the search object described in any one of Supplementary Notes 1 to 3,
the imaging information acquisition unit acquires a captured image captured in a specific area by a second user, the captured image including second origin position information;
the second origin location information includes information on a second origin location that is a origin of location information of a second user;
the object position information acquisition unit acquires the position information of the search object output by the search object information linking unit,
the virtual information generation unit generates a virtual image relating to the specific area;
the map information management unit manages location information of the search object by linking it to a map of the specific area;
the terminal information acquisition unit acquires location information and orientation information of the terminal based on the second origin location information;
the image information output unit outputs a composite image in which the virtual image is superimposed on the captured image;
the virtual image generated by the virtual information generation unit includes a virtual image for each of the search objects in the specific area,
the virtual image for each of the search objects includes a position image of the search object, and the position images are linked to each other for each of the search objects;
The image information output unit outputs to the terminal an image in which the position image is superimposed at a position corresponding to the search target object present in the captured image.
(Appendix 5)
the virtual image for each of the search objects further includes a detailed information image for the search object, and the position image and the detailed information image are linked to each other for each of the search objects;
The image processing device described in Appendix 4, wherein the image information output unit provides a window portion within the captured image and superimposes the detailed information image linked to the position image of the search object present in the captured image on the window portion.
(Appendix 6)
A selection information acquisition unit is provided,
the selection information acquisition unit acquires selection information for selecting the search objects;
The image processing device according to claim 4 or 5, wherein the image information output unit selects the search object present in the captured image based on the selection information, and outputs to the terminal an image in which the position image is superimposed at a position corresponding to the selected search object.
(Appendix 7)
It includes an imaging unit, a communication unit, and a display unit,
the imaging unit captures an image including starting point position information in a specific area;
the communication unit is capable of communicating with the image processing device according to any one of Supplementary Notes 4 to 6, capable of transmitting the captured image to the image processing device, and capable of receiving an image in which the position image is superimposed at a position corresponding to the search target object in the captured image;
The display unit displays an image on which the position image is superimposed.
(Appendix 8)
an input section,
the input unit is capable of inputting the selection information,
the communication unit is capable of transmitting the selection information to the image processing device described in Supplementary Note 6, and is capable of receiving an image in which the position image is superimposed at a position corresponding to the selected search object;
The image display terminal according to claim 7, wherein the display unit displays an image in which the position image is superimposed at a position corresponding to the selected search object.
(Appendix 9)
A server and a terminal,
the server and the terminal are capable of communicating with each other,
the server is an image processing device according to any one of Supplementary Notes 4 to 6,
9. An image processing system apparatus, wherein the terminal is an image display terminal according to claim 7 or 8.
(Appendix 10)
The method includes a starting point position information acquisition step, a movement information acquisition step, an image information acquisition step, an identification information extraction step, a position information extraction step, and an output step,
The origin position information acquisition step acquires first origin position information,
the first origin location information includes information on a first origin location that is a origin of location information of a first user;
The movement information acquisition step acquires first user movement information,
the first user movement information includes information on a movement amount of the first user from the first origin position;
the image information acquisition step acquires image information of a search target;
the identification information extraction step extracts identification information of the search object from image information of the search object;
the position information extraction step extracts position information of the first user when image information of the search object is acquired based on the first origin position information and the first user movement information;
The output process links the identification information of the search object with the location information of the first user when the image information of the search object was acquired, and outputs the linked information as the location information of the search object.
(Appendix 11)
The image information acquiring step includes acquiring an origin marker image,
11. The information linking method according to claim 10, wherein the origin position information acquiring step acquires the first origin position information from the origin marker image.
(Appendix 12)
An attribute information acquisition step is included,
the attribute information acquisition step acquires attribute information of the search object based on identification information of the search object;
12. The information linking method according to claim 10, wherein the output step links attribute information of the search object to the location information of the search object and outputs the linked information.
(Appendix 13)
The method includes a search object information linking step, an imaging information acquisition step, an object position information acquisition step, a virtual information generation step, a map information management step, a terminal information acquisition step, a specification step, and an image information output step,
The information linking step of the search object is a method for linking information of a search object according to any one of Supplementary Notes 10 to 12,
The image information acquisition step acquires a captured image captured in a specific area by a second user, the captured image including second origin position information;
the second origin location information includes information on a second origin location that is a origin of location information of a second user;
the object position information acquisition step acquires the position information of the search object output by the search object information linking step,
the virtual information generating step generates a virtual image relating to the specific area;
the map information management step links and manages position information of the search object on a map of the specific area;
the terminal information acquisition step acquires location information and orientation information of the terminal based on the second origin location information;
the image information output step outputs a composite image in which the virtual image is superimposed on the captured image,
the virtual image generated in the virtual information generating step includes a virtual image for each of the search objects in the specific area,
the virtual image for each of the search objects includes a position image of the search object, and the position images are linked to each other for each of the search objects;
The image processing method includes outputting to the terminal an image in which the position image is superimposed at a position corresponding to the search target in the captured image, for the search target present in the captured image, in the image information output step.
(Appendix 14)
the virtual image for each of the search objects further includes a detailed information image for the search object, and the position image and the detailed information image are linked to each other for each of the search objects;
The image processing method described in Appendix 13, wherein the image information output process provides a window process within the captured image and superimposes the detailed information image linked to a position image of the search object present in the captured image onto the window process.
(Appendix 15)
A selection information acquisition step is provided,
the selection information acquisition step acquires selection information for selecting the search objects;
The image processing method according to claim 13 or 14, wherein the image information output step selects the search object present in the captured image based on the selection information, and outputs to the terminal an image in which the position image is superimposed at a position corresponding to the selected search object.
(Appendix 16)
A program for causing a computer to execute a starting point position information acquisition procedure, a movement information acquisition procedure, an image information acquisition procedure, an identification information extraction procedure, a position information extraction procedure, and an output procedure,
The origin position information acquisition step includes acquiring first origin position information;
the first origin location information includes information on a first origin location that is a origin of location information of a first user;
The movement information acquisition step includes acquiring first user movement information;
the first user movement information includes information on a movement amount of the first user from the first origin position;
The image information acquisition step acquires image information of a search target,
the identification information extraction step extracts identification information of the search object from image information of the search object;
the location information extraction step extracts location information of the first user when image information of the search target object is acquired based on the first origin location information and the first user movement information;
The program is characterized in that the output procedure links identification information of the search object with location information of the first user when image information of the search object was acquired, and outputs the linked information as location information of the search object.
(Appendix 17)
The image information acquisition step includes acquiring an origin marker image;
17. The program according to claim 16, wherein the step of acquiring the origin position information acquires the first origin position information from the origin marker image.
(Appendix 18)
A program for causing a computer to further execute an attribute information acquisition procedure,
the attribute information acquisition step acquires attribute information of the search object based on identification information of the search object;
18. The program according to claim 16, wherein the output step links attribute information of the search object to location information of the search object and outputs the linked information.
(Appendix 19)
A program for causing a computer to execute a search object information linking procedure, an image capture information acquisition procedure, an object position information acquisition procedure, a virtual information generation procedure, a map information management procedure, a terminal information acquisition procedure, a specification procedure, and an image information output procedure,
The information linking procedure for the search object is executed by a program described in any one of Supplementary Notes 16 to 17,
The image information acquisition step includes acquiring a captured image captured in a specific area by a second user and including second origin position information;
the second origin location information includes information on a second origin location that is a origin of location information of a second user;
the object position information acquisition step acquires position information of the search object output by the search object information linking step,
The virtual information generation step generates a virtual image relating to the specific area;
the map information management step includes managing location information of the object to be searched for by linking it to a map of the specific area;
the terminal information acquisition step includes acquiring location information and orientation information of the terminal based on the second origin location information;
the image information output step includes outputting a composite image in which the virtual image is superimposed on the captured image;
the virtual image generated in the virtual information generation step includes a virtual image for each of the search objects in the specific area;
the virtual image for each of the search objects includes a position image of the search object, and the position images are linked to each other for each of the search objects;
The image information output step is a program that outputs to the terminal an image in which the position image is superimposed at a position corresponding to the search target in the captured image, for the search target that exists in the captured image.
(Appendix 20)
the virtual image for each of the search objects further includes a detailed information image for the search object, and the position image and the detailed information image are linked to each other for each of the search objects;
The program described in Appendix 19, wherein the image information output procedure provides a window procedure within the captured image and superimposes the detailed information image linked to a position image of the search object present in the captured image onto the window procedure.
(Appendix 21)
A program for causing a computer to further execute a selection information acquisition procedure,
the selection information acquisition step acquires selection information for selecting the search objects;
The program described in Appendix 19 or 20, wherein the image information output procedure selects the search object present in the captured image based on the selection information, and outputs to the terminal an image in which the position image is superimposed at a position corresponding to the selected search object.
(Appendix 22)
A computer-readable recording medium having recorded thereon a program according to any one of appendices 16 to 21.

The present invention makes it possible to easily link the identification information of a search object with the location information of the search object. Therefore, the present invention makes it possible to easily build a database that associates the type of search object with location information, allowing for efficient searches for search objects. This makes the invention particularly useful, for example, in various fields where the management of items is required.

10, 10A Information matching device, information matching unit 11 Starting point position information acquisition unit 12 Mobile image acquisition unit 13 Image information acquisition unit 14 Identification information extraction unit 15 Position information extraction unit 16 Output unit 17 Attribute information extraction unit 101 Central processing unit 102 Memory 103 Bus 104 Storage device 105 Input device 106 Display device 107 Communication device 200, 200A Image processing system device 20, 20A Image processing device 21 Imaging information acquisition unit 22 Object position information acquisition unit 23 Virtual information generation unit 24 Map information management unit 25 Terminal information acquisition unit 26 Image information output unit 27 Selection information acquisition unit 201 Central processing unit 202 Memory 203 Bus 204 Storage device 205 Input device 206 Display device 207 Communication device 30, 30A Image display terminal 31 Imaging unit 32 Communication unit (communication device)
33 Display unit (display)
34 Input section 301 Central processing unit 302 Memory 303 Bus 304 Storage device 305 Input section

Claims (9)

The system includes a starting point position information acquisition unit, a movement information acquisition unit, an image information acquisition unit, an identification information extraction unit, a position information extraction unit, and an output unit,
the image information acquisition unit acquires an origin marker image;
the origin position information acquisition unit acquires first origin position information from the origin marker image;
the first origin location information includes information on a first origin location that is a origin of location information of a first user;
The movement information acquisition unit acquires first user movement information,
the first user movement information includes information on a movement amount of the first user from the first origin position;
the image information acquisition unit acquires image information of a search target;
the identification information extraction unit extracts identification information of the search object from image information of the search object;
the location information extraction unit extracts location information of the first user by calculating a movement amount of the first user up to a point when image information of the search target object was acquired, based on the first origin location information and the first user movement information;
The output unit links the identification information of the search object with the location information of the first user when image information of the search object was acquired, and outputs it as location information of the search object. An attribute information acquisition unit is included,
the attribute information acquisition unit acquires attribute information of the search object based on identification information of the search object;
The information linking device according to claim 1 , wherein the output unit links attribute information of the search object to the location information of the search object and outputs the linked information. The system includes a search object information linking unit, an imaging information acquisition unit, an object position information acquisition unit, a virtual information generation unit, a map information management unit, a terminal information acquisition unit, an identification unit, and an image information output unit,
The information linking unit for the search object is an information linking device for the search object according to claim 1 or 2 ,
the imaging information acquisition unit acquires a captured image captured in a specific area by a second user, the captured image including second origin position information;
the second origin location information includes information on a second origin location that is a origin of location information of a second user;
the object position information acquisition unit acquires the position information of the search object output by the search object information linking unit,
the virtual information generation unit generates a virtual image relating to the specific area;
the map information management unit manages location information of the search object by linking it to a map of the specific area;
the terminal information acquisition unit acquires position information of the terminal of the second user and orientation information of the terminal of the second user based on the second origin position information;
the image information output unit outputs a composite image in which the virtual image is superimposed on the captured image;
the virtual image generated by the virtual information generation unit includes a virtual image for each of the search objects in the specific area,
the virtual image for each of the search objects includes a position image of the search object, and the position images are linked to each other for each of the search objects;
The image information output unit outputs to the second user's terminal an image in which the position image is superimposed at a position corresponding to the search target object present in the captured image. the virtual image for each of the search objects further includes a detailed information image for the search object, and the position image and the detailed information image are linked to each other for each of the search objects;
The image processing device according to claim 3 , wherein the image information output unit provides a window portion within the captured image, and superimposes the detailed information image linked to a position image of the search object present in the captured image on the window portion. A selection information acquisition unit is provided,
the selection information acquisition unit acquires selection information for selecting the search objects;
5. The image processing device according to claim 3, wherein the image information output unit selects the search object present in the captured image based on the selection information, and outputs an image in which the position image is superimposed at a position corresponding to the selected search object to the terminal of the second user. The method includes a first image information acquisition step, a starting point position information acquisition step, a movement information acquisition step, a second image information acquisition step, an identification information extraction step, a position information extraction step, and an output step,
The first image information acquisition step includes acquiring an origin marker image;
the origin position information acquiring step acquires first origin position information from the origin marker image;
the first origin location information includes information on a first origin location that is a origin of location information of a first user;
The movement information acquisition step acquires first user movement information,
the first user movement information includes information on a movement amount of the first user from the first origin position;
the second image information acquisition step acquires image information of a search target object;
the identification information extraction step extracts identification information of the search object from image information of the search object;
The position information extraction step extracts the position information of the first user by calculating a movement amount of the first user up to a time when image information of the search target object was acquired based on the first origin position information and the first user movement information;
The output step links the identification information of the search object with location information of the first user at the time when image information of the search object was acquired, and outputs the linked information as location information of the search object.
A method for linking information on search objects, comprising: The method includes a search object information linking step, an imaging information acquisition step, an object position information acquisition step, a virtual information generation step, a map information management step, a terminal information acquisition step, a specification step, and an image information output step,
The information linking step of the search object is a method for linking information of the search object according to claim 6 ,
The image information acquisition step acquires a captured image captured in a specific area by a second user, the captured image including second origin position information;
the second origin location information includes information on a second origin location that is a origin of location information of a second user;
the object position information acquisition step acquires the position information of the search object output by the search object information linking step,
the virtual information generating step generates a virtual image relating to the specific area;
the map information management step links and manages position information of the search object on a map of the specific area;
the terminal information acquisition step acquires position information of the terminal of the second user and orientation information of the terminal of the second user based on the second origin position information;
the image information output step outputs a composite image in which the virtual image is superimposed on the captured image,
the virtual image generated in the virtual information generating step includes a virtual image for each of the search objects in the specific area,
the virtual image for each of the search objects includes a position image of the search object, and the position images are linked to each other for each of the search objects;
The image information output process is an image processing method in which an image in which the position image is superimposed at a position corresponding to the search target in the captured image is output to the second user's terminal. A program for causing a computer to execute a first image information acquisition procedure, a starting point position information acquisition procedure, a movement information acquisition procedure, a second image information acquisition procedure, an identification information extraction procedure, a position information extraction procedure, and an output procedure,
The first image information acquisition step includes acquiring an origin marker image;
the origin position information acquisition step includes acquiring first origin position information from the origin marker image;
the first origin location information includes information on a first origin location that is a origin of location information of a first user;
The movement information acquisition step includes acquiring first user movement information;
the first user movement information includes information on a movement amount of the first user from the first origin position;
the second image information acquisition step acquires image information of a search target;
the identification information extraction step extracts identification information of the search object from image information of the search object;
the location information extraction step extracts location information of the first user by calculating a movement amount of the first user up to a time when image information of the search target object was acquired based on the first origin location information and the first user movement information;
The program is characterized in that the output procedure links identification information of the search object with location information of the first user when image information of the search object was acquired, and outputs the linked information as location information of the search object. A program for causing a computer to execute a search object information linking procedure, an image capture information acquisition procedure, an object position information acquisition procedure, a virtual information generation procedure, a map information management procedure, a terminal information acquisition procedure, a specification procedure, and an image information output procedure,
The information linking procedure for the search object is executed by a program according to claim 8 ,
The image information acquisition step includes acquiring a captured image captured in a specific area by a second user and including second origin position information;
the second origin location information includes information on a second origin location that is a origin of location information of a second user;
the object position information acquisition step acquires position information of the search object output by the search object information linking step,
The virtual information generation step generates a virtual image relating to the specific area;
the map information management step includes managing location information of the object to be searched for by linking it to a map of the specific area;
the terminal information acquisition step includes acquiring position information of the terminal of the second user and orientation information of the terminal of the second user based on the second origin position information;
the image information output step includes outputting a composite image in which the virtual image is superimposed on the captured image;
the virtual image generated in the virtual information generation step includes a virtual image for each of the search objects in the specific area;
the virtual image for each of the search objects includes a position image of the search object, and the position images are linked to each other for each of the search objects;
The image information output procedure is a program that outputs to the second user's terminal an image in which the position image is superimposed at a position corresponding to the search target in the captured image, for the search target present in the captured image.