JP5733720B2 - Information providing system and terminal device - Google Patents

Description

  The present invention relates to an information providing system, and more particularly to a technique for displaying information on an object in association with an image of the object.

  When investigating an object installed on the site or inspecting the object, it is necessary to carry a map on which the object is installed. In addition, since the information of the installed object is stored in the server of the sales office or the head office, it is possible to confirm the information of the installed object by accessing the server in the company. However, it was difficult to confirm information on the object on site.

  For this reason, for example, a power recovery induction device described in Patent Document 1 has been proposed as a technique for confirming information on distribution lines, utility poles, and the like on site. In the power recovery induction device described in Patent Document 1, the power pole is arranged on the display screen of the map display unit carried by the worker based on various information of the power pole about the distribution line being monitored. It is superimposed and displayed.

JP-A-9-261896

  However, as described above, the worker on site has to carry a heavy map, and it is difficult to confirm the position on the map. Moreover, even if the information of the object is described on the map carried by the worker, it is difficult to access the latest information on the site if the information stored in the in-house server is updated. Furthermore, in the technique described in Patent Document 1, since information on an object is displayed by being superimposed on a map, it is difficult to confirm the position on the map as with a paper map. In addition, when displaying an object superimposed on a map, positioning errors and map errors cannot be corrected, and the display of the object may be inaccurate.

  It is an object of the present invention to accurately confirm an object and its information by displaying the information of the object by superimposing it with an on-site image.

A typical example of the invention disclosed in the present application is as follows. That is, an information providing system including a terminal device and a server that provides information to the terminal device, the server being provided to the processor that executes a program, a memory that stores the program, and the terminal device A computer having a database for storing information, wherein the terminal device includes a processor that executes a program, a memory that stores the program, a camera that captures surrounding images, and a positioning that acquires position information of the terminal device. And a communication device that holds an image recognition pattern for recognizing an image included in the surrounding image , and the terminal device acquires position information of the current position and a surrounding image, and The server requests the server to provide information by transmitting the location information, and the server provides the information from the terminal device. When receiving a determined, by searching the database location information included in the received request as a key, obtains the information of the equipment installed in a predetermined range from the position indicated by the position information, the information A first distance between the acquired equipment and the position of the terminal device is calculated, information on the acquired equipment is ranked according to the calculated first distance, and information on the ranked equipment is obtained. Is transmitted to the terminal device, the terminal device holds an image recognition pattern for recognizing an image included in the peripheral image, and the image is displayed on the peripheral image according to the ranking given by the server. in the distance as is superimposed on the front close, the display information of the facilities, based on the acquired equipment information, calculating temporary position for displaying the equipment on the image around the said By comparing the image included in the side image with the image recognition pattern, one or more images determined to be predetermined equipment are identified from the surrounding images, and the one or more identified images are identified. A second distance between a position and the calculated temporary position is calculated, the identified image having the shortest calculated second distance is selected, and the position of the selected image and the temporary position are selected. Based on the difference from the position, an error in displaying the equipment on the surrounding image is calculated, and the calculated error is written in the memory of the terminal device .

  Further, the terminal device holds an image recognition pattern for recognizing an image included in the peripheral image, and collates the image included in the peripheral image with the image recognition pattern, thereby The display error of the equipment on the image is calculated, and the calculated error is written in the memory of the terminal device.

  According to the representative embodiment of the present invention, the facility information can be referred to using the position information as a key, so the facility information can be confirmed even outside the company. Further, since an error for correcting the position information is calculated using the image, the accuracy of the position where the equipment is displayed can be improved.

It is a block diagram of the structure of the information provision system of embodiment of this invention. It is a block diagram of the structure of the information provision server 100 of this embodiment. It is a block diagram of the structure of the terminal 200 of this embodiment. It is explanatory drawing of the equipment information database 110 of this embodiment. It is a flowchart of the menu process of this embodiment. It is a flowchart of the real mode process of this embodiment. It is a flowchart of the real mode process of this embodiment. It is a flowchart of the map mode process of this embodiment. It is a flowchart of the map mode process of this embodiment. It is a flowchart of the equipment information screen process of this embodiment. It is a flowchart of the facility information update process of this embodiment. It is a flowchart of the map information update process of this embodiment. It is explanatory drawing of the screen transition diagram of this embodiment. It is explanatory drawing of the real-time screen 1103 of this embodiment. It is explanatory drawing of the map screen 1104 of this embodiment. It is explanatory drawing of the equipment information detail screen 1105 of this embodiment. It is a sequence diagram of the real mode process of this embodiment. It is a sequence diagram of the map mode process of this embodiment. It is a flowchart of the present position correction process of this embodiment. It is explanatory drawing of the present position correction process of this embodiment. It is a flowchart of the equipment position correction process of this embodiment. It is explanatory drawing of the display position table 217 of this embodiment.

  FIG. 1 is a block diagram showing a configuration of an information providing system according to an embodiment of the present invention.

  The information providing system of this embodiment includes an information providing server 100, a terminal 200, and a network 300.

  The information providing server 100 includes a processor, a memory, a storage device, and a network interface, and is a computer installed in the company (head office or sales office). The storage device of the information providing server 100 stores an equipment information database 110 and a map information database 120. The equipment information database 110 and the map information database 120 stored in the information providing server 100 constitute a master database. The configuration of the information providing server 1 will be described later with reference to FIG.

  The terminal 200 includes a processor, a memory, and a network interface, and is a terminal device that is used by a worker to acquire facility information at a work site. The terminal 200 includes a GPS unit 206 and a camera 209. The storage device of the terminal 200 stores an equipment information database 212 and a map information database 213. The facility information database 110 and the map information database 120 stored in the information providing server 100 constitute a cache database. The configuration of the terminal 200 will be described later with reference to FIG.

  The network 300 connects the information providing server 100 and the terminal 200. Since the terminal 200 of this embodiment is used in the field, the network 300 is normally a wireless communication network, and for example, a mobile phone network, a Wi-Fi network, a WiMAX network, or the like can be used.

  The existing system 400 includes a processor, a memory, a storage device, and a network interface, and is a computer installed in the company (head office or sales office), and provides information to the information providing server 100. The storage device of the existing system 400 stores an equipment information database 410 and a map information database 420. The existing database is constituted by the facility information database 410 and the map information database 420 stored in the existing system 400.

  FIG. 2 is a block diagram showing the configuration of the information providing server 100 of this embodiment.

  The information providing server 100 is a computer that includes a processor 101, a RAM 102, a storage device 103, and a network interface 104.

  The processor 101 controls the operation of the information providing server 100 by executing a program stored in the RAM 102. The RAM 102 stores a program executed by the processor 101 and data necessary for executing the program. The RAM 102 includes a non-volatile storage area configured by a flash memory or the like, and the non-volatile storage area stores fixed data (for example, BIOS) necessary for controlling the operation of the information providing server 100. Is done. The storage device 103 is a nonvolatile storage device such as a magnetic disk drive or a flash memory (SSD), and stores the facility information database 110 and the map information database 120. The network interface 104 controls a protocol for communicating with other devices via the network 300.

  FIG. 3 is a block diagram illustrating a configuration of the terminal 200 according to the present embodiment.

  The terminal 200 is a communication terminal including a processor 201, a memory 210, a display 202, an operation unit 203, a transceiver 204, an antenna 205, a GPS unit 206, a magnetic sensor 207, an acceleration sensor 208, and a camera 209. Smartphones, netbook computers, etc.

  The processor 201 controls the operation of the terminal 200 by executing a program stored in the memory 210. The memory 210 stores a program executed by the processor 201. The memory 210 includes a non-volatile storage area configured by a flash memory or the like, and a cache database 211, selected equipment information 214, position correction data 215, and a setting information file 216 are stored in the non-volatile storage area. The cache database 211 includes an equipment information database 212 and a map information database 213.

  When the facility icon is selected on the real-time screen (1103 in FIG. 11) or the map screen (1104 in FIG. 11), the currently-selected facility information 214 stores the facility ID of the selected facility and displays it on another screen. It is used to take over and display the information of the selected equipment even after the transition.

  The position correction data 215 stores the difference between the current position calculated by the current position correction process (FIG. 17) and the position on the map (difference between latitude and longitude). The recognition pattern 216 is used for recognizing an equipment image included in a peripheral image in the equipment position correction process (FIG. 18), and stores, for example, a feature amount of the recognition image. The display position table 217 stores the correction amount of the position of the facility icon calculated by the facility position correction process (FIG. 18).

  The display 202 is a display screen for presenting information from the terminal 200 to the user, and is configured by, for example, a liquid crystal display panel. The operation unit 203 is a human interface that allows a user to give an operation instruction to the terminal 200. For example, even a software keyboard configured by a touch panel provided to be superimposed on the display 202 is provided separately from the display 202. Hardware operation keys may be used.

  Note that the terminal 200 according to the present embodiment preferably includes a [menu] key, a [home] key, a [return] key, and a [search] key as hardware operation keys. Note that some or all of these keys may be realized by a software keyboard.

  The [Menu] key is a key operated to call a function menu of the terminal 200. The [Home] key is a key operated to call a home screen (1101 in FIG. 11) of the terminal 200. The [Return] key is a key operated to return to the transition source screen. The [Search] key is a key operated when searching for programs and data in the terminal 200.

  The wireless transceiver 204 is a network interface for connecting to the network 300 via a wireless communication line, and may be connected to a public wireless communication network such as a mobile phone network, a wireless LAN, or WiMAX, for example. The antenna 205 is connected to the radio transceiver 204 and transmits / receives radio signals to / from radio base stations in the network 300.

  The GPS unit 206 receives a signal from a GPS satellite and calculates the position (latitude, longitude, altitude) of the terminal 200. In the present embodiment, the position of terminal 200 is measured using GPS, but terminal 200 is used using other methods (for example, ID of a receivable wireless LAN base station, ID of an IC tag that can be updated). May be measured.

  The magnetic sensor 207 is configured by, for example, a Hall element, and detects the orientation of the terminal 200 by detecting three orthogonal geomagnetic components. The acceleration sensor 208 detects the movement of the terminal 200 by detecting three-axis accelerations that are orthogonal to each other. Further, the acceleration sensor 208 detects the inclination of the terminal 200 from the vertical direction by detecting the components of the three-axis gravitational acceleration orthogonal to each other. The magnetic sensor 207 and the acceleration sensor 208 constitute an electronic compass.

  The camera 209 is configured by a CMOS sensor or a CCD camera, and is a device that captures a landscape image around the terminal 200 and outputs image data.

  FIG. 4 is a diagram illustrating the configuration of the facility information database 110 according to this embodiment.

  In FIG. 4, the facility information database 110 includes an item number 111, an item name 112, an indispensable 113, and a remark 114, and an entry including these items is stored for each facility.

  The item number 111 is an identifier for uniquely identifying an item included in the facility information database 110. The item name 112 indicates the type of data stored in the item. Mandatory 113 is a flag indicating whether the data of the item is mandatory for each entry. Remark 114 is the data content of the item.

  The distance data of item number 11 is referred to only when a distance data valid flag described later is set.

  Although the equipment information database 110 has been described with reference to FIG. 4, the equipment information database 212 of the cache database 211 of the terminal 200 is configured by a part of the equipment information database 110 of the master database of the information providing server 100. The databases 110 and 212 have the same configuration.

  Note that the equipment information database 110 and the map information database 120 of the master database of the information providing server 100 are updated by receiving data from the equipment information database 410 and the map information database 420 of the existing system 400. The provision of data from the existing system 400 to the information providing server 100 may be performed by a request from the information providing server 100 or by batch processing.

  FIG. 5 is a flowchart of menu processing according to the present embodiment. This menu process is executed by the processor 201 of the terminal 200.

  When activation of power equipment information processing is selected from the home screen (1101 in FIG. 11) (the user taps the icon of power equipment information), menu processing is started. First, equipment information update processing and map information update processing Is started (501, 502).

  Equipment information update processing operates in the background such as menu processing, map mode processing, real mode processing, equipment information screen processing, etc., and acquires equipment information from the information providing server 100 according to a change in the position of the terminal 200, This is a process for updating the facility information database 212 of the cache database. Details of the facility information update process will be described later with reference to FIG.

  The map information update process operates in the background such as menu process, map mode process, real mode process, facility information screen process, etc., and acquires map information from the information providing server 100 according to a change in the position of the terminal 200, This is a process for updating the map information database 213 of the cache database. Details of the map information update process will be described later with reference to FIG.

  Thereafter, a menu screen is displayed on the display 202 (503). The menu screen includes an icon for selecting the real mode or the map mode, as indicated by 1102 in FIG.

  Thereafter, when the user selects the real mode by tapping the real mode icon (Yes in 504), the real mode processing is executed (505). Details of the real mode processing will be described with reference to FIGS. 6A and 6B. On the other hand, when the user selects the map mode by tapping the map mode icon (Yes in 506), the map mode process is executed (507). Details of the map mode processing will be described with reference to FIGS. 7A and 7B.

  Further, when the user operates the [Home] key (or [Return] key) on the menu screen to select the end of the power equipment information processing (Yes in 508), the equipment information update process and the map information update process are performed. (509, 510), the menu screen is closed, and the menu process is terminated (511).

  If no operation by the user is detected (No in 508), the process returns to step 504 and waits for an operation by the user.

  6A and 6B are flowcharts of real mode processing according to the present embodiment. This real mode process is started in step 505 of the menu process and executed by the processor 201 of the terminal 200.

  When the real mode is selected on the menu screen, real mode processing is started, and a real time screen (1103 in FIG. 11) is first displayed (601).

  Thereafter, information on the current position of the terminal 200 is obtained from the GPS unit 206 (602), and information on equipment within a predetermined range is obtained from the obtained position information from the equipment information database 212 of the cache DB (603). The range for acquiring the facility information is set to a predetermined peripheral area from the peripheral image area displayed by the real mode process.

  The acquisition of the facility information may be acquired in the order from a long distance to a short distance from the current position. This is because when equipment information is displayed on the surrounding image, it is superimposed and displayed so that the closest one is in front, so if the equipment information acquired from the cache DB is arranged in order of increasing distance, the acquired equipment This is because there is no need to rearrange the information.

  Thereafter, the camera 209 is instructed to capture an image around the current position, and the photographed peripheral image is acquired from the camera 209 (604). Furthermore, the direction of the electronic compass (magnetic sensor 207, acceleration sensor 208) terminal 200 and the inclination from the vertical direction are acquired (605).

  Thereafter, a real-time screen is displayed (606). As shown in FIG. 12, the real-time screen includes an image display area 1210 and an icon display area 1220.

  In the image display area 1210, an equipment icon 1211 and an equipment outline 1212 are displayed superimposed on the surrounding image. In the icon display area 1220, an icon 1221 for transitioning to the map screen is displayed. Further, the current position 1213 and the equipment radar 1214 are displayed on the real-time screen. The current position 1213 and the equipment radar 1214 may be displayed in either the image display area 1210 or the icon display area 1220.

  The facility icon 1211 is a mark displayed at the position of the facility in the peripheral image. The equipment outline 1212 shows an outline of the equipment. In FIG. 12, the equipment ID (electric pole number) is displayed in a balloon form. By tapping the facility outline, the facility information screen process (FIG. 8) is activated, and detailed information on the facility can be viewed.

  The current position 1213 indicates the longitude and latitude of the terminal 200 acquired from the GPS unit 206 and the orientation of the terminal 200 acquired from the magnetic sensor 207. The equipment radar 1214 displays the direction of the selected equipment.

  Note that the real-time screen may include a button that can be switched between stop and restart of image update.

  Specifically, the display of the real-time screen in step 606 is performed according to the following procedure. First, a peripheral image taken by the camera 209 is drawn, and equipment icons 1211 are drawn in order of increasing distance from the current position. Further, referring to the selected equipment information 214, if the equipment information is stored in the selected equipment information 214, that is, if the equipment is selected, the equipment outline 1212 is drawn, and the selected equipment information 214 The equipment radar 1214 indicating the direction is drawn. If no equipment is selected (no equipment information is stored in the selected equipment information 214), the equipment outline 1212 may be drawn without drawing the equipment outline 1212.

  Thereafter, the facility information database 212 of the cache DB is referred to, and it is determined whether there is a facility displaying the facility overview (607). When the facility overview balloon is selected while displaying the facility overview (608). Yes), the facility information screen process (FIG. 8) is activated to display detailed information on the selected facility (609).

  On the other hand, if the equipment summary is not displayed, the equipment information read out from the equipment information database 212 of the cache DB and buffered in the RAM 210 is rearranged in the order of distance from the current position to the distance (610). This is because there is a high possibility of selecting a device that is present closer when selecting a facility by a screen tap.

  Thereafter, when the facility icon 1211 is selected (Yes in 611), the facility ID of the selected facility is written in the selected facility information 214 and a balloon 1212 of the facility overview is displayed (612).

  When the map mode icon 1221 is selected (Yes in 613), the map mode process (FIGS. 7A and 7B) is started, and the facility information is displayed on the map screen (614).

  When the [Return] key is operated (Yes in 615), the real-time screen is closed, the real mode process is terminated, and the menu process is returned (616). Instead of operating the [Return] key, a return icon may be displayed on the screen, and the real mode process may be terminated when the return icon is tapped.

  If no operation from the user is detected (No in 615), it is determined by a timer whether or not a certain time has elapsed, and after a certain time has elapsed (617), the process returns to step 602 to return to the GPS unit 206. The process is repeated from the step of acquiring the position information from. Even if the predetermined time has not elapsed, the process may return to step 602 when the terminal 200 moves a distance equal to or greater than a predetermined threshold and when an update event is received.

  7A and 7B are flowcharts of the map mode processing of the present embodiment. This map mode process is started in step 507 of the menu process and executed by the processor 201 of the terminal 200.

  When the map mode is selected on the menu screen, the map mode processing is started, and first, the map screen (1104 in FIG. 11) is displayed (701).

  Thereafter, information on the current position of the terminal 200 is obtained from the GPS unit 206 (702), and information on equipment within a predetermined range is obtained from the obtained position information from the equipment information database 212 of the cache DB (703). The range for acquiring the facility information is a predetermined area from the map area displayed by the map mode process.

  The acquisition of the facility information may be acquired in the order from a long distance to a short distance from the current position. This is because when equipment information is displayed on the surrounding image, it is superimposed and displayed so that the closest one is in front, so if the equipment information acquired from the cache DB is arranged in order of increasing distance, the acquired equipment This is because there is no need to rearrange the information.

  Thereafter, map data within a predetermined range is acquired from the acquired location information from the map database 213 of the cache DB (704). Further, the direction of the terminal 200 and the inclination from the vertical direction are acquired from the electronic compass (magnetic sensor 207, acceleration sensor 208) (705).

  Thereafter, the map screen is displayed (706). As shown in FIG. 13, the map screen includes a map display area 1310 and an icon display area 1320. In the map display area 1310, an equipment icon 1311, an equipment outline 1312, and a current position marker 1313 are displayed superimposed on the map. In the icon display area 1320, an icon 1321 for transitioning to a real-time screen and a current position correction icon 1322 are displayed. When the current position correction icon 1322 is tapped, the current position correction process (FIG. 17) is executed. Further, the current position (latitude, longitude, direction) may be displayed on the map screen, as in the real-time screen.

  The equipment icon 1311 is a mark displayed at the position of the equipment in the peripheral image. The equipment outline 1312 shows an outline of the equipment. In FIG. 13, the equipment ID (electric pole number) is displayed in a balloon form. By tapping the facility outline, the facility information screen process (FIG. 8) is activated, and detailed information on the facility can be viewed.

  Specifically, the display of the map screen in step 706 is performed according to the following procedure. First, map data acquired from the map database 213 is drawn, and an equipment icon 1311 is drawn on the drawn map. If the facility icons 1311 are drawn in order of increasing distance from the current position, the facility icons 1311 may be dense. Further, the current position marker 1313 is drawn based on the current position acquired from the GPS unit 206. Furthermore, with reference to the selected equipment information 214, if the equipment information is stored in the selected equipment information 214, that is, if the equipment is selected, the equipment outline 1312 is drawn. If no equipment is selected (no equipment information is stored in the selected equipment information 214), the equipment outline 1312 may be drawn without drawing the equipment outline 1312.

  Thereafter, the facility information database 212 of the cache DB is referenced to determine whether there is a facility displaying the facility summary (707). When the facility summary balloon is selected while the facility overview is displayed (708). Yes), the facility information screen process (FIG. 8) is activated to display detailed information on the selected facility (709).

  On the other hand, if the equipment summary is not displayed, the equipment information read out from the equipment information database 212 of the cache DB and buffered in the RAM 210 is rearranged in the order of distance from the current position to the distance (710). This is because there is a high possibility of selecting a device that is present closer when selecting a facility by a screen tap.

  After that, when the facility icon 1311 is selected (Yes in 711), the facility ID of the selected facility is written in the selected facility information 214, and a facility summary balloon 1212 is displayed (712).

  When the real mode icon 1321 is selected (Yes in 713), the real mode processing (FIGS. 6A and 6B) is started, and the facility information is displayed on the real time screen (714).

  When the [Return] key is operated (Yes in 715), the map screen is closed, the map mode process is terminated, and the menu process is returned (716). Instead of operating the [Return] key, a return icon may be displayed on the screen, and the map mode process may be terminated when the return icon is tapped.

  If no operation from the user is detected (No in 715), it is determined by a timer whether or not a certain time has elapsed, and after a certain time has passed (717), the process returns to step 702 to return to the GPS unit 206. The process is repeated from the step of acquiring the position information from. Even if the predetermined time has not elapsed, the process may return to step 702 when the terminal 200 moves a distance equal to or greater than a predetermined threshold and when an update event is received.

  FIG. 8 is a flowchart of the facility information screen process according to this embodiment. This equipment information screen process is activated in step 609 of the real mode process and step 709 of the map mode process, and is executed by the processor 201 of the terminal 200.

  When a balloon for equipment summary is selected on the real-time screen or the map screen, equipment information screen processing is started, and equipment information of the equipment whose equipment summary is being displayed is acquired from the RAM 210 (801).

  Thereafter, the equipment information detail screen (1105 in FIG. 11) is displayed, and the equipment information acquired from the RAM 210 is displayed (802).

  Thereafter, when the [Return] key is operated (Yes in 803), the facility information detail screen is closed, the facility information screen processing is terminated, and the process returns to the caller real mode processing or map mode processing (804). Instead of operating the [Return] key, a return icon may be displayed on the screen, and the facility information screen process may be terminated when the return icon is tapped.

  FIG. 9 is a flowchart of the facility information update process of the present embodiment. This facility information update process is activated in step 501 of the menu process and is executed by the processor 201 of the terminal 200.

  First, information on the current position of the terminal 200 is acquired from the GPS unit 206 (901).

  If the movement distance from the position information at the time when the facility information was updated last time is less than the predetermined reference distance (No in 902), it is not necessary to update the facility information. Information on the current position of 200 is acquired. On the other hand, if the movement distance is equal to or greater than the predetermined reference distance (Yes in 902) from the position information when the facility information was updated last time (Yes in 902), the facility information needs to be updated.

  First, it connects with the information providing server 100 (903), searches the equipment information database 110 of the master database using the obtained position information as a key, and obtains information on equipment within a predetermined range from the obtained position information. (904). At this time, when the requested information is not stored in the facility information database 110 of the master database, the information providing server 100 requests the existing system 400 for the information requested from the terminal 200. Thereafter, the connection with the information providing server 100 is disconnected (905).

  After that, the facility information acquired from the information providing server 100 is written in the facility information database 212 of the cache database (906), and the fact that the facility information has been updated is processed in real mode (FIG. 6A, FIG. 6B) and map mode processing (FIG. 7A and FIG. 7B) (907). In the real mode process and the map mode process, when the update of the facility information is notified, the updated facility information is read from the facility information database 212, and the real time screen and the map screen are updated respectively.

  Thereafter, it is determined whether or not a fixed time has passed by a timer, and after the fixed time has passed (908), it is determined whether or not to end the facility information update process (909). If the end of the facility information update process is instructed in step 509 of the menu process, the facility information update process is terminated (910). On the other hand, when the facility information update process is not terminated, the process returns to step 901, and information on the current position of the terminal 200 is acquired.

  FIG. 10 is a flowchart of the map information update process of the present embodiment. This map information update process is activated in step 502 of the menu process and executed by the processor 201 of the terminal 200.

  First, information on the current position of the terminal 200 is acquired from the GPS unit 206 (1001).

  If the movement distance from the position information when the map information was updated last time is less than the predetermined reference distance (No in 1002), there is no need to update the map information. Information on the current position of 200 is acquired. On the other hand, if the moving distance is greater than or equal to the predetermined reference distance from the position information when the map information was updated last time (Yes in 1002), the map information needs to be updated, and the process proceeds to step 1003.

  First, it connects to the information providing server 100 (1003), searches the map information database 120 of the master database using the acquired position information as a key, and acquires information on equipment within a predetermined range from the acquired position information. (1004). At this time, if the requested information is not stored in the map information database 120 of the master database, the information providing server 100 requests the existing system 400 for the information requested from the terminal 200. Thereafter, the connection with the information providing server 100 is disconnected (1005).

  Thereafter, the facility information acquired from the information providing server 100 is written in the map information database 213 of the cache database (1006), and the fact that the map information has been updated is processed in real mode processing (FIGS. 6A and 6B) and map mode processing (FIG. 7A and FIG. 7B) (1007). In the real mode process and the map mode process, when update of the map information is notified, the updated map information is read from the map information database 213, and the real time screen and the map screen are updated respectively.

  Thereafter, it is determined whether or not a certain time has passed by a timer, and after a certain time has passed (1008), it is determined whether or not to end the map information update process (1009). If the end of the map information update process is instructed in step 510 of the menu process, the map information update process ends (1010). On the other hand, if the map information update process is not terminated, the process returns to step 1001 and information on the current position of the terminal 200 is acquired.

  FIG. 11 is a diagram for explaining a screen transition diagram of the present embodiment.

  When the terminal 200 is activated, a home screen 1101 is displayed. Home screen 1101 includes an icon of power equipment information. When the user taps the icon of the power equipment information, the menu process (FIG. 5) is activated and the menu screen 1102 is displayed.

  The menu screen 1102 includes a real mode icon and a map mode icon. When the user taps the real mode icon, the real mode processing (FIG. 6A, FIG. 6B) is activated, and the real time screen 1103 is displayed. In addition, when the user taps the map mode icon, the map mode process (FIGS. 7A and 7B) is activated, and the map screen 1104 is displayed. Details of the real-time screen 1103 and the map screen 1104 are as described above with reference to FIGS.

  When a balloon of equipment outline is selected on the real-time screen 1103 and the map screen 1104, the equipment information screen process (FIG. 8) is activated and the equipment information detail screen 1105 (FIG. 14) is displayed.

  When the [Return] key is operated on each screen, the screen returns to the previous screen. When the [Return] key is operated on the menu screen, the menu screen 1102 is closed and the menu process is terminated. Note that instead of operating the [Return] key, a return icon may be displayed on the screen, and the process described above may be performed when the return icon is tapped.

  FIG. 14 is a diagram illustrating the facility information detail screen 1105 of the present embodiment.

  The equipment information detail screen 1105 includes the type of equipment (electric pole), equipment attached to the equipment (transformer, switch, etc.), equipment ID (electric pole number), serial number, manufacturer, model name, installation location (latitude, longitude) ) Etc. are displayed.

  FIG. 15 is a sequence diagram of real mode processing according to the present embodiment.

  First, when the power facility information icon is tapped on the home screen 1101, menu processing is started, the menu screen 1102 is displayed (1501), and facility information update processing is started (1502). When the real mode icon is tapped on the menu screen 1102, real mode processing is started and the real time screen 1103 is displayed (1503).

  Thereafter, the facility information update process connects to the information providing server 100 at a predetermined timing (for example, at a predetermined time interval) and calls the facility information distribution process. At this time, information on the current position of the terminal 200 is transmitted to the facility information distribution process (1504). When called from the equipment information update process of the terminal 200, the equipment information distribution process instructs the equipment information database 110 of the master database to perform a search using the position information transmitted from the terminal 200 as a key (1505).

  The equipment information database 110 extracts equipment information within a predetermined range from the position specified by the search instruction, and returns it to the equipment information distribution process (1506). At this time, if the requested information is not stored in the facility information database 110, the facility information distribution process requests the existing system 400 for the information requested from the facility information update process. In the facility information distribution process, the facility information acquired from the facility information database 110 is returned to the facility information update process of the terminal 200 (1507).

  In the facility information update process, the facility information received from the facility information distribution process is written in the facility information database 212 of the cache database (1508). After updating the facility information, the facility information database 212 returns the end of the update to the facility information update process (1509). The facility information update process notifies the real mode process of the update of the facility information database 212 (1510).

  In the real mode process, when the notification of the update of the facility information database is received, the facility information database 212 is referred to (1511), and the facility information displayed on the real time screen is received from the facility information database 212 (1512). Thereafter, the real mode process transmits an ACK message to the facility information update process (1513).

  Note that the processing of 1504 to 1513 is repeatedly executed at predetermined time intervals as described above, and is also executed when the terminal 200 moves a distance equal to or greater than a predetermined threshold or when an update event is received.

  Thereafter, when a balloon of the equipment outline is tapped on the real-time screen, the equipment information screen processing is started (1514). The facility information screen process refers to the facility information database 212 (1515), and receives provision of facility information to be displayed on the facility information detail screen from the facility information database 212 (1516). Thereafter, when the operation of the [Return] key is detected, the facility information screen processing closes the facility information detail screen, ends the facility information screen processing, returns to the real mode processing, and displays the real time screen (1517).

  When the operation of the [Return] key is detected, the real mode process closes the real time screen, ends the real mode process, returns to the menu process, and displays the menu screen (1518).

  When the menu process detects an operation of the [Home] key, the menu screen is closed and the menu process is terminated (1519).

  FIG. 16 is a sequence diagram of map mode processing according to the present embodiment.

  First, when the power facility information icon is tapped on the home screen 1101, menu processing is started, the menu screen 1102 is displayed (1601), and facility information update processing and map information update processing are started (1602). When a map mode icon is tapped on the menu screen 1102, map mode processing is started and the map screen 1103 is displayed (1603).

  Thereafter, the facility information update process connects to the information providing server 100 at a predetermined timing (for example, at a predetermined time interval) and calls the facility information distribution process. At this time, information on the current position of the terminal 200 is transmitted to the facility information distribution process (1604). When called from the equipment information update process of the terminal 200, the equipment information distribution process instructs the equipment information database 110 of the master database to perform a search using the position information transmitted from the terminal 200 as a key (1605).

  The equipment information database 110 extracts equipment information within a predetermined range from the position specified by the search instruction, and returns it to the equipment information distribution process (1606). At this time, if the requested information is not stored in the facility information database 110, the facility information distribution process requests the existing system 400 for the information requested from the facility information update process. In the facility information distribution process, the facility information acquired from the facility information database 110 is returned to the facility information update process of the terminal 200 (1607).

  In the facility information update process, the facility information received from the facility information distribution process is written into the facility information database 212 of the cache database (1608). After updating the facility information, the facility information database 212 returns the end of the update to the facility information update process (1609).

  The map information update process connects to the information providing server 100 and calls the map information distribution process. At this time, information on the current position of the terminal 200 is transmitted to the map information distribution process (1610). When the map information distribution process is called from the map information update process of the terminal 200, the map information database 120 of the master database is instructed to search using the position information transmitted from the terminal 200 as a key (1611).

  The map information database 120 extracts map information within a predetermined range from the position specified by the search instruction, and returns it to the map information distribution process (1612). At this time, if the requested information is not stored in the map information database 120, the map information distribution process requests the existing system 400 for the information requested from the map information update process. In the map information distribution process, the facility information acquired from the map information database 120 is returned to the map information update process of the terminal 200 (1613).

  In the map information update process, the map information received from the map information distribution process is written in the map information database 213 of the cache database (1614). After updating the map information, the map information database 213 returns the end of the update to the map information update process (1615). In the equipment information update process and the map information update process, the update of the equipment information database 212 and the map information database 213 is notified to the map mode process (1616). In this embodiment, after both the facility information database 212 and the map information database 213 are updated, the update of the database is notified to the map mode processing. However, one of the facility information database 212 and the map facility information database 213 After the update, the map mode process may be notified of the database update.

  In the map mode process, when a database update notification is received, the facility information database 212 is referred to (1617), and the facility information displayed on the map time screen is received from the facility information database 212 (1618). Further, the map mode process refers to the map information database 213 (1619), and receives provision of facility information to be displayed on the map time screen from the map information database 212 (1620). Thereafter, the map mode process transmits an ACK message to the facility information update process and the map information update process (1621).

  Note that the processing of 1604 to 1621 is repeatedly executed at predetermined time intervals as described above, and is also executed when the terminal 200 moves a distance equal to or greater than a predetermined threshold or when an update event is received.

  Further, the facility information database 212 and the map information database 213 may be updated at the same timing or at different timings (for example, different periods).

  Thereafter, when a balloon of the equipment outline is tapped on the map screen, the equipment information screen processing is started (1622). The facility information screen process refers to the facility information database 212 (1623), and receives provision of facility information to be displayed on the facility information detail screen from the facility information database 212 (1624). After that, when the operation of the [Return] key is detected, the facility information screen processing closes the facility information detail screen, ends the facility information screen processing, returns to the map mode processing, and displays the real time screen (1625).

  In the map mode process, when the operation of the [Return] key is detected, the map screen is closed, the map mode process is terminated, the menu process is returned, and the menu screen is displayed (1626).

  When the menu processing detects the operation of the [Home] key, the menu screen is closed and the menu processing is terminated (1627).

  FIG. 17 is a flowchart of the current position correction process of this embodiment.

  The current position correction process is started when the current position correction icon 1322 is tapped and the current position correction mode is turned on. In the current position correction process, first, the coordinates on the map screen 1104 of the current position marker 1313A are acquired (1701).

  Then, as shown in FIG. 18, after the user moves (drags) the current position marker to the true current position on the screen, the coordinates of the current position marker 1313B after the dragging are acquired (1702).

  Thereafter, the difference between the coordinates before dragging and the coordinates after dragging of the current position marker is calculated, the difference between the coordinates is converted into a difference between latitude and longitude (1703), and the calculated difference between latitude and longitude is calculated as position correction data. It is stored in 215 (1704).

  The position correction data 215 is referred to when the map screen and the real-time screen are drawn. The position information output from the GPS unit 206 is corrected only by the longitude and latitude stored in the position correction data 215, and the map and its surroundings are corrected. Draw an image.

  FIG. 19 is a flowchart of the equipment position correction process of this embodiment.

  The equipment position correction process is executed for each drawing of the real time screen in the real mode process when the equipment position correction mode is ON. Switching of the equipment position correction mode ON and OFF can be set, for example, on a setting screen (not shown).

  First, the position of the equipment (equipment icon) displayed on the real-time screen is acquired from the equipment information database 212 of the cache database (1901), and the coordinates of the center of the temporary display position on the real-time screen are calculated (1902).

  Thereafter, the peripheral image displayed on the real-time screen is analyzed to recognize an image of equipment (for example, a utility pole, a transformer, etc.) included in the peripheral image (1903). For example, the peripheral image is analyzed, an image included in the peripheral image is extracted, and a feature amount of the extracted image is calculated. Then, the calculated feature amount of the image is compared with the feature amount of the image stored in the recognition pattern 216, an image similar to the recognition pattern 216 is recognized from the peripheral image, and the recognized image A rectangle or circle circumscribing is defined, and the position is stored in the RAM 210.

  Thereafter, it is determined whether there is a candidate for the display position of the facility icon (1904).

  As a result, if there is no image recognized in step 1903, that is, the display position of the equipment icon, the display position of the equipment icon cannot be corrected. Therefore, the equipment icon is translucent to the temporary display position. It is displayed (1909). The drawing of the equipment icon at the temporary display position may be displayed in a form that can be distinguished from the equipment icon whose position is corrected, even if it is not translucent. In this way, by displaying the equipment icon that has not been corrected and the corrected equipment icon in different forms, it is possible to display whether or not correction has been made (that is, the degree of position error) in an easy-to-understand manner.

  On the other hand, if there is an image recognized in step 1903, that is, a candidate for the display position of the facility icon, the display position of the facility icon can be corrected, so that each recognized image (that is, circumscribing the image) (1905), the distance between the center coordinates of each recognized image and the center coordinates of the temporary display position is calculated (1906), and the distance between the center positions is the shortest. An image (display position candidate) is selected (1907). In this manner, by selecting the display position candidate having the shortest distance, the correct position of the equipment can be automatically determined and appropriate correction can be performed without causing the user to select the correction position.

  Then, the center coordinates of the selected display position candidate are stored in the display position table 217.

  By storing the center coordinates of the display position candidates in the display position table 217, when the display position table 219 is referred to when the real-time screen is drawn, the facility icon is displayed at the coordinates stored in the display position table 217.

  FIG. 20 is a diagram for explaining the display position table 217 of the present embodiment.

  The display position table 217 includes an identifier 2171 for uniquely identifying a display object, coordinates 2172 on the screen on which the display object is displayed, a size 2173 on which the display object is displayed on the screen, and a display form 2174. The display form 2174 is a flag that indicates whether the display object is displayed normally or translucent (that is, another display form).

  As described above, according to the embodiment of the present invention, using the AR (augmented reality) technology, the equipment database information is displayed by superimposing it on the site image, so the equipment information can be confirmed even outside the company. Can do.

  Further, since the current position is corrected on the map, the accuracy of the current position can be improved. Furthermore, since the position information is corrected using an image recognition technique, the facility information can be displayed at an accurate position.

  As described above, the embodiment of the present invention has been described with respect to the information providing system that provides information on power facilities such as utility poles. However, the information providing system of the present invention is not limited to providing information on power facilities, and is installed in the field. The present invention can also be applied to the case of providing information on the property (for example, plants such as factories and power plants, gas or water distribution facilities).

100 Information providing server 200 Terminal 300 Network 400 Existing system

Claims (6)

An information providing system comprising a terminal device and a server for providing information to the terminal device,
The server is a computer having a processor that executes a program, a memory that stores the program, and a database that stores information provided to the terminal device,
The terminal device includes a processor that executes a program, a memory that stores the program, a camera that captures a peripheral image, and a positioning device that acquires position information of the terminal device, and is included in the peripheral image. A communication device that holds an image recognition pattern for recognizing
The terminal device
Obtain location information of the current location and surrounding images,
Requesting the server to provide information by sending the location information;
The server
When the request for providing the information is received from the terminal device, the database is searched using the location information included in the received request as a key,
Obtain information on the equipment installed within a predetermined range from the position indicated by the position information,
Calculating a first distance between the facility from which the information was acquired and the position of the terminal device;
Ranking the acquired equipment information according to the calculated first distance;
Send information of the ranked equipment to the terminal device,
The terminal device
Holding an image recognition pattern for recognizing an image included in the surrounding image;
According to the ranking given by the server, the information on the equipment is displayed on the surrounding image so that the one with a short distance is superimposed on the front ,
Based on the acquired equipment information, calculate a temporary position to display the equipment on the surrounding image,
By collating the image included in the peripheral image with the image recognition pattern, one or more images determined to be predetermined equipment are identified from the peripheral image,
Calculating a second distance between the position of the one or more identified images and the calculated temporary position;
Selecting the identified image with the shortest calculated second distance;
The difference between the position of the selected image and the temporary position is used to calculate the display error of the equipment on the surrounding image,
The information providing system, wherein the calculated error is written in a memory of the terminal device .   The terminal device
  Based on the acquired equipment information, calculate a temporary position to display the equipment on the surrounding image,
  By comparing the image included in the peripheral image and the pattern for image recognition, it is determined whether the image included in the peripheral image is of a predetermined facility,
  When it is determined that the image of the predetermined facility is included in the surrounding image, the facility is displayed at the position of the image determined to be the predetermined facility,
  The information providing system according to claim 1, wherein when it is determined that the image of the predetermined facility is not included in the peripheral image, the facility is displayed at the temporary position.   The terminal device
  Based on the acquired equipment information, calculate a temporary position to display the equipment on the surrounding image,
  In the case of displaying the equipment at the position of the image determined to be the predetermined equipment by collating with the image recognition pattern, and in the case of displaying the equipment at the temporary position in different display forms The information providing system according to claim 1, wherein the facility is displayed.   A terminal device connected to a server having a database for storing information provided to the terminal device,
  A processor that executes a program; a memory that stores the program; a communication unit that communicates with the server; a camera that captures a peripheral image; and a positioning device that acquires position information of the terminal device;
  Obtain location information of the current location and surrounding images,
  Requesting the server to provide information by sending the location information;
  Information on equipment installed within a predetermined range from the position indicated by the position information included in the request, ranked according to a first distance between the position of the equipment and the position of the terminal device Obtain equipment information from the server,
  According to the ranking given by the server, the information on the equipment is displayed on the surrounding image so that the one with a short distance is superimposed on the front,
  Holding an image recognition pattern for recognizing an image included in the surrounding image;
  Based on the acquired equipment information, calculate a temporary position to display the equipment on the surrounding image,
  By collating an image included in the peripheral image and the image recognition pattern, one or more images determined to be predetermined equipment are identified from the peripheral image,
  Calculating a second distance between the position of the one or more identified images and the calculated temporary position;
  Selecting the identified image with the shortest calculated second distance;
  The difference between the position of the selected image and the temporary position is used to calculate the display error of the equipment on the surrounding image,
  The terminal device, wherein the calculated error is written in a memory of the terminal device.   The terminal device
  Based on the acquired equipment information, calculate a temporary position to display the equipment on the surrounding image,
  By comparing the image included in the peripheral image and the pattern for image recognition, it is determined whether the image included in the peripheral image is of a predetermined facility,
  When it is determined that the image of the predetermined facility is included in the surrounding image, the facility is displayed at the position of the image determined to be the predetermined facility,
  The terminal device according to claim 4, wherein when it is determined that an image of the predetermined facility is not included in the peripheral image, the facility is displayed at the temporary position.   The terminal device
  Based on the acquired equipment information, calculate a temporary position to display the equipment on the surrounding image,
  In the case of displaying the equipment at the position of the image determined to be the predetermined equipment by collating with the image recognition pattern, and in the case of displaying the equipment at the temporary position in different display forms The terminal device according to claim 4, wherein the equipment is displayed.