JP7840258B2 - Display device, program, display method, and display system - Google Patents

Description

This invention relates to a display device, a program, a display method, and a display system.

Conventionally, techniques have been known for ensuring a certain distance between utility poles and power lines and surrounding buildings when installing them. For example, the distance measuring system described in Patent Document 1 generates a linear AR (Augmented Reality) object connecting a first point and a second point in a captured area within space. The distance between a third point in space and the linear AR object is then calculated, thereby determining the distance between the virtual linear object and a point in real space.

Japanese Patent Publication No. 2021-156634

The technology described in Patent Document 1 above accepts user input to adjust the sag and position of the electric wire and places an AR object at a specified location in the image captured by the camera. Therefore, the technology described in Patent Document 1 may place a significant burden on the user to accurately set the position of the AR object. Furthermore, depending on the user's skill level, it may not be possible to accurately set the position of the AR object using the technology described in Patent Document 1.

This disclosure is made in view of these circumstances and aims to provide a display device, program, display method, and display system that allow for easy confirmation of the installation status of utility poles and power lines after utility pole construction.

This disclosure was made to solve the aforementioned problems. One aspect of this disclosure is a display device comprising: an imaging unit for capturing images of the surroundings; and a display unit for displaying utility pole objects and power line objects superimposed on the captured image captured by the imaging unit. The display unit, in response to receiving instructions regarding utility pole construction, displays the utility pole objects and power line objects using height information of the power lines connected to the utility poles.

Another aspect of this disclosure is a program that causes the computer of a display device to perform the following steps: causing the imaging unit to capture images of the surroundings; superimposing utility pole objects and power line objects onto the captured images captured by the imaging unit and displaying them on the display unit; and updating the display state of the utility pole objects and power line objects using height information of the power lines connected to the utility pole in response to receiving instructions regarding utility pole construction.

Another aspect of this disclosure is a display method that includes the steps of: having an imaging unit capture images of the surroundings; superimposing utility pole objects and power line objects onto the captured images captured by the imaging unit and displaying them on a display unit; and, in response to receiving instructions regarding utility pole construction, displaying the utility pole objects and power line objects using height information of the power lines connected to the utility pole.

Another aspect of this disclosure is a display system comprising: a data management device that provides geographic information; an imaging unit that images the surroundings; a utility pole and power line recognition unit that recognizes utility pole and power line objects included in the imaged images; a geographic information acquisition unit that acquires the geographic information based on the position of the device; an association unit that associates the utility pole and power line objects recognized by the utility pole and power line recognition unit with the utility poles and power lines included in the geographic information; and a display unit that superimposes the utility pole and power line objects associated by the association unit onto the imaged images captured by the imaging unit, wherein the display unit displays the utility pole and power line objects using height information of the power lines connected to the utility poles in response to receiving instructions regarding utility pole construction.

According to one aspect of the present invention, the installation status of utility poles and power lines after utility pole construction can be easily confirmed.

This is a block diagram showing an example of a display system in an embodiment. This figure shows an example of GIS data in the embodiment. This is a block diagram showing an example of a display device in an embodiment. A flowchart illustrating an example of the operation procedure of the display device in the embodiment. This is an example of an image captured in the embodiment. This figure shows an example of a utility pole object, a power line object, and the height of the power line recognized in the captured image. This figure shows an example of how utility pole objects corresponding to utility poles and power line objects corresponding to power lines are removed from an captured image. This figure shows an example of how a background object is superimposed on an captured image. This figure shows an example of an image captured in an embodiment in which objects corresponding to utility poles and power lines have been removed in response to a request to relocate a utility pole. This figure shows an example of an image captured with objects representing the power lines connected to the original utility pole, the relocated utility pole, and the utility pole at the new location. This is a sequence diagram showing an example of the operation procedure of the display system in the embodiment. This figure shows an example of a utility pole update table included in the GIS data in the embodiment. This is an example of an image taken with equipment objects before the relocation of the utility pole. This is an example of an image captured including equipment objects after the relocation of a utility pole. This is an example of an image captured including local objects before the relocation of a utility pole. This is an example of an image captured after the removal of a feature object. This is an example of an image taken after a utility pole was relocated.

The display device, program, display method, and display system to which the present invention is applied will be described below with reference to the drawings.

(Display system configuration)
Figure 1 is a block diagram showing an example of a display system in an embodiment. The display system comprises, for example, a display device 100, a data management device 200, and a data storage device 300. The display device 100 is a terminal device operated by workers involved in construction work to install, relocate, or remove utility poles. The display device 100, the data management device 200, and the data storage device 300 have a communication interface such as a NIC (Network Interface Card) or a wireless communication module for connecting to a network NW such as the Internet. The network NW may include, for example, a general-purpose network such as the Internet, and a private network such as local 5G or Wi-Fi®.

The data management device 200 is, for example, a server device that manages GIS (Geographic Information System) data. The data management device 200 includes, for example, a data provision unit 210 and a data update unit 220. The data provision unit 210 receives requests from the display device 100 and provides the display device 100 with GIS data corresponding to the requests. The data update unit 220 receives requests from the display device 100 and updates the GIS data based on the requests. The data storage device 300 is a server device that includes a storage device for storing GIS data. The data storage device 300 may be configured as an integral part of the data management device 200 and may be connected to the data management device 200 via a network NW. The data storage device 300 provides GIS data to the data management device 200 in accordance with requests from the data management device 200. The data storage device 300 updates the GIS data in accordance with requests from the data management device 200. In this embodiment, the geographic information is GIS data, but it is not limited to this; any data that registers information on utility poles, power lines, facilities, and geographical features in association with map information is acceptable.

Figure 2 shows an example of GIS data in the embodiment. GIS data includes, for example, map information, feature information, and utility pole information. Map information includes, for example, a map information table in which information contained in a map, such as location information expressed in latitude and longitude, type information indicating the type of road, etc., shape information indicating the shape of road, etc., and height information expressed in altitude, is registered in association with each feature. Feature information includes a feature information table in which information for identifying a feature, such as location information, type information, shape information, and height information, is registered in association with each feature. Feature examples include trees and buildings standing on the ground, such as houses. Utility pole information includes a utility pole information table in which information for identifying a utility pole, such as location information expressed in latitude and longitude, shape information indicating the shape of the utility pole, utility pole connection information, power line height information, and equipment information, is registered in association with each utility pole. Utility pole connection information is, for example, information that identifies other utility poles connected via power lines. Power line height information is information indicating the height at which power lines are installed on the utility pole. Power line height information indicates the height from the base of the utility pole. Equipment information identifies the equipment installed on the utility pole. Examples of equipment installed on utility poles include transformers, switches, and IEDs (Intelligent Electronic Devices).

Figure 3 is a block diagram showing an example of the display device 100 in an embodiment. The display device 100 is, for example, a tablet terminal. The display device 100 includes, for example, an imaging unit 102, a display unit 104, an operation unit 106, an operation reception unit 108, a GPS processing unit 110, a position acquisition unit 112, a magnetic sensor unit 114, a direction acquisition unit 116, a communication unit 118, a geographic information acquisition unit 120, and a display control unit 130. The imaging unit 102 is a camera device mounted on the display device 100 and generates an image. The display unit 104 is, for example, a display device such as an LCD (Liquid Crystal Display). The operation reception unit 108 is a touch panel built into the display unit 104. The imaging unit 102 captures the surroundings. The display unit 104 displays utility pole objects and power line objects superimposed on the image captured by the imaging unit 102. The display unit 104, upon receiving instructions regarding utility pole construction, displays utility pole objects and power line objects using height information of the power lines connected to the utility pole. Instructions regarding utility pole construction include, for example, instructions to delete, relocate, or install a new utility pole, but are not limited to these; any instruction that changes the position of a utility pole or power line, equipment, etc., due to the construction is acceptable.

The GPS (Global Positioning System) processing unit 110 receives signals from GPS satellites. The position acquisition unit 112 determines the position of the display device 100 based on the GPS signals. The magnetic sensor unit 114 measures the magnetic field around the display device 100. The direction acquisition unit 116 determines the direction the display device 100 is facing based on the detection results of the magnetic sensor unit 114. The communication unit 118 is a NIC that communicates with the data management device 200, etc., via the network NW. The geographic information acquisition unit 120 acquires geographic information based on the position of its own device. For example, the geographic information acquisition unit 120 provides the position information acquired by the position acquisition unit 112 to the data management device 200 and acquires geographic information around the position indicated by the position information.

The display control unit 130 includes, for example, a recognition unit 132, a mapping unit 134, a self-position correction unit 136, a wire height calculation unit 138, and an image processing unit 140. Functional units such as the recognition unit 132, mapping unit 134, self-position correction unit 136, wire height calculation unit 138, and image processing unit 140 are realized, for example, by a processor such as a CPU (Central Processing Unit) executing a program stored in program memory. Some or all of these functional units may be realized by hardware such as an LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), or FPGA (Field-Programmable Gate Array), or they may be realized through the cooperation of software and hardware.

The recognition unit 132 includes, for example, a utility pole/power line recognition unit 150, an equipment recognition unit 152, and a geographical feature recognition unit 154.

The utility pole/power line recognition unit 150 recognizes utility pole objects and power line objects based on the captured image or geographic information. For example, the utility pole/power line recognition unit 150 recognizes areas containing utility poles and areas containing power lines within the captured image by performing pattern recognition processing on the captured image. If the geographic information acquired by the geographic information acquisition unit 120 includes utility pole information, the utility pole/power line recognition unit 150 may recognize utility pole objects and power line objects in the captured image based on the utility pole information included in the geographic information.

The equipment recognition unit 152 recognizes equipment objects installed on utility poles based on the captured image or geographic information. For example, the equipment recognition unit 152 recognizes the area containing equipment within the captured image by performing pattern recognition processing on the captured image. If the geographic information acquired by the geographic information acquisition unit 120 includes equipment information, the equipment recognition unit 152 may recognize equipment objects in the captured image based on the equipment information included in the geographic information.

The feature recognition unit 154 recognizes feature objects around the utility poles and power lines based on the captured image or geographic information. For example, the feature recognition unit 154 recognizes the region containing feature objects in the captured image by performing pattern recognition processing on the captured image. If the geographic information acquired by the geographic information acquisition unit 120 includes feature information, the feature recognition unit 154 may recognize feature objects in the captured image based on the feature information included in the geographic information. In this embodiment, objects in the captured image recognized by the recognition unit 132, regions in the captured image, and objects included in the geographic information are referred to as "objects."

The mapping unit 134 maps the utility pole objects and power line objects recognized by the utility pole/power line recognition unit 150 to the utility pole information included in the geographic information. For example, the mapping unit 134 estimates the latitude and longitude of the utility poles recognized by the utility pole/power line recognition unit 150 based on its own position and the position of the utility pole objects in the captured image, and searches for utility poles included in the geographic information that are close to the estimated latitude and longitude. The mapping unit 134 compares the shape of the utility poles included in the searched geographic information with the shape of the utility poles recognized by the utility pole/power line recognition unit 150, and determines that the utility pole objects recognized by the utility pole/power line recognition unit 150 correspond to the utility poles included in the geographic information if the degree of match exceeds a threshold. The mapping unit 134 then maps the power line objects connecting the mapped utility poles to the power lines included in the utility pole information.

The self-position correction unit 136 corrects the position of its own device based on the difference between the position information of the utility pole object, which is associated by the mapping unit 134 and recognized by the utility pole/power line recognition unit 150, and the position information of the utility pole included in the geographic information. The self-position correction unit 136 calculates the difference between the position of the utility pole object recognized by the utility pole/power line recognition unit 150 and the position of the utility pole object in the geographic information associated by the mapping unit 134. The self-position correction unit 136 corrects the position of the display device 100, acquired by the position acquisition unit 112, by the calculated difference.

The self-position correction unit 136 may also correct the direction of the display device 100 acquired by the direction acquisition unit 116. For example, the self-position correction unit 136 corrects the direction of the display device 100 acquired by the direction acquisition unit 116 by the difference between the direction of the utility pole object recognized by the utility pole/power line recognition unit 150 and the direction in the geographic information of the utility pole object associated by the association unit 134.

The power line height calculation unit 138 calculates the in-image height from the base position of the utility pole object to the power line installation position of the utility pole object, and converts the in-image height of the power line to the actual height based on the relationship between the in-image distance from the position of the device to the position of the utility pole and the actual distance from the position of the device to the position of the utility pole. The power line height calculation unit 138 identifies the base position of the utility pole object in the captured image by pattern recognition of the base position in the captured image. For example, the power line height calculation unit 138 may identify the base position of the utility pole object by information indicating the latitude, longitude, and height of the utility pole in geographic information. The power line height calculation unit 138 identifies the power line installation position of the utility pole object by pattern recognition of the base position in the captured image.

The image processing unit 140, upon receiving a change request related to utility pole construction from the display device 100, performs image processing to display utility pole objects and power line objects using height information of power lines connected to the utility pole. The image processing unit 140 includes, for example, an object deletion unit 160 and an object drawing unit 162. The object deletion unit 160 performs image processing to delete objects included in the captured image. The object drawing unit 162 performs image processing to draw objects on the captured image. In this embodiment, objects include not only utility poles and power lines, but also landmarks, equipment, trees, buildings, and other objects related to utility pole construction, such as the installation, relocation, and deletion of utility poles. In this embodiment, objects may be marks indicating the height position of power lines identified on the utility pole by the power line height derivation unit 138.

Figure 4 is a flowchart showing an example of the operation procedure of the display device 100 in the embodiment. First, the display device 100 starts the program (step S100) and starts the imaging unit 102 in accordance with the operator's operation, thereby starting imaging (step S102). Figure 5 is an example of an image captured in the embodiment. The captured image includes, for example, utility poles A, B, and C, and power lines D and E.

The utility pole/power line recognition unit 150 of the display control unit 130 recognizes utility poles A, B, C and power lines D, E included in the image captured by the imaging unit 102 (step S104). Furthermore, when the power line height derivation unit 138 recognizes utility poles, power lines, and the in-image height of power lines, it may highlight the recognized objects to clearly indicate to the operator that they have been recognized. Figure 6 shows an example of utility pole objects, power line objects, and power line heights recognized in the captured image. The utility pole/power line recognition unit 150 can, for example, recognize the in-image heights of utility poles A, power lines D, E, and power lines on utility poles B, C as objects. Note that the connection position of the power lines on the utility poles may not be recognized in the object recognition process in step S104, but may be recognized in the subsequent process of deriving the power line height.

The geographic information acquisition unit 120 acquires geographic information about the area around the display device 100 (step S106). Next, the mapping unit 134 maps the utility poles A, B, and C recognized by the utility pole/power line recognition unit 150 to the utility poles included in the geographic information (step S108). Next, the self-position correction unit 136 corrects the position of the display device 100 based on the difference in position between the utility pole objects (A, B, C) recognized by the utility pole/power line recognition unit 150 and the utility pole objects included in the geographic information, which have been mapped by the mapping unit 134 (step S110). Next, the power line height derivation unit 138 derives the heights of the power lines D and E connected to the utility pole objects (A, B, C) that have been mapped by the mapping unit 134 (step S112). As shown in Figure 6, the wire height derivation unit 138 can derive the height of the connection point of wire D on utility pole B, and the height of the connection point of wire E on utility pole C.

The display control unit 130 performs the processing in steps S104 to S112 for each of the utility pole objects (A, B, C) in the captured image generated in step S102. Furthermore, even if the captured image generated by the operator is changed due to the imaging direction, the display control unit 130 continues to perform the processing in steps S104 to S112 for the utility pole objects included in the captured image.

The display control unit 130 determines whether it has received a request to delete a utility pole as an instruction related to utility pole construction, based on the operation received by the operation reception unit 108 (step S114). If the object deletion unit 160 receives a request to delete a utility pole as an instruction related to utility pole construction (step S114: YES), it deletes the areas corresponding to the utility pole object (A) and power line objects (D, E) included in the captured image (step S116). Figure 7 shows an example of the deletion of the utility pole object (A) corresponding to utility pole A, the power line object (D) corresponding to power line D, and the power line object (E) corresponding to power line E from the captured image.

Next, the object drawing unit 162 overlays a background created using the surrounding image onto the deleted area. Specifically, the object drawing unit 162 generates a background to be overlaid on the deleted utility pole object (A) and power line objects (D, E) using the surrounding images (step S118). The object drawing unit 162 overlays the generated background onto the area of the deleted utility pole object (A) and power line objects (D, E) and displays it on the display unit 104 (step S120). Figure 8 is a diagram showing an example of how the background is overlaid on the captured image. As shown in Figure 7, the object drawing unit 162 creates the background by interpolating the image of the area occupied by the deleted utility pole object (A) and power line objects (D, E) from the surrounding images of the deleted utility pole object (A) and power line objects (D, E). As a result, the display unit 104 can display an image showing the work performed to remove utility pole A and power lines D and E, as shown in Figure 8.

The image processing unit 140 determines whether it has received a request to relocate a utility pole as an instruction regarding utility pole construction, based on the operation received by the operation reception unit 108 (step S122). The operation received by the operation reception unit 108 includes, for example, selecting a utility pole object (A) and moving the utility pole object (A) from its current position to the relocation destination. The display control unit 130 determines that a relocation request has been received when the operation reception unit 108 receives this operation. If the object deletion unit 160 receives a request to relocate the utility pole object (A) (step S114: YES), it deletes the areas corresponding to the utility pole object (A) and power line objects (D, E) included in the captured image (step S124, Figure 7).

Next, the object rendering unit 162 generates a background to be superimposed on the deleted utility pole object (A) and power line objects (D, E) using the images surrounding the deleted utility pole object (A) and power line objects (D, E) (step S126). The object rendering unit 162 then superimposes and displays the background created using the images surrounding the deleted area (step S128, Figure 9). Next, the object rendering unit 162 displays the utility pole object (F) at the relocation site and, using the power line height information, displays the power line objects (G, H) connected to the utility pole object (F) displayed at the relocation site (step S130, Figure 10). This allows the object rendering unit 162 to superimpose and display the utility pole object (F) and power line objects (G, H) on the captured image.

Figure 9 shows an example of an image captured in an embodiment where objects corresponding to utility pole A, power lines D and E have been deleted in response to a request to relocate a utility pole corresponding to utility pole A. Figure 10 shows an example of an image captured where objects corresponding to utility pole F, to which utility pole A has been relocated, and power lines G and H connected to utility pole F have been drawn. Suppose the display control unit 130 receives a request to relocate, for example, utility pole A shown in Figure 5 to utility pole F shown in Figure 10. When the relocation request is received, the power line height derivation unit 138 derives the power line heights at utility poles B and C, to which power lines D and E, which were connected to utility pole A, were connected, as shown in Figure 9. The object drawing unit 162 places a utility pole object corresponding to utility pole F at the relocation destination in the image captured, as shown in Figure 10, and draws a power line object for power line G that connects the power line connection position of the utility pole object corresponding to utility pole F to the power line height position of utility pole B. Similarly, the object drawing unit 162 draws the wire object for the wire H that connects the wire connection position of the utility pole object corresponding to utility pole F to the wire height position of utility pole C. The object drawing unit 162 may also highlight the relocated objects (utility poles, wires, height, etc.) to indicate to the operator that the drawing process has been completed.

Next, the image processing unit 140 determines whether or not it has received a request to install a new utility pole as an instruction regarding utility pole construction based on the operation received by the operation reception unit 108 (step S132). The operation received by the operation reception unit 108 includes, for example, an operation to specify the location where the new utility pole will be installed, and the display control unit 130 determines that it has received a request to install a new utility pole when it receives such an operation from the operation reception unit 108. If the object drawing unit 162 has received a request to install a new utility pole object (step S132: YES), it displays the utility pole object and the power line object connected to the utility pole object at the received new installation location (step S134). This allows the object drawing unit 162 to display the newly installed utility pole object and power line object superimposed on the captured image. If the request to install a new utility pole object has not been received (step S132: NO), the display control unit 130 determines whether or not the program has been terminated (step S136). If the program has not been terminated (step S136: NO), the display control unit 130 repeats the processing from step S102 onward. If the program has been terminated (step S136: YES), it terminates the processing of this flowchart.

Figure 11 is a sequence diagram showing an example of the operation procedure of the display system in the embodiment. When the program is started, the display device 100 provides the data management device 200 with the location information (self-position) of the display device 100 acquired by the GPS processing unit 110 and the location acquisition unit 112. The data provision unit 210 of the data management device 200 requests geographic information about the area surrounding the display device 100's location from the data storage device 300, and the data storage device 300 provides the geographic information about the area surrounding the display device 100's location to the data provision unit 210. The data provision unit 210 then provides the geographic information about the area surrounding the display device 100's location to the display device 100.

The display unit 104 displays the captured image generated by the imaging unit 102, and also displays the utility pole and power line objects that have been associated by the mapping unit 134 based on the geographical information of the area surrounding the location of the display device 100 (step S200). The display control unit 130 receives a change request for utility poles as an instruction regarding utility pole construction from the operation unit 106 and the operation reception unit 108 (step S202). The change request may include, for example, utility pole construction such as deletion, relocation, or new construction. Accordingly, the display control unit 130 deletes the object using the object deletion unit 160 and draws the object using the object drawing unit 162. The display device 100 can then display the results of the object deletion and drawing performed in response to the change request on the display unit 104. This completes the object modification stage.

Next, the display system performs the update phase of the GIS data, storing the object changes in the data storage device 300. When the operation unit 106 and the operation reception unit 108 receive an operation to save the GIS data changes, they provide the data update unit 220 with change data indicating the changes to the objects. The change data may include, for example, information indicating deleted utility pole objects and power line objects from the geographic information. The change data may also include information indicating relocated utility pole objects, information indicating the relocation destination of power line objects, information indicating power line objects connected to relocated utility pole objects, information indicating the utility pole to which the power line objects are connected, and information indicating the power line height of the utility pole to which the power line objects are connected.

The data update unit 220, upon receiving the change data from the display device 100, sends a request to the data storage device 300 to save the change data. The data storage device 300 updates the GIS data based on the change data provided via the data update unit 220 (step S206).

Figure 12 shows an example of a utility pole update table included in the GIS data in this embodiment. The data storage device 300 generates the utility pole update table based on the change data provided by the data update unit 220. The utility pole update table includes, for example, the location of the utility pole, the shape of the utility pole, utility pole connection information, power line height information, and equipment information updated according to instructions regarding utility pole construction. The data storage device 300 transmits completion report information to the data update unit 220 indicating that the creation of the utility pole update table is complete, and the data update unit 220 provides completion report information to the display device 100 indicating that the saving of the change data is complete. This completes the GIS data update stage.

For example, when explaining the details of utility pole construction to residents near the construction site, the operation unit 106 and operation reception unit 108 receive a request to view the change data based on the worker's operation (step S208). The display device 100 then requests the data provision unit 210 to provide the change data. Upon receiving the request for change data from the display device 100, the data provision unit 210 requests the data storage device 300 to provide the change data. The data storage device 300 provides the GIS data included in the utility pole update table to the data provision unit 210 in response to the request for change data. This allows the data provision unit 210 to provide the change data (GIS data) requested by the display device 100. The display device 100 can then display an image including the utility pole object and power line object after the utility pole construction is completed using the requested change data (step S210). This completes the change confirmation stage for utility pole construction.

Figure 13 is an example of an image captured including equipment objects before the relocation of the utility pole, and Figure 14 is an example of an image captured including equipment objects after the relocation of the utility pole. As shown in Figure 13, the equipment recognition unit 152 recognizes the equipment object corresponding to equipment I when equipment I is installed on utility pole A. When the display device 100 receives a request to delete utility pole A, it deletes the area corresponding to the equipment object recognized by the equipment recognition unit 152 and superimposes a background object created using the surrounding image onto the deleted area.

When the object drawing unit 162 receives a request to relocate utility pole A, on which equipment object I is installed, as an instruction regarding utility pole construction, it draws a utility pole object corresponding to utility pole F at the relocation site, as shown in Figure 14, and then draws the equipment object J installed on the relocated utility pole F. Furthermore, the object drawing unit 162 draws a wire object corresponding to the wire G connected to the utility pole object corresponding to utility pole F, connecting it to the wire height position on utility pole B. Finally, the object drawing unit 162 draws a wire object corresponding to the wire H connected to the utility pole object corresponding to utility pole F, connecting it to the wire height position on utility pole C. As a result, according to the display device 100, even when equipment is attached to a utility pole, equipment objects corresponding to the equipment can be drawn in accordance with the change in the utility pole. Consequently, according to the display device 100, changes in the equipment installation status, in addition to the installation status of utility poles and wires, can be easily confirmed.

Figure 15 shows an example of an image captured including a feature object before the utility pole was relocated, Figure 16 shows an example of an image captured after the feature object was removed, and Figure 17 shows an example of an image captured after the utility pole was relocated. As shown in Figure 15, the feature recognition unit 154 recognizes a feature object corresponding to a tree K if such a tree exists around the utility pole A. When the display device 100 receives a request to remove the tree K, as shown in Figure 16, it deletes the area corresponding to the feature object recognized by the feature recognition unit 154 and overlays a background object created using the surrounding image onto the deleted area. As a result, the display device 100 allows for easy confirmation of the post-construction situation even when features are removed during utility pole construction.

When the display device 100 receives a request to relocate utility pole A to the location where tree K exists, it deletes the area corresponding to the recognized feature object by the feature recognition unit 154 and superimposes a background object created using the surrounding image onto the deleted area. Furthermore, as shown in Figure 17, the object drawing unit 162 draws a utility pole object corresponding to utility pole L at the relocation site. The object drawing unit 162 then draws a power line object corresponding to the power line M connected to the utility pole object corresponding to utility pole L, connecting it to the power line height position of utility pole B. Finally, the object drawing unit 162 draws a power line object corresponding to the power line N connected to the utility pole object corresponding to utility pole L, connecting it to the power line height position of utility pole C. As a result, the display device 100 can delete the feature object and draw the utility pole object even if a feature exists at the relocation site of the utility pole. Consequently, the display device 100 allows for easy confirmation of the post-construction situation after the removal of the feature and relocation of the utility pole.

As described above, the display device 100 of this embodiment comprises an imaging unit 102 that captures images of the surroundings, and a display unit 104 that superimposes and displays utility pole objects and power line objects onto the image captured by the imaging unit 102. The display unit 104 can display utility pole objects and power line objects using height information of the power lines connected to the utility poles in response to receiving instructions regarding utility pole construction. Because the display device 100 displays utility pole objects and power line objects in response to receiving instructions regarding utility pole construction, it is possible to easily check the installation status of utility poles and power lines after utility pole construction, such as deletion, relocation, or new installation.

According to this embodiment, a program can be implemented that causes the computer of the display device 100 to perform the following steps: to have the imaging unit 102 capture images of the surroundings; to superimpose utility pole objects and power line objects onto the captured images on the display unit 104; and to update the display state of the utility pole objects and power line objects using height information of the power lines connected to the utility poles in response to receiving instructions regarding utility pole construction. Because the program displays utility pole objects and power line objects in response to receiving instructions regarding utility pole construction, it is possible to easily check the installation status of utility poles and power lines after utility pole construction, such as deletion, relocation, or new installation.

According to this embodiment, a display method can be realized that includes the steps of: having the imaging unit 102 capture images of the surroundings; superimposing utility pole objects and power line objects onto the captured images on the display unit 104; and displaying utility pole objects and power line objects using height information of the power lines connected to the utility pole in response to receiving instructions regarding utility pole construction. This display method allows for easy confirmation of the installation status of utility poles and power lines after utility pole construction, such as deletion, relocation, or new installation.

According to this embodiment, a display system can be realized that includes a data management device 200 that provides geographic information, an imaging unit 102 that images the surroundings, a utility pole/power line recognition unit 150 that recognizes utility pole objects and power line objects included in the captured image, a geographic information acquisition unit 120 that acquires geographic information based on the position of the device, a mapping unit 134 that maps the utility pole objects and power line objects recognized by the utility pole/power line recognition unit 150 to utility poles and power lines included in the geographic information, and a display unit 104 that superimposes and displays the utility pole objects and power line objects mapped by the mapping unit 134 onto the captured image taken by the imaging unit 102, and a display device 100 that displays utility pole objects and power line objects using height information of power lines connected to utility poles in response to receiving instructions regarding utility pole construction via the display unit 104. According to the display system, utility pole and power line objects are displayed in response to instructions received regarding utility pole construction. This allows for easy confirmation of the installation status of utility poles and power lines after construction work, including deletion, relocation, and new installation.

Furthermore, according to this embodiment, the system includes a self-position correction unit 136 that corrects the position of the device based on the difference between the position information of the utility pole and power line objects, which are associated by the matching unit 134 and recognized by the utility pole and power line recognition unit 150, and the position information of the utility poles included in the geographic information. According to this embodiment, the utility pole and power line objects can be displayed based on the position of the highly accurate display device 100. As a result, according to this embodiment, the utility pole and power line objects can be displayed at more accurate positions in the captured image.

Furthermore, according to this embodiment, the system includes a height derivation unit that calculates the in-image height from the base position of the utility pole object to the position where the power lines are installed on the utility pole object, and converts the in-image height to the actual height based on the relationship between the in-image distance from the position of the device to the position of the utility pole and the actual distance from the position of the device to the position of the utility pole. According to this embodiment, since the height information of the power lines on the utility pole object can be derived, the work of setting the power line height by the operator can be eliminated. Also, according to this embodiment, the work of setting the start and end points of the power line object by the operator can be eliminated.

While various embodiments and modifications have been described, these are merely examples and not limiting. For example, one embodiment or modification, or a part of one embodiment or modification, may be combined with one or more other embodiments or modifications to realize one aspect of the present invention.

100…Display device, 100a…Captured image, 102…Imaging unit, 104…Display unit, 106…Operation unit, 108…Operation reception unit, 110…GPS processing unit, 112…Position acquisition unit, 114…Magnetic sensor unit, 116…Direction acquisition unit, 118…Communication unit, 120…Geographic information acquisition unit, 130…Display control unit, 132…Recognition unit, 134…Mapping unit, 136…Self-position correction unit, 138…Power line height derivation unit, 140…Image processing unit, 150…Utility pole/power line recognition unit, 152…Equipment recognition unit, 154…Feature recognition unit, 160…Object deletion unit, 162…Object drawing unit, 200…Data management device, 210…Data provision unit, 220…Data update unit, 300…Data storage device

Claims (19)

An imaging unit that captures images of the surroundings,
The system includes a display unit that superimposes and displays utility pole objects and power line objects onto the captured image captured by the aforementioned imaging unit,
The aforementioned display unit is
In response to receiving instructions regarding utility pole construction, the system displays the utility pole object and the power line object using the height information of the power lines connected to the utility pole .
When a request to delete the utility pole is received as an instruction regarding the aforementioned utility pole construction, the areas corresponding to the utility pole object and the power line object included in the captured image are deleted, and a background object created using the image surrounding the deleted area is superimposed and displayed on the deleted area.
Display device. An imaging unit that captures images of the surroundings,
The system includes a display unit that superimposes and displays utility pole objects and power line objects onto the captured image captured by the aforementioned imaging unit,
The aforementioned display unit is
In response to receiving instructions regarding utility pole construction, the system displays the utility pole object and the power line object using the height information of the power lines connected to the utility pole .
If a request to relocate the aforementioned utility pole is received as an instruction regarding the aforementioned utility pole construction,
The regions corresponding to the utility pole object and the power line object included in the captured image are deleted, and a background object created using the surrounding image of the region is superimposed and displayed on the deleted region.
The utility pole object is displayed at the relocation site, and the power line object connected to the utility pole object displayed at the relocation site is displayed using the power line height information.
Display device. An imaging unit that captures images of the surroundings,
The system includes a display unit that superimposes and displays utility pole objects and power line objects onto the captured image captured by the aforementioned imaging unit,
In response to receiving instructions regarding utility pole construction, the display unit displays the utility pole object and the power line object using height information of the power lines connected to the utility pole .
A geographic information acquisition unit that acquires geographic information based on the position of the device,
A utility pole and power line recognition unit that recognizes the utility pole object and the power line object based on the captured image or the geographical information,
The system includes a matching unit that associates the utility pole object and the power line object recognized by the utility pole and power line recognition unit with the utility poles included in the geographical information .
Display device. The display device according to claim 3, further comprising a correction unit that corrects the position of the device based on the difference between the position information of the utility pole object, which has been associated by the association unit and recognized by the utility pole and power line recognition unit, and the position information of the utility pole included in the geographical information. An imaging unit that captures images of the surroundings,
The system includes a display unit that superimposes and displays utility pole objects and power line objects onto the captured image captured by the aforementioned imaging unit,
In response to receiving instructions regarding utility pole construction, the display unit displays the utility pole object and the power line object using height information of the power lines connected to the utility pole .
A display device comprising a height derivation unit that calculates the in-image height from the base position of the utility pole object to the position where the power lines are installed on the utility pole object, and converts the in-image height to the actual height based on the relationship between the in-image distance from the position of the device to the position of the utility pole and the actual distance from the position of the device to the position of the utility pole . The system includes an equipment recognition unit that recognizes equipment objects installed on the utility pole based on the captured image or the geographical information,
The aforementioned display unit is
When a request to delete the aforementioned utility pole is received, the area corresponding to the equipment object recognized by the equipment recognition unit is deleted, and a background object created using the image surrounding the area is superimposed on the deleted area.
When a request is received to relocate the aforementioned utility pole, the equipment object will be displayed at the relocation site.
The display device according to claim 3 . The system includes a feature recognition unit that recognizes feature objects around the utility pole and the power line based on the captured image or the geographic information,
When the display unit receives a request to delete a feature, it deletes the area corresponding to the feature recognized by the feature recognition unit, and overlays and displays a background object created using the image surrounding the area onto the deleted area.
The display device according to claim 3 . In the computer of the display device,
The steps include: having the imaging unit capture images of the surrounding area,
The steps include: superimposing utility pole objects and power line objects onto the captured image captured by the imaging unit and displaying them on the display unit;
In response to receiving instructions regarding utility pole construction, the system performs the step of updating the display state of the utility pole object and the power line object using the height information of the power lines connected to the utility pole.
A program that, upon receiving a request to delete the utility pole as an instruction related to the aforementioned utility pole construction, deletes the area corresponding to the utility pole object and the power line object included in the captured image, and overlays and displays a background object created using the image surrounding the deleted area . In the computer of the display device,
The steps include: having the imaging unit capture images of the surrounding area,
The steps include: superimposing utility pole objects and power line objects onto the captured image captured by the imaging unit and displaying them on the display unit;
In response to receiving instructions regarding utility pole construction, the system performs the step of updating the display state of the utility pole object and the power line object using the height information of the power lines connected to the utility pole.
If a request to relocate the aforementioned utility pole is received as an instruction regarding the aforementioned utility pole construction,
The regions corresponding to the utility pole object and the power line object included in the captured image are deleted, and a background object created using the surrounding image of the region is superimposed and displayed on the deleted region.
The utility pole object is displayed at the relocation site, and the power line object connected to the utility pole object displayed at the relocation site is displayed using the power line height information.
program. In the computer of the display device,
The steps include: having the imaging unit capture images of the surrounding area,
The steps include: superimposing utility pole objects and power line objects onto the captured image captured by the imaging unit and displaying them on the display unit;
In response to receiving instructions regarding utility pole construction, the display state of the utility pole object and the power line object is updated using height information of the power lines connected to the utility pole.
A step of acquiring geographic information based on the location of the device,
The steps include recognizing the utility pole object and the power line object based on the captured image or the geographical information,
The steps include: associating the recognized utility pole object and the power line object with the utility poles included in the geographic information;
A program that executes something. In the computer of the display device,
The steps include: having the imaging unit capture images of the surrounding area,
The steps include: superimposing utility pole objects and power line objects onto the captured image captured by the imaging unit and displaying them on the display unit;
In response to receiving instructions regarding utility pole construction, the display state of the utility pole object and the power line object is updated using height information of the power lines connected to the utility pole.
The steps include: calculating the in-image height from the base position of the utility pole object to the position where the power lines are installed on the utility pole object; and converting the in-image height to the actual height based on the relationship between the in-image distance from the position of the device to the position of the utility pole and the actual distance from the position of the device to the position of the utility pole;
A program that executes something. The display device takes the step of having the imaging unit capture images of the surroundings,
The display device includes the steps of superimposing utility pole objects and power line objects onto the captured image captured by the imaging unit and displaying them on the display unit,
The display device, upon receiving instructions regarding utility pole construction, displays the utility pole object and the power line object using height information of the power lines connected to the utility pole;
When the display device receives a request to delete the utility pole as an instruction regarding the utility pole construction, it deletes the area corresponding to the utility pole object and the power line object included in the captured image, and overlays and displays a background object created using the image surrounding the deleted area.
Display methods, including those mentioned above. The display device takes the step of having the imaging unit capture images of the surroundings,
The display device includes the steps of superimposing utility pole objects and power line objects onto the captured image captured by the imaging unit and displaying them on the display unit,
The display device, upon receiving instructions regarding utility pole construction, includes the step of displaying the utility pole object and the power line object using height information of the power lines connected to the utility pole,
The aforementioned display device
If a request to relocate the aforementioned utility pole is received as an instruction regarding the aforementioned utility pole construction,
The regions corresponding to the utility pole object and the power line object included in the captured image are deleted, and a background object created using the surrounding image of the region is superimposed and displayed on the deleted region.
The utility pole object is displayed at the relocation site, and the power line object connected to the utility pole object displayed at the relocation site is displayed using the power line height information.
Display method. The display device takes the step of having the imaging unit capture images of the surroundings,
The display device includes the steps of superimposing utility pole objects and power line objects onto the captured image captured by the imaging unit and displaying them on the display unit,
The display device, upon receiving instructions regarding utility pole construction, displays the utility pole object and the power line object using height information of the power lines connected to the utility pole;
The display device acquires geographic information based on its own location.
The display device includes the steps of recognizing the utility pole object and the power line object based on the captured image or the geographic information,
The display device performs the steps of associating the recognized utility pole object and the power line object with the utility pole included in the geographic information,
Display methods, including those mentioned above. The display device takes the step of having the imaging unit capture images of the surroundings,
The display device includes the steps of superimposing utility pole objects and power line objects onto the captured image captured by the imaging unit and displaying them on the display unit,
The display device, upon receiving instructions regarding utility pole construction, displays the utility pole object and the power line object using height information of the power lines connected to the utility pole;
The display device calculates the in-image height from the base position of the utility pole object to the position where the power lines are installed on the utility pole object, and converts the in-image height to the actual height based on the relationship between the in-image distance from the position of the device to the position of the utility pole and the actual distance from the position of the device to the position of the utility pole.
Display methods, including those mentioned above. A data management device that provides geographic information,
The device comprises: an imaging unit that images the surroundings; a utility pole and power line recognition unit that recognizes utility pole and power line objects included in the captured image; a geographic information acquisition unit that acquires geographic information based on the position of the device; a mapping unit that maps the utility pole and power line objects recognized by the utility pole and power line recognition unit to the utility poles and power lines included in the geographic information; and a display unit that superimposes the utility pole and power line objects mapped by the mapping unit onto the captured image taken by the imaging unit, wherein the display unit, in response to receiving instructions regarding utility pole construction, displays the utility pole and power line objects using height information of the power lines connected to the utility pole.
A display system equipped with the following features. The display system according to claim 16, further comprising a correction unit that corrects the position of the device based on the difference between the position information of the utility pole object, which has been associated by the association unit and recognized by the utility pole and power line recognition unit, and the position information of the utility pole included in the geographical information. The system includes an equipment recognition unit that recognizes equipment objects installed on the utility pole based on the captured image or the geographical information,
The aforementioned display unit is
When a request to delete the aforementioned utility pole is received, the area corresponding to the equipment object recognized by the equipment recognition unit is deleted, and a background object created using the image surrounding the area is superimposed on the deleted area.
When a request is received to relocate the aforementioned utility pole, the equipment object will be displayed at the relocation site.
The display system according to claim 16 . The system includes a feature recognition unit that recognizes feature objects around the utility pole and the power line based on the captured image or the geographic information,
When the display unit receives a request to delete a feature, it deletes the area corresponding to the feature recognized by the feature recognition unit, and overlays and displays a background object created using the image surrounding the area onto the deleted area.
The display system according to claim 16 .