JP7704184B2 - Management device, display system, display method and program - Google Patents

Description

The present disclosure relates to a management device, a display system, a display method, and a program.

To efficiently perform work at facilities such as substations, workers may carry out work while checking a procedure manual displayed on a portable terminal device. Patent Document 1 describes the configuration of a work support system that displays a manual or the like superimposed on the real-world facility as seen through an AR (Augmented Reality) display.

JP 2020-149140 A

However, the work support system described in Patent Document 1 only displays manuals related to equipment within the field of view of the user wearing the AR display. Therefore, when there are many pieces of equipment in a wide area, there are many pieces of equipment that are not within the user's field of view. In this case, the manuals related to equipment that is not within the user's field of view cannot be checked in advance, which creates a problem of reduced efficiency in the work performed by the workers on the equipment.

The objective of the present disclosure is to provide a management device, display system, display method, and program that allows workers to easily check procedures related to equipment that requires work, even if the equipment is not visible to the workers.

The management device according to the first aspect of the present disclosure includes an acquisition unit that acquires three-dimensional data including a monitored facility, an identification unit that identifies a plurality of parts that constitute the monitored facility, a display control unit that rotatably displays the monitored facility in an equipment data display area of a display unit and displays a plurality of procedures to be performed by an operator for the monitored facility in a procedure display area of the display unit, and an input unit that accepts selection of at least one part displayed in the equipment data display area or at least one procedure displayed in the procedure display area, and the display control unit highlights the procedure associated with the selected at least one part, or the part associated with the selected at least one procedure.

A display system according to a second aspect of the present disclosure comprises a display device that displays an equipment data display area including monitored equipment and a procedure display area including a plurality of procedures to be performed by an operator for the monitored equipment, an acquisition unit that acquires three-dimensional data including the monitored equipment, an identification unit that identifies a plurality of parts that constitute the monitored equipment, a display control unit that rotatably displays the monitored equipment in the equipment data display area of the display device and displays a plurality of procedures to be performed by an operator for the monitored equipment in the procedure display area of the display device, and an input unit that accepts selection of at least one part displayed in the equipment data display area or at least one procedure displayed in the procedure display area, wherein the display control unit highlights the procedure associated with the selected at least one part, or the part associated with the selected at least one procedure.

A display method according to a third aspect of the present disclosure includes acquiring three-dimensional data including a monitored facility,
The system identifies a plurality of parts that constitute the monitored equipment, rotatably displays the monitored equipment in an equipment data display area of the display unit, displays a plurality of procedures to be performed by workers related to the monitored equipment in a procedure display area of the display unit, accepts selection of at least one part displayed in the monitored equipment data display area or at least one procedure displayed in the procedure display area, and highlights and displays the procedure associated with the selected at least one part, or the part associated with the selected at least one procedure.

A program according to a fourth aspect of the present disclosure includes acquiring three-dimensional data including a monitored facility,
The computer is caused to identify a plurality of parts that constitute the monitored equipment, rotatably display the monitored equipment in an equipment data display area of a display unit, display a plurality of procedures to be performed by an operator for the monitored equipment in a procedure display area of the display unit, accept selection of at least one part displayed in the monitored equipment data display area or at least one procedure displayed in the procedure display area, and highlight and display the procedure associated with the selected at least one part, or the part associated with the selected at least one procedure.

This disclosure provides a management device, display system, display method, and program that allows workers to easily check procedures for equipment that requires work, even if the equipment is out of the worker's field of vision.

FIG. 2 is a configuration diagram of a management device according to the present disclosure. FIG. 2 is a diagram showing a display image of a display unit according to the present disclosure. FIG. 11 is a diagram showing a flow of a display process in a management device according to the present disclosure. FIG. 2 is a configuration diagram of a management device according to the present disclosure. FIG. 2 is a diagram showing management information managed by a management unit according to the present disclosure. FIG. 2 is a diagram showing management information managed by a management unit according to the present disclosure. FIG. 2 is a diagram showing a display image of a display unit according to the present disclosure. FIG. 2 is a diagram showing a display image of a display unit according to the present disclosure. FIG. 2 is a diagram showing a display image of a display unit according to the present disclosure. FIG. 11 is a diagram showing a flow of a display process in a management device according to the present disclosure. FIG. 2 is a configuration diagram of a management device according to the present disclosure.

(Embodiment 1)
An example of the configuration of the management device 10 will be described below with reference to Fig. 1. The management device 10 may be a computer device that operates when a processor executes a program stored in a memory. The management device 10 may be, for example, a server device.

The management device 10 has an acquisition unit 11, an identification unit 12, a display control unit 13, and an input unit 14. The acquisition unit 11, the identification unit 12, the display control unit 13, and the input unit 14 may be software or modules in which processing is performed by a processor executing a program stored in a memory. Alternatively, the acquisition unit 11, the identification unit 12, the display control unit 13, and the input unit 14 may be hardware such as a circuit or a chip.

The acquisition unit 11 acquires three-dimensional data including the monitored equipment. The monitored equipment may be equipment that is the subject of monitoring, inspection, or maintenance work performed by an operator periodically or at any time. Maintenance work includes repair of the monitored equipment, replacement of parts, and the like. The monitored equipment may be, for example, a substation or equipment such as a transformer substation, or may be other equipment that requires monitoring or management.

The three-dimensional data may be, for example, point cloud data, which is a set of points identified using three-dimensional coordinates. The three-dimensional data may be, for example, generated using a device that measures the distance to an object. The device that measures the distance to an object may be, for example, a LiDAR (Light Detection And Ranging) device. The LiDAR device measures the distance to an object using a ToF (Time of Flight) method. Furthermore, the LiDAR device generates points that indicate the shape of the object based on the measurement results. A set of points that indicate the shape of the object becomes point cloud data. The points that indicate the shape of the object may be identified using three-dimensional coordinates in a specified space. In other words, the points that indicate the shape of the object may be indicated using three-dimensional coordinates in a predetermined coordinate system.

Alternatively, the three-dimensional data may be generated based on multiple captured images of the monitored equipment. For example, the device that generates the three-dimensional data may use Structure from Motion (SfM) to generate three-dimensional data of the monitored equipment from multiple captured images.

Alternatively, the three-dimensional data may be generated based on drawing data or the like showing the structure of the monitored equipment. For example, the generation of the three-dimensional data based on drawing data or the like showing the structure of the monitored equipment may be executed by an application or the like.

The acquisition unit 11 may receive three-dimensional data from another device via a network. Alternatively, the acquisition unit 11 may acquire three-dimensional data from another device via a wired cable or a wireless communication line. Alternatively, the acquisition unit 11 may generate three-dimensional data as a LiDAR device.

The identification unit 12 identifies multiple components that make up the monitored equipment. The components may be some of the items or devices that make up the monitored equipment. For example, if the monitored equipment is a substation, the components may be cables, insulators, etc. Or, the components may be transformers, circuit breakers, etc.

The identification unit 12 may, for example, identify multiple parts that make up the monitored equipment using a learning model that takes three-dimensional data of the monitored equipment as input and outputs the parts that make up the monitored equipment. The learning model may be an AI (Artificial Intelligence) model, a machine learning model, or the like. The learning model may be a trained model that has been trained in advance using three-dimensional data and three-dimensional data or image data related to the parts. The learning model may be generated in the management device 10, or may be generated in a device different from the management device 10.

Alternatively, the identification unit 12 may use semantic segmentation to divide the monitored equipment, which is three-dimensional data, into multiple parts. In semantic segmentation, for example, each pixel constituting an image showing the three-dimensional data may be labeled. Alternatively, in semantic segmentation, each point constituting a point cloud, which is three-dimensional data, may be labeled. The label may be, for example, information indicating the name of the part. Alternatively, the label may be identification information of the part, such as an ID (Identification or Identifier).

The display control unit 13 displays the monitored equipment in a rotatable manner in the equipment data display area 21 of the display unit 20. Furthermore, the display control unit 13 displays a number of procedures that the worker should perform in the monitored equipment in the procedure display area 22 of the display unit 20.

The display unit 20 may be, for example, a display that displays image data. The display unit 20 may be used integrally with the management device 10. Alternatively, the display unit 20 may be a device different from the management device 10, for example, a display device. In this case, the management device 10 and the display device may be connected via a cable or the like, or may be connected via a network.

As shown in FIG. 2, the equipment data display area 21 is a partial area of the screen that constitutes the display unit 20. The procedure display area 22 is also a partial area of the screen that constitutes the display unit 20. The equipment data display area 21 and the procedure display area 22 may be areas of the same size, or may be areas of different sizes. Furthermore, the equipment data display area 21 and the procedure display area 22 may be displayed on the left and right sides of the display unit 20 as shown in FIG. 2, or may be displayed on the upper and lower sides of the display unit 20.

Displaying the monitored equipment in a rotatable manner may mean displaying the shape of the monitored equipment when the monitored equipment, which is three-dimensional data, is viewed from viewpoints in various directions or at various angles in the equipment data display area 21. In the equipment data display area 21, the monitored equipment may be rotated around at least one of the X, Y, and Z axes that represent three-dimensional coordinates. Alternatively, in the equipment data display area 21, the monitored equipment may be rotated around any rotation axis set in the three-dimensional space represented by the three-dimensional coordinates.

The multiple procedures that the worker should execute in the monitored equipment may be, for example, procedures indicating monitoring work, inspection work, maintenance work, etc. of the monitored equipment. Data in which the procedures are explained may be displayed in the procedure display area 22. The data in which the procedures are explained may include, for example, at least one of text data, image data, and video data. The worker may scroll through the procedures displayed in the procedure display area 22 to display a desired procedure in the procedure display area 22. The multiple procedures may include procedures for different types of parts, or may include procedures for the same type of parts but with different versions or sizes, etc.

The equipment data display area 21 and the procedure display area 22 may be displayed simultaneously on the display unit 20. Alternatively, the equipment data display area 21 and the procedure display area 22 may be displayed sequentially on the display unit 20. For example, the equipment data display area 21 may be displayed first on the display unit 20, and then the procedure display area 22 may be displayed. The equipment data display area 21 and the procedure display area 22 may be the same size, or one may be larger than the other. The areas of the equipment data display area 21 and the procedure display area 22 may be variable. Alternatively, the equipment data display area 21 and the procedure display area 22 may display data of the same resolution, or each may display data of different resolutions.

The input unit 14 accepts the selection of at least one part displayed in the equipment data display area 21 or at least one procedure displayed in the procedure display area 22. The input unit 14 may accept the selection of a part or procedure, for example, by a worker touching a touch panel. Alternatively, the input unit 14 may accept the selection of a part or procedure by a worker pressing a button. Alternatively, the input unit 14 may accept the selection of a part or procedure by voice, line of sight, etc.

The display control unit 13 highlights the procedure associated with at least one selected part, or the part associated with at least one selected procedure. For example, the display control unit 13 may display the procedure associated with at least one selected part in a color different from other procedures. Alternatively, the display control unit 13 may display the procedure associated with at least one selected part larger than the other procedures. Alternatively, the display control unit 13 may display only the procedure associated with at least one selected part in the procedure display area 22. Alternatively, the display control unit 13 may blink the procedure associated with at least one selected part. Alternatively, the display control unit 13 may display the procedure display area 22 displaying the procedure associated with at least one selected part larger than the equipment data display area 21. Alternatively, the display control unit 13 may display the resolution of the data displayed in the procedure display area 22 displaying the procedure associated with at least one selected part higher than the resolution of the data displayed in the equipment data display area 21.

Alternatively, the display control unit 13 may display the parts associated with the at least one selected procedure in a color different from that of the other parts. Alternatively, the display control unit 13 may display the parts associated with the at least one selected procedure larger than the other parts. Alternatively, the display control unit 13 may display only the parts associated with the at least one selected procedure in the equipment data display area 21. Alternatively, the display control unit 13 may blink the parts associated with the at least one selected procedure. Alternatively, the display control unit 13 may display the equipment data display area 21 displaying the parts associated with the at least one selected procedure larger than the procedure display area 22. Alternatively, the display control unit 13 may increase the resolution of the data displayed in the equipment data display area 21 displaying the parts associated with the at least one selected procedure to be higher than the resolution of the data displayed in the procedure display area 22.

Next, the flow of the display process executed in the management device 10 will be described with reference to FIG. 3. First, the acquisition unit 11 acquires three-dimensional data including the monitored equipment (S11). Next, the identification unit 12 identifies a plurality of parts constituting the monitored equipment (S12). Next, the display control unit 13 rotatably displays the monitored equipment in the equipment data display area of the display unit 20, and displays a plurality of procedures to be performed by the worker for the monitored equipment in the procedure display area of the display unit 20 (S13). Next, the input unit 14 accepts the selection of at least one part displayed in the equipment data display area or at least one procedure displayed in the procedure display area (S14). Next, the display control unit 13 highlights and displays the procedure associated with the at least one part, or the part associated with the selected at least one procedure (S15).

As described above, the management device 10 displays the monitored equipment in a rotatable manner in the equipment data display area 21 of the display unit 20. This allows the worker to check the parts of the monitored equipment that are located out of the worker's field of vision in the equipment data display area 21. Furthermore, the management device 10 can accept the selection of the parts of the monitored equipment that are located out of the worker's field of vision. As a result, the management device 10 can highlight and display the procedures associated with the parts of the monitored equipment that are located out of the worker's field of vision.

In addition, the management device 10 may display parts of the monitored equipment that are located outside the worker's field of vision by accepting the selection of a procedure. This allows the worker to recognize the parts associated with the selected procedure.

(Embodiment 2)
Next, a configuration example of the management device 30 will be described with reference to Fig. 4. In Fig. 4, an example will be described in which the management device 10 and the display unit 20 in Fig. 1 are configured as an integrated unit. Furthermore, the management device 30 has a configuration in which a management unit 31 is added to the management device 10. In the following description, detailed description of processes, operations, functions, etc. that are the same as those described in Fig. 1 will be omitted.

The identification unit 12 divides the point cloud data indicating the monitored equipment received from the acquisition unit 11 into a plurality of parts constituting the monitored equipment using a learning model or semantic segmentation. Furthermore, the identification unit 12 assigns identification information to each part based on the shape, size, color, and position of each part in the monitored equipment. For example, the identification unit 12 further assigns identification information to each part to the point cloud data to which a label indicating the type of part has been assigned by semantic segmentation. Specifically, the identification unit 12 assigns identification information to identify which part of the monitored equipment the cable is installed in to the point cloud data to which a label has been assigned as a cable. In other words, the identification unit 12 may further assign product-based identification information, such as ID_1, ID_2, etc., to the point cloud data to which a label has been assigned as a cable. A product-based unit may be rephrased as an item-based unit.

The identification unit 12 may assign identification information for each part to the point cloud data of labeled parts using a learning model that takes the shape, size, color, and position of each part in the monitored equipment as input and outputs parts to which identification information has been assigned. The learning model may be a trained model that has been trained in advance using the shape, size, color, and position of each part in the monitored equipment, and the identification information.

Here, the management information managed by the management unit 31 will be described with reference to Figs. 5 and 6. Fig. 5 shows management information in which part names and IDs are associated with each other. The part name may be, for example, the name of a product unit. The ID may be identification information given to each part of a product unit. Fig. 5 shows that ID_1 is associated with cable_1, and ID_2 is associated with cable_2. Furthermore, Fig. 5 shows that ID_3 is associated with insulator_1, and ID_4 is associated with insulator_2.

Figure 6 shows management information in which IDs are associated with procedure manuals. It is assumed that a procedure manual is defined for each part. Procedure manual_1 to procedure manual_4 shown in Figure 6 indicate that they are procedure manuals with different contents. Figure 7 shows that procedure manual_1 is associated with ID_1, and procedure manual_2 is associated with ID_2. Furthermore, Figure 6 shows that procedure manual_3 is associated with ID_3, and procedure manual_4 is associated with ID_4. Alternatively, the same procedure manual may be associated with ID_1 and ID_2, which are parts of the same type.

As shown in Figures 5 and 6, the management unit 31 may separately manage management information that associates part names with IDs and management information that associates IDs with instruction manuals, or may manage one piece of management information that associates part names with IDs and instruction manuals. Furthermore, the management unit 31 may manage data that indicates instruction manuals.

The management unit 31 may, for example, manage the management information using a database. The management unit 31 may, for example, record or store the management information in a memory or the like within the management device 30. Managing may also be rephrased as recording, storing, or holding, etc.

Returning to FIG. 4, the display control unit 13 displays monitored equipment including parts with IDs in the equipment data display area 21. The input unit 14 accepts the selection of the parts with IDs. For example, the worker may select the parts by tapping the equipment data display area 21 displayed on the tablet terminal. Alternatively, the worker may select the parts by operating a VR controller. Alternatively, the worker may select the parts using a mouse, which is an input device.

The input unit 14 outputs the ID assigned to the selected part to the display control unit 13. The display control unit 13 uses management information managed in the management unit 31 to identify the procedure manual associated with the received ID. The display control unit 13 displays the identified procedure manual in the procedure display area 22. The display control unit 13 may display the identified procedure manual in a distinguished manner from other procedure manuals. For example, the display control unit 13 may display the identified procedure manual in an emphasized manner.

The input unit 14 may also receive input of instruction information for rotating the display of the monitored equipment before receiving the selection of a part. For example, the input unit 14 may receive information indicating the amount of rotation along the X-axis, Y-axis, and Z-axis.

Here, the information displayed in the equipment data display area 21 and the procedure display area 22 will be described with reference to FIG. 7. The equipment data display area 21 in FIG. 7 shows point cloud data representing the monitored equipment, and the X-axis, Y-axis, and Z-axis. For example, the worker may rotate the monitored equipment by selecting the arrows shown in dotted lines on the X-axis, Y-axis, and Z-axis. For example, the worker may determine the amount of rotation around each axis by tracing the arrows shown in dotted lines on the X-axis, Y-axis, and Z-axis on the screen. The monitored equipment displayed in the equipment data display area 21 is displayed rotated according to the amount of rotation of each axis input by the worker. Alternatively, the input unit 14 may receive input from the worker of information specifying the rotation axis from the X-axis, Y-axis, and Z-axis, and information indicating the amount of rotation about the specified rotation axis. An area for inputting the rotation axis and the amount of rotation may be provided at any position in the display unit 20.

The procedure display area 22 in Figure 7 contains multiple procedure manuals.

Next, the display of the procedure manual associated with the selected part will be described using FIG. 8. In FIG. 8, the arrow 25 in the equipment data display area 21 indicates the part selected by the worker. Specifically, in FIG. 8, it is assumed that the cable indicated by the tip of the arrow 25 has been selected by the worker.

When the cable indicated at the tip of the arrow 25 displayed in the equipment data display area 21 is selected, the procedure manual associated with the ID of the selected cable is displayed in the procedure display area 22. Only the procedure manual associated with the ID of the selected cable is displayed in the procedure display area 22. In other words, by displaying only the procedure manual associated with the ID of the selected cable in the procedure display area 22, the procedure manual associated with the ID of the selected cable is emphasized. The procedure manual displayed in the procedure display area 22 is not limited to the display in FIG. 8. For example, multiple procedure manuals may be displayed in the procedure display area 22, and the procedure manual associated with the ID of the selected cable may be displayed with emphasis so as to be distinguished from the other procedure manuals by color or size, etc.

Alternatively, as shown in FIG. 9, the procedure manual associated with the ID of the selected cable may be displayed in an enlarged manner. In other words, when a part is selected in the equipment data display area 21, the equipment data display area 21 may be reduced and the procedure display area 22 may be enlarged.

Next, the flow of the display process in the management device 30 will be described with reference to FIG. 10. First, the acquisition unit 11 acquires point cloud data representing the monitored equipment (S21). When a LiDAR device is used as the acquisition unit 11, acquiring the point cloud data may mean generating the point cloud data.

Next, the identification unit 12 divides the point cloud data into a plurality of parts constituting the monitored equipment (S22). For example, the identification unit 12 may divide the point cloud data indicating the monitored equipment into a plurality of parts constituting the monitored equipment using a learning model or semantic segmentation. Dividing into a plurality of parts may mean identifying the type of part of each point included in the point cloud data.

Next, the identification unit 12 assigns identification information to the divided parts (S23). For example, the identification unit 12 further assigns identification information to each part to the point cloud data to which a label indicating the type of part has been assigned by semantic segmentation. For each part, for example, it may be rephrased as each product of the part.

Next, the display control unit 13 displays the point cloud data indicating the monitored equipment in a rotatable manner in the equipment data display area 21. Furthermore, the display control unit 13 displays a procedure manual for the monitored equipment in the procedure display area 22 (S24). The equipment data display area 21 may further display an X-axis, a Y-axis, and a Z-axis. The worker may input information indicating the amount of rotation of the X-axis, Y-axis, and Z-axis displayed in the equipment data display area 21. According to the information input by the worker, the point cloud data indicating the monitored equipment is rotated and displayed in the equipment data display area 21.

Next, the input unit 14 accepts the selection of at least one part displayed in the equipment data display area 21 (S25). Each part is assigned identification information. For example, the worker may select a part displayed in the equipment data display area 21 by a tap operation or the like. The input unit 14 receives the operation information by the worker as input information.

The display control unit 13 identifies the procedure manual associated with the selected part using the management information managed in the management unit 31 (S26). The display control unit 13 may identify the procedure manual associated with the identification information assigned to the selected part from a database managed in the management unit 31. Furthermore, the display control unit 13 may receive data of the identified procedure manual from the management unit 31.

Next, the display control unit 13 highlights and displays the identified procedure manual in the procedure display area 22 of the display unit 20 (S27). To highlight and display the identified procedure manual may, for example, be displayed larger than the other procedure manuals or may be displayed using a different color. Alternatively, to highlight and display the identified procedure manual may mean to display only the identified procedure manual.

As described above, the management device 30 can rotatably display the point cloud data of the monitored equipment in the equipment data display area 21. This allows the worker to select parts of the monitored equipment that cannot be seen from the current position in the equipment data display area 21.

Furthermore, the management device 30 highlights and displays the procedure manual associated with the part selected in the equipment data display area 21. This allows the worker to easily understand the procedure that should be performed.

Furthermore, in the second embodiment, an example is described in which a procedure associated with at least one part selected in the equipment data display area 21 is highlighted and displayed in the procedure display area 22, but this is not limited to this. For example, a part associated with at least one procedure selected in the procedure display area 22 may be highlighted and displayed in the equipment data display area 21.

FIG. 11 is a block diagram showing an example configuration of a management device 10 and a management device 30 (hereinafter referred to as the management device 10, etc.). Referring to FIG. 11, the management device 10, etc. includes a network interface 1201, a processor 1202, and a memory 1203. The network interface 1201 may be used to communicate with a network node. The network interface 1201 may include, for example, a network interface card (NIC) that complies with the IEEE 802.3 series. IEEE stands for Institute of Electrical and Electronics Engineers.

The processor 1202 reads out and executes software (computer programs) from the memory 1203 to perform the processing of the RU device 10 and the like described using the flowcharts in the above-mentioned embodiment. The processor 1202 may be, for example, a microprocessor, an MPU, or a CPU. The processor 1202 may include multiple processors.

Memory 1203 is composed of a combination of volatile memory and non-volatile memory. Memory 1203 may include storage located away from processor 1202. In this case, processor 1202 may access memory 1203 via an I/O (Input/Output) interface not shown.

In the example of FIG. 11, the memory 1203 is used to store a group of software modules. The processor 1202 can read and execute these software modules from the memory 1203 to perform the processing of the management device 10 and the like described in the above embodiment.

As explained using FIG. 11, each of the processors in the management device 10 executes one or more programs that include a set of instructions for causing a computer to execute the algorithm explained using the drawing.

In the above examples, the program includes instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more functions described in the embodiments. The program may be stored on a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, computer-readable media or tangible storage media include random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technology, CD-ROM, digital versatile disc (DVD), Blu-ray® disk or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device. The program may be transmitted on a transitory computer-readable medium or communication medium. By way of example and not limitation, the transitory computer-readable medium or communication medium includes electrical, optical, acoustic, or other forms of propagated signals.

Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-mentioned embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

Each drawing is merely an example for describing one or more embodiments. Each drawing may relate not only to one particular embodiment, but to one or more other embodiments. As will be appreciated by those skilled in the art, various features or steps described with reference to any one drawing may be combined with features or steps shown in one or more other figures to create, for example, an embodiment not explicitly shown or described. Not all features or steps shown in any one drawing are necessary to describe an exemplary embodiment, and some features or steps may be omitted. The order of steps described in any drawing may be changed as appropriate.

A part or all of the above-described embodiments can be described as, but is not limited to, the following supplementary notes.
(Appendix 1)
An acquisition unit that acquires three-dimensional data including a monitored facility;
An identification unit that identifies a plurality of components that configure the monitored equipment;
a display control unit that rotatably displays the monitored equipment in an equipment data display area of a display unit, and displays a plurality of procedures to be performed by an operator regarding the monitored equipment in a procedure display area of the display unit;
an input unit that accepts a selection of at least one part to be displayed in the equipment data display area or at least one procedure to be displayed in the procedure display area,
The display control unit is
A management device that highlights and displays the procedure associated with the at least one selected part, or the part associated with the at least one selected procedure.
(Appendix 2)
The input unit includes:
A management device as described in Appendix 1, which accepts selection of a viewpoint direction of the monitored equipment to rotate the monitored equipment displayed in the equipment data display area.
(Appendix 3)
The input unit includes:
The management device according to claim 1 or 2, further comprising: a step of receiving an input of a scroll operation of a plurality of procedures to select one of the procedures displayed in the procedure display area.
(Appendix 4)
The display control unit is
The management device according to any one of claims 1 to 3, wherein the equipment data display area and the procedure display area are displayed on a single screen.
(Appendix 5)
The display control unit is
A management device described in any one of appendices 1 to 4, wherein the procedure display area displaying the procedure associated with the at least one selected part is displayed larger than the equipment data display area, or the equipment data display area displaying the part associated with the at least one selected procedure is displayed larger than the procedure display area.
(Appendix 6)
The management device according to any one of claims 1 to 5, further comprising a management unit that manages the part identification information and the procedure in association with each other.
(Appendix 7)
The identification unit is
Identifying an identity for each of said components;
The display control unit is
The management device according to claim 6, further comprising: a step of extracting the procedure managed in association with the identification information of the selected at least one part, based on the identification information of the at least one part.
(Appendix 8)
A display device that displays an equipment data display area including a monitored equipment and a procedure display area including a plurality of procedures to be performed by an operator regarding the monitored equipment;
a display control unit that rotatably displays the monitored equipment in the equipment data display area of the display device and displays a plurality of procedures to be performed by an operator related to the monitored equipment in the procedure display area of the display device; and an input unit that accepts selection of at least one part to be displayed in the equipment data display area or at least one procedure to be displayed in the procedure display area, wherein the display control unit highlights the procedure associated with the selected at least one part, or the part associated with the selected at least one procedure.
(Appendix 9)
Acquire 3D data including the monitored equipment,
Identifying a plurality of components constituting the monitored equipment;
The monitored equipment is rotatably displayed in an equipment data display area of a display unit,
displaying a plurality of procedures to be performed by an operator regarding the monitored equipment in a procedure display area of the display unit;
Accepting a selection of at least one component to be displayed in the monitored equipment data display area or at least one procedure to be displayed in the procedure display area;
A display method for highlighting and displaying the procedure associated with the at least one selected part, or the part associated with the at least one selected procedure.
(Appendix 10)
Acquire 3D data including the monitored equipment,
Identifying a plurality of components constituting the monitored equipment;
The monitored equipment is rotatably displayed in an equipment data display area of a display unit,
displaying a plurality of procedures to be performed by an operator regarding the monitored equipment in a procedure display area of the display unit;
Accepting a selection of at least one component to be displayed in the monitored equipment data display area or at least one procedure to be displayed in the procedure display area;
A program that causes a computer to highlight and display the procedure associated with the at least one selected part, or the part associated with the at least one selected procedure.

Some or all of the elements (e.g., configurations and functions) described in Supplementary Notes 2 to 7 that are dependent on Supplementary Note 1 may also be dependent on Supplementary Notes 8 to 10 in a similar dependent relationship to Supplementary Notes 2 to 7. Some or all of the elements described in any of the supplementary notes may be applied to various hardware, software, recording means for recording software, systems, and methods.

REFERENCE SIGNS LIST 10 Management device 11 Acquisition unit 12 Identification unit 13 Display control unit 14 Input unit 20 Display unit 21 Equipment data display area 22 Procedure display area 30 Management device 31 Management unit

Claims (9)

an acquisition unit that acquires point cloud data as three-dimensional data including the monitored facility;
an identification unit that identifies the plurality of parts by assigning labels indicating names of the plurality of parts that constitute the monitored facility to each point that constitutes the point cloud data using semantic segmentation;
a display control unit that rotatably displays the monitored equipment in an equipment data display area of a display unit, and displays a plurality of procedures to be performed by an operator regarding the monitored equipment in a procedure display area of the display unit;
an input unit that accepts a selection of at least one part to be displayed in the equipment data display area or at least one procedure to be displayed in the procedure display area,
The display control unit is
highlighting the steps associated with the at least one selected part or the parts associated with the at least one selected steps;
The input unit is
In order to rotate the monitored equipment displayed in the equipment data display area, an operator can operate an arrow indicated on the rotation axis of the monitored equipment, thereby selecting the rotation axis of the monitored equipment and adjusting the amount of rotation.
Management device. The input unit is
The management device according to claim 1 , further comprising: a step of receiving an input of a scroll operation of a plurality of procedures displayed in the procedure display area so as to select one of the procedures. The display control unit is
The management device according to claim 1 , wherein the facility data display area and the procedure display area are displayed on a single screen. The display control unit is
The management device according to claim 1 or 2, wherein the procedure display area displaying the procedure associated with the at least one selected part is displayed larger than the equipment data display area, or the equipment data display area displaying the part associated with the at least one selected procedure is displayed larger than the procedure display area. The management device according to claim 1 or 2, further comprising a management unit that manages the part identification information and the procedure in association with each other. The identification unit is
Identifying an identity for each of said components;
The display control unit is
6. The management device according to claim 5 , further comprising: a step of extracting the procedure managed in association with the identification information of the selected at least one part, based on the identification information of the at least one part. A display device that displays an equipment data display area including a monitored equipment and a procedure display area including a plurality of procedures to be performed by an operator regarding the monitored equipment;
a display control unit that rotatably displays the monitored equipment in the equipment data display area of the display device and displays a plurality of procedures to be performed by an operator for the monitored equipment in the procedure display area of the display device; and an input unit that accepts selection of at least one part displayed in the equipment data display area or at least one procedure displayed in the procedure display area, wherein the display control unit highlights the procedure associated with the at least one selected part, or the part associated with the at least one selected procedure, and the input unit allows an operator to operate an arrow indicated on the rotation axis of the monitored equipment to rotate the monitored equipment displayed in the equipment data display area, thereby allowing the selection of the axis of rotation of the monitored equipment and the adjustment of the amount of rotation . Acquire point cloud data as 3D data including the monitored equipment,
identifying the plurality of parts constituting the monitored facility by assigning labels indicating the names of the plurality of parts constituting the monitored facility to each point constituting the point cloud data using semantic segmentation;
The monitored equipment is rotatably displayed in an equipment data display area of a display unit,
displaying a plurality of procedures to be performed by an operator regarding the monitored equipment in a procedure display area of the display unit;
In order to rotate the monitored equipment displayed in the equipment data display area, an operator can operate an arrow indicated on the rotation axis of the monitored equipment, thereby selecting the rotation axis of the monitored equipment and adjusting the amount of rotation;
Accepting a selection of at least one part to be displayed in the equipment data display area or at least one procedure to be displayed in the procedure display area;
A display method for highlighting and displaying the procedure associated with the at least one selected part, or the part associated with the at least one selected procedure. Acquire point cloud data as 3D data including the monitored equipment,
identifying the plurality of parts constituting the monitored facility by assigning labels indicating the names of the plurality of parts constituting the monitored facility to each point constituting the point cloud data using semantic segmentation;
The monitored equipment is rotatably displayed in an equipment data display area of a display unit,
displaying a plurality of procedures to be performed by an operator regarding the monitored equipment in a procedure display area of the display unit;
In order to rotate the monitored equipment displayed in the equipment data display area, an operator can operate an arrow indicated on the rotation axis of the monitored equipment, thereby selecting the rotation axis of the monitored equipment and adjusting the amount of rotation;
Accepting a selection of at least one part to be displayed in the equipment data display area or at least one procedure to be displayed in the procedure display area;
A program that causes a computer to highlight and display the procedure associated with the at least one selected part, or the part associated with the at least one selected procedure.

Images