JP7826751B2 - Information processing device, measurement device, information processing method, and information processing program - Google Patents

Description

The present invention relates to an information processing device, a measurement device, an information processing method, and an information processing program.

In recent years, laser scanners, LiDAR, and other technologies have been used to acquire three-dimensional information about real-world three-dimensional space. Three-dimensional information is used in industries such as construction, facility maintenance, and real estate for tasks such as visually confirming locations on a screen while checking dimensions. Prior art discloses a system that displays a two-dimensional image of the three-dimensional space represented by the three-dimensional information viewed from a virtual viewpoint of a virtual camera, and measures distance and area by accepting input of multiple measurement points from the user via the display screen of the two-dimensional image.

However, with conventional technology, if the position of the measurement target in three-dimensional space represented by three-dimensional information is in the blind spot of the virtual camera, the position of the measurement target is not displayed on the screen, making it difficult for the user to specify the desired position.

The present invention has been made in consideration of the above, and aims to provide an information processing device, a measurement device, an information processing method, and an information processing program that enable easy specification of the position of a measurement target within a three-dimensional space represented by three-dimensional information.

In order to solve the above-mentioned problems and achieve the object, the present invention provides an information processing device that displays on a display unit a two-dimensional image of a three-dimensional space represented by three-dimensional information viewed from a virtual viewpoint of a virtual camera, and includes: a derivation unit that acquires first position information related to a first position in the three-dimensional space through input by a user to an input unit, and calculates position information related to a second position based on the first position information; and a display control unit that displays the two-dimensional image including the first position and the second position on the display unit.

The present invention has the advantage of making it possible to easily specify the position of a measurement target within a three-dimensional space represented by three-dimensional information.

FIG. 1 is a block diagram of an example of a measurement system according to this embodiment. FIG. 2 is a schematic diagram showing an example of a two-dimensional image displayed on the display unit. FIG. 3 is a schematic diagram showing an example of a scene in which the user specifies two measurement points. FIG. 4 is a schematic diagram showing an example of a scene in which a two-dimensional image including a blind spot is displayed. FIG. 5 is a schematic diagram of a three-dimensional space represented by three-dimensional information. FIG. 6 is an explanatory diagram of an example of calculating position information of a candidate position. FIG. 7 is an explanatory diagram of an example of generating a two-dimensional image in which the position of the virtual viewpoint is moved. FIG. 8 is a schematic diagram showing an example of a scene in which both the first two-dimensional image and the second two-dimensional image are displayed on the display unit. FIG. 9 is an explanatory diagram of an example of generating a two-dimensional image in which obstacles are not displayed. FIG. 10 is a flowchart illustrating an example of the flow of information processing executed by the information processing apparatus of the embodiment. FIG. 11 is a schematic diagram showing an example of a measurement system according to a modified example. FIG. 12 is a diagram showing the hardware configuration.

Embodiments of an information processing device, a measurement device, an information processing method, and an information processing program are described in detail below with reference to the accompanying drawings.

Figure 1 is a block diagram of an example of a measurement system 1 according to this embodiment.

The measurement system 1 includes an information processing device 10 and an imaging device 11. The information processing device 10 and the imaging device 11 are connected so that they can communicate with each other.

The imaging device 11 is a device that obtains three-dimensional information of a three-dimensional space, which is real space. In this embodiment, a device that obtains three-dimensional information by photographing is used as the imaging device 11. The imaging device 11 is, for example, a ToF (Time-of-Flight) camera or a stereo camera. The ToF method irradiates an object with infrared light and determines distance from the time it takes for the reflected light to return. A stereo camera is a camera that obtains depth information as distance information using the distance between two cameras and the parallax information of the images obtained by each of the two cameras.

The image capture device 11 may be any device capable of obtaining three-dimensional information, such as a system using LiDAR (Light Detection and Ranging) or photogrammetry. The image capture device 11 may also be a smartphone equipped with a laser scanner or LiDAR.

Three-dimensional information is information that represents a three-dimensional space, which is a real space. The file format of the three-dimensional information is not limited. Three-dimensional information is represented, for example, by shape data that represents the three-dimensional shape of a three-dimensional object contained in real space. Three-dimensional information is, for example, a point cloud format file that represents three-dimensional space using discrete points, or a polygon mesh format file that represents three-dimensional space using vertices and faces. Point cloud format files are sometimes called depth maps, distance images, etc.

Examples of point cloud files include files with extensions such as ".xyz", ".e57", and ".ply". Examples of polygon mesh files include files with extensions such as ".obj", ".fbx", and ".stl".

The imaging device 11 outputs the obtained three-dimensional information to the information processing device 10.

The information processing device 10 is an information processing device that displays a two-dimensional image of a three-dimensional space represented by three-dimensional information viewed from a virtual viewpoint of a virtual camera. The information processing device 10 is, for example, a smartphone, a tablet terminal, a personal computer, etc.

The information processing device 10 includes a communication unit 12, a UI (user interface) unit 14, a storage unit 16, and a control unit 20. The communication unit 12, the UI unit 14, the storage unit 16, and the control unit 20 are connected to each other so that they can communicate with each other.

The communication unit 12 communicates with an external information processing device via a network or the like. In this embodiment, the communication unit 12 communicates with the imaging device 11.

The UI unit 14 includes a display unit 14A and an input unit 14B. The display unit 14A is a display that displays various types of information. The input unit 14B accepts operational instructions from the user. The input unit 14B is, for example, a keyboard, a pointing device, or a mouse. The display unit 14A and the input unit 14B may be an integrated touch panel. The memory unit 16 stores various types of information.

The control unit 20 executes information processing. A measurement application program is pre-installed in the control unit 20. The measurement application program displays a two-dimensional image of a three-dimensional space represented by three-dimensional information viewed from a virtual viewpoint of a virtual camera on the display unit 14A, and measures distance and area by accepting input of multiple measurement points from the user via the display surface of the two-dimensional image. Hereinafter, the measurement application program may be referred to as a measurement application.

In this embodiment, the control unit 20 has an acquisition unit 20A, a reception unit 20B, a derivation unit 20C, a display control unit 20D, and a distance calculation unit 20E. The acquisition unit 20A, the reception unit 20B, the derivation unit 20C, the display control unit 20D, and the distance calculation unit 20E are realized, for example, by one or more processors. For example, each of the above units may be realized by having a processor such as a CPU (Central Processing Unit) execute a program, i.e., by software. Each of the above units may also be realized by a processor such as a dedicated IC (Integrated Circuit), i.e., by hardware. Each of the above units may also be realized by a combination of software and hardware. When multiple processors are used, each processor may realize one of the units, or two or more of the units.

The acquisition unit 20A acquires three-dimensional information. In this embodiment, the acquisition unit 20A acquires three-dimensional information from the imaging device 11 via the communication unit 12. Note that the acquisition unit 20A may also acquire three-dimensional information from another information processing device that is communicatively connected to the information processing device 10 via the communication unit 12 and a network, etc. Examples of other information processing devices include, but are not limited to, smartphones that are capable of acquiring three-dimensional information. The acquisition unit 20A may also acquire three-dimensional information from the memory unit 16.

The three-dimensional information acquired by the acquisition unit 20A may be either monochrome information or color information including color information. The information processing device 10 uses color information as the three-dimensional information and converts the three-dimensional information into a two-dimensional image for display, thereby providing the user with a two-dimensional image that is more visually understandable.

The acquisition unit 20A outputs the acquired 3D information to the derivation unit 20C. Note that the acquisition unit 20A may convert the 3D information acquired from the imaging device 11 into any file format before outputting it to the derivation unit 20C. For example, the acquisition unit 20A may convert a point cloud format file into a polygon mesh format file using the method disclosed in Japanese Patent Application Laid-Open No. 2019-049533, and output the converted 3D information to the derivation unit 20C. This file format conversion process may also be performed by the derivation unit 20C.

The reception unit 20B receives operation instructions from the user via the input unit 14B.

Here, we will explain an example of the basic functions of a measurement application.

Figure 2 is a schematic diagram showing an example of a two-dimensional image 40A displayed on the display unit 14A. The two-dimensional image 40A is an example of the two-dimensional image 40 displayed on the display unit 14A. The two-dimensional image 40 is a two-dimensional image of the three-dimensional space S represented by the three-dimensional information 30 viewed from the virtual viewpoint of a virtual camera.

By operating the input unit 14B while viewing the displayed two-dimensional image 40, the user can freely move the virtual viewpoint in the three-dimensional space S and switch the display to a two-dimensional image 40 in which the three-dimensional space S is viewed from any virtual viewpoint.

Figure 3 is a schematic diagram showing an example of a scene in which the user has specified two measurement points Q.

Measurement point Q is a point used to specify the measurement target that the user wishes to measure, and is a point on the two-dimensional image 40 displayed on the display unit 14A.

For example, consider a scenario in which the user operates the input unit 14B to specify a first measurement point Q1 and a second measurement point Q2 as measurement points Q at any positions on the two-dimensional image 40A displayed on the display unit 14A.

The first measurement point Q1 refers to the measurement point Q that is specified first among multiple measurement points Q specified by the user. The second measurement point Q2 refers to the measurement point Q that is specified next.

In this case, the measurement application identifies a first position P1 and a second position P2 in the three-dimensional space S represented by the three-dimensional information 30, which correspond to each of the two specified measurement points Q (first measurement point Q1, second measurement point Q2). The measurement application then calculates and displays the distance between the first position P1 and the second position P2. Figure 3 shows an example in which "2.0 [m]" is displayed as the distance between the first position P1 and the second position P2 in the three-dimensional space S, which correspond to each of the two specified measurement points Q (first measurement point Q1, second measurement point Q2).

As shown in Figure 3, if the point in three-dimensional space S at which the user wishes to measure is displayed in two-dimensional image 40, the user can easily specify measurement point Q.

However, there are cases where the point in three-dimensional space S that the user wishes to measure is in the blind spot of the virtual camera. In this case, the point in three-dimensional space S that the user wishes to measure will not be displayed in the two-dimensional image 40 that is obtained by viewing the three-dimensional space S from the virtual viewpoint of the virtual camera.

Figure 4 is a schematic diagram showing an example of a scene in which a two-dimensional image 40B including a blind spot BS is displayed. Figure 4 shows an example in which the two-dimensional image 40B includes a blind spot BS caused by an obstacle 50. For example, consider a scene in which a user specifies a first measurement point Q1 and then a second measurement point Q2. Also consider a case in which the position in three-dimensional space S corresponding to second measurement point Q2 is a position that becomes a blind spot BS when the three-dimensional space S is viewed from the virtual viewpoint used when creating the two-dimensional image 40B. In this case, as shown in Figure 4, points within the blind spot BS included in the three-dimensional space S are not displayed in the two-dimensional image 40B.

In this case, with conventional technology, the user would need to operate the input unit 14B while viewing the displayed two-dimensional image 40B to move the virtual viewpoint in the three-dimensional space S, and perform viewpoint manipulation to convert the viewpoint so that the desired point is displayed within the two-dimensional image 40.

However, such viewpoint manipulation can increase operation time or result in incorrect manipulation for users unfamiliar with manipulating the three-dimensional information 30. Furthermore, users may lose sight of the desired point during manipulation. For this reason, with conventional technology, it can be difficult for users to easily specify the position of the measurement target within the three-dimensional space S represented by the three-dimensional information 30.

The control unit 20 of this embodiment therefore displays on the display unit 14A a two-dimensional image 40 of the three-dimensional space S represented by the three-dimensional information 30, viewed from the virtual viewpoint of the virtual camera. Then, when the user specifies a first measurement point Q1, which is one of multiple measurement points Q, via the two-dimensional image 40, the control unit 20 controls the display unit 14A to always display a candidate point R, which is a candidate for the second measurement point Q2, which is another measurement point Q. Therefore, with the information processing device 10 of this embodiment, the user can complete measurements of the distance and area between multiple measurement points Q with minimal or no viewpoint manipulation of the virtual viewpoint.

The derivation unit 20C, display control unit 20D, and distance calculation unit 20E included in the control unit 20 will be described in detail below.

Figure 5 is a schematic diagram of the three-dimensional space S represented by the three-dimensional information 30.

The derivation unit 20C acquires first position information relating to a first position P1 in the three-dimensional space S represented by the three-dimensional information 30 acquired by the acquisition unit 20A through user input to the input unit 14B, and acquires position information relating to a second position P2 from the user input to the input unit 14B or calculates it based on the first position P1.

The first position P1 represents a position in the three-dimensional space S represented by the three-dimensional information 30 that corresponds to the first measurement point Q1 specified by the user via the displayed two-dimensional image 40.

Position information related to the first position P1 is information that represents the first position P1 in the three-dimensional information 30. The position information related to the first position P1 is represented, for example, by position coordinates in the three-dimensional space S represented by the three-dimensional information 30.

The second position P2 represents a position in the three-dimensional space S represented by the three-dimensional information 30 that corresponds to the second measurement point Q2, which is specified by the user via the displayed two-dimensional image 40 after the first measurement point Q1, and the candidate point R for the second measurement point Q2. The candidate point R is a location that is a candidate for the second measurement point Q2. When the user inputs an instruction to confirm the candidate point R by operating the input unit 14B, the candidate point R is confirmed as the second position P2 (details will be described later).

The position information related to the second position P2 is information representing the second position P2 in the three-dimensional information 30 and the candidate position CP of the candidate point R. The candidate position CP represents a position in the three-dimensional space S represented by the three-dimensional information 30, which corresponds to the candidate point R displayed in the two-dimensional image 40. In other words, when the user inputs an instruction to confirm the candidate point R by operating the input unit 14B, the position information of the candidate position CP corresponding to the candidate point R is confirmed as the position information related to the second position P. The position information related to the second position P2 is represented, for example, by position coordinates in the three-dimensional space S represented by the three-dimensional information 30.

When the derivation unit 20C receives the three-dimensional information 30 from the acquisition unit 20A, it determines a first virtual viewpoint VP1, which is a virtual viewpoint serving as an initial position, and a viewing direction for viewing the three-dimensional space S from the first virtual viewpoint VP1. For example, the storage unit 16 pre-stores initial viewpoint information representing the first virtual viewpoint VP1 and the initial direction. The derivation unit 20C reads the initial viewpoint information stored in the storage unit 16 and determines the first virtual viewpoint VP1 and initial direction represented by the read initial viewpoint information.

The derivation unit 20C then generates a two-dimensional image 40 in which the three-dimensional space S represented by the three-dimensional information 30 received from the acquisition unit 20A is viewed from the first virtual viewpoint VP1 in the initial direction. The display control unit 20D displays the two-dimensional image 40 generated by the derivation unit 20C on the display unit 14A. Therefore, for example, the two-dimensional image 40A shown in FIG. 2 is displayed on the display unit 14A.

The user specifies the first measurement point Q1 by operating the input unit 14B while viewing the two-dimensional image 40A displayed on the display unit 14A. In response to the user's specification, the reception unit 20B outputs information representing the position of the specified first measurement point Q1 in the displayed two-dimensional image 40A to the derivation unit 20C.

When the derivation unit 20C receives information representing the position of the first measurement point Q1 in the two-dimensional image 40A, it derives first position information relating to the first position P1, which is the position corresponding to the first measurement point Q1 in the three-dimensional space S represented by the three-dimensional information 30, using a known coordinate transformation process.

The derivation unit 20C may perform coordinate conversion using a known coordinate conversion method, such as ray casting, which converts the screen coordinates of the display unit 14A into coordinates in the three-dimensional space S represented by the three-dimensional information 30.

Through these processes, the derivation unit 20C obtains first position information related to a first position P1 in the three-dimensional space S represented by the three-dimensional information 30 through input by the user to the input unit 14B.

The derivation unit 20C derives a position Q1' corresponding to the first position P1 in the two-dimensional image 40B. The position Q1' in the two-dimensional image 40B corresponding to the first position P1 is a position obtained by converting the first position P1, which is a coordinate in the three-dimensional space S, into a position in the two-dimensional image 40B viewed from the virtual viewpoint VP1.

The display control unit 20D displays first position identification information, which is information identifying the first measurement point Q1, at position Q1' in the two-dimensional image 40B. The first position identification information is displayed as a circular image hatched with diagonal lines, which indicates the first measurement point Q1 in Figures 3 and 4. The first position identification information is not limited to a circular image, as long as it allows the position Q1' corresponding to the first position P1 to be identified in the two-dimensional image 40B. Other examples of the first identification information include a symbol (such as an arrow or polygonal icon), text information indicating the first position P1 (for example, "Measurement Point 1"), or a combination of a symbol and text information.

When the virtual viewpoint is moved by a user's input to the input unit 14B, the display control unit 20D updates the position of the first identification information in the two-dimensional image 40B in the aforementioned step in accordance with the changes accompanying the movement of the virtual viewpoint in the two-dimensional image 40B.

When a first measurement point Q1 is specified via the displayed two-dimensional image 40, the derivation unit 20C derives position information related to a second position P2 corresponding to a second measurement point Q2. In this embodiment, the derivation unit 20C derives position information related to the second position P2, that is, position information of a candidate position CP in three-dimensional space S corresponding to a candidate point R for the second measurement point Q2 on the two-dimensional image 40.

As described above, candidate point R is a point on the two-dimensional image 40 that is a candidate for the second measurement point Q2. Furthermore, candidate position CP represents a position in the three-dimensional space S represented by the three-dimensional information 30 that corresponds to candidate point R displayed on the two-dimensional image 40. In other words, candidate position CP is a position that is a candidate for the second position P2.

The derivation unit 20C derives the position information of the candidate position CP by either acquiring it from the user's input to the input unit 14B or calculating it based on the first position information related to the first position P1.

For example, the derivation unit 20C determines a method for deriving the position information of the candidate position CP based on the input form entered by the user via the input unit 14B.

For example, consider a case where the user's input to the input unit 14B is a pointing device such as a mouse, or where the input is a drag operation on a touch panel, and the pointer position PP on the display unit 14A displaying the two-dimensional image 40 is clear. A case where the pointer position PP is clear means that the pointer position PP on the display unit 14A has been accepted by the accepting unit 20B from the input unit 14B.

In this case, the derivation unit 20C derives the position information of the candidate position CP by obtaining it from the user's input to the input unit 14B.

In detail, the derivation unit 20C identifies a pointer position PP on the two-dimensional image 40 that is specified by a user's operation using the input unit 14B. The derivation unit 20C sets the identified pointer position PP as a candidate point R, and identifies a position in the three-dimensional space S that corresponds to the candidate point R as a candidate position CPa. The candidate position Cpa is an example of a candidate position CP. The derivation unit 20C then derives position information for the candidate position CPa in the three-dimensional space S.

Through these processes, the derivation unit 20C obtains the position information of the candidate position CP, which is position information related to the second position P2, from the user's input to the input unit 14B.

The pointer position PP, which is input by the user to the input unit 14B, is always displayed on the display unit 14A. Therefore, in this case, the derivation unit 20C can derive the position information of the candidate position CP, which is position information related to the second position P2, in real time. Therefore, in this case, the user can specify the candidate position CP more intuitively and simply.

The derivation unit 20C may also set as the candidate position CP any point on an extension of the line L1 connecting the first position P1 and the position in the three-dimensional space S corresponding to the pointer position PP. Figure 5 shows a case where the candidate position CPb, which is any point on the line L1, is set as the candidate position CP. In this case, the derivation unit 20C derives the position information of the candidate position CPb in the three-dimensional space S.

On the other hand, consider a case where the input form of the user on the input unit 14B does not satisfy the above conditions. A case where the above conditions are not satisfied is, for example, a case where the input form of the user on the input unit 14B is a touch operation on the UI unit 14, which is a touch panel, and the pointer position PP is not clear.

In this case, the derivation unit 20C derives the position information of the candidate position CP by calculating it based on the first position information.

Figure 6 is an explanatory diagram of an example of calculating position information for a candidate position CP. Figure 6 shows a schematic diagram of a three-dimensional space S represented by three-dimensional information 30.

In this case, the derivation unit 20C derives position information of the candidate position CP based on the virtual viewpoint VP of the two-dimensional image 40 and the position information of the first position P1.

In detail, the derivation unit 20C identifies a straight line L2, which is a line segment connecting the virtual viewpoint VP and the first position P1. The derivation unit 20C then identifies any position within a cylindrical region 56 of a predetermined radius with the straight line L2 as its central axis as a candidate position CP. Figure 6 shows an example of a scene in which the derivation unit 20C has identified a candidate position CPc within the cylindrical region 56. The candidate position CPc is an example of a candidate position CP.

The derivation unit 20C identifies an area BS2 within the cylindrical area 56 that overlaps with the blind spot BS of the virtual camera 52 positioned at the virtual viewpoint VP, and identifies any position within this overlapping area as a candidate position CPc. The derivation unit 20C may also identify the center of gravity position within the overlapping area BS2 as a candidate position CPc.

Here, the blind spot BS in Figure 6 will be explained. The space FV within the field of view of the virtual camera in the three-dimensional space S has a cone shape with the virtual viewpoint VP of the virtual camera as its apex and the line of sight as its central axis. The space FV within the field of view in Figure 6 is an illustration of its cross-sectional shape. The blind spot BS in Figure 6 is a region of the space FV within the field of view of the virtual camera 52 that is hidden from view due to an obstacle 50. Note that known technology can be used to identify the blind spot BS. By setting a candidate position CPc within the region BS2 where the cylindrical region 56 and the blind spot BS overlap, the user can view the blind spot region that is not visible to the user in the two-dimensional image 40 from the virtual viewpoint VP using the processing described below. By setting the candidate position CPc to the overlapping region BS2, it can be set as a candidate position that is not displayed in the two-dimensional image 40 from the virtual viewpoint VP2 and that is likely to be designated as the second position.

The derivation unit 20C then derives the position information of the identified candidate position CP by identifying it in the three-dimensional space S represented by the three-dimensional information 30. In other words, through these processes, the derivation unit 20C calculates the position information of the candidate position CP, which is position information related to the second position P2, based on the position information of the first position P1.

It is preferable that the radius of the cylindrical region 56 is greater than the predicted distance between the first position P1 and the second position P2, for example. When the radius of the cylindrical region 56 satisfies this condition, the candidate point R corresponding to the candidate position CP is displayed visibly within the two-dimensional image 40. Therefore, by visually viewing the two-dimensional image 40, the user can designate the candidate point R displayed on the two-dimensional image 40 as the second measurement point Q2.

The derivation unit 20C can pre-calculate a value that satisfies the above conditions as the radius of the cylindrical region 56 and store it in advance in the memory unit 16.

For example, the derivation unit 20C calculates the radius of the cylindrical region 56 in advance using the maximum and average dimensions of the three-dimensional object OB included in the three-dimensional information 30. The derivation unit 20C also calculates the radius of the cylindrical region 56 in advance based on the acquisition environment of the three-dimensional information 30 acquired by the acquisition unit 20A. For example, assume that the acquisition environment of the three-dimensional information 30 is an office environment that includes one or more pieces of furniture as three-dimensional objects OB. In this case, the derivation unit 20C may acquire in advance the maximum and average dimensions of the three-dimensional objects OB that may be included in the office environment using a known method, and use these dimensions to calculate the radius of the cylindrical region 56. The derivation unit 20C may acquire the maximum and average dimensions of the three-dimensional objects OB from specification information or standard information that specifies the standards for the three-dimensional objects OB that may be included in the office environment.

The derivation unit 20C may calculate the radius of the cylindrical region 56 based on the selected standard information from the standard information corresponding to each of the multiple acquisition environments of the three-dimensional information 30. As another example of the acquisition environment and standard information, if the acquisition environment is a construction site such as a house or building, the standard information may be standard information used for building materials and construction methods. The user may also specify the acquisition environment or standard information in advance. The user may input the desired acquisition environment or standard information into the input unit 14B, and the derivation unit 20C may calculate the radius of the cylindrical region 56 based on the acquisition environment or standard information entered by the user. The user may also input the radius of the cylindrical region 56 itself in advance. By the user specifying the acquisition environment, standard information, radius, etc., the candidate point R can be more accurately displayed on the two-dimensional image 40. The standard information may be stored in a database by the information processing device (e.g., ROM 90B in FIG. 12) or may be acquired from an external database.

The derivation unit 20C may also calculate the radius of the cylindrical region 56 based on the two-dimensional image 40. For example, the derivation unit 20C may set the range of the first measurement point Q1, which corresponds to 1/4 of the image size of the two-dimensional image 40 when displayed, as the radius of the cylindrical region 56. In this case, the derivation unit 20C can identify the region around the first measurement point Q1 in the two-dimensional image 40, which corresponds to approximately 1/2 of the image size, as the candidate position CP.

The derivation unit 20C also prepares, as training data, a combination of the three-dimensional information 30 and the radius of the cylindrical region 56 used when identifying the candidate position CP for the three-dimensional information 30. The derivation unit 20C may then derive the radius of the cylindrical region 56 from the three-dimensional information 30 using a learning model that has been machine-learned in advance using multiple pieces of training data. The learning of the learning model may be performed by the control unit 20, or may be performed by an external information processing device connected via the communication unit 12.

It should be noted that the three-dimensional information 30 may include information related to the three-dimensional object OB. That is, the three-dimensional information 30 may be information segmented for each three-dimensional object OB. It is also assumed that the first position P1 corresponding to the first measurement point Q1 input by the user through operation of the input unit 14B is a position on the three-dimensional object OB.

In this case, it is preferable that the derivation unit 20C set the candidate position CP for the second position P2 at a position on the three-dimensional object OB that is the farthest from the first position P1. In this case, the derivation unit 20C may also set the candidate position CP at a position on the three-dimensional object OB that is the vertex of the three-dimensional object OB that is the farthest from the first position P1.

In detail, the derivation unit 20C identifies a three-dimensional object OB at which a first position P1 exists in the three-dimensional space S represented by the three-dimensional information 30, corresponding to a first measurement point Q1 specified by a user's input to the input unit 14B. The derivation unit 20C then identifies a blind spot BS on the identified three-dimensional object OB from a first virtual viewpoint VP, which is the virtual viewpoint VP of the two-dimensional image 40 used by the user when specifying the first measurement point Q1. The derivation unit 20C may set a candidate position CP at the center of gravity position within the identified blind spot BS, or at the position of the vertex on the three-dimensional object OB that is farthest from the first position P1.

When the derivation unit 20C sets the candidate position CP at the position of the vertex on the three-dimensional object OB that is farthest from the first position P1, the display control unit 20D (described later) can display a two-dimensional image 40 including the entire three-dimensional object OB where the first measurement point Q1 is located on the display unit 14A. This allows the display control unit 20D to display a two-dimensional image 40 from which the second measurement point Q2 can be easily selected.

Through the above processing, the derivation unit 20C derives the position information of the candidate position CP, which is position information related to the second position P2, by obtaining it from the user's input to the input unit 14B or by calculating it based on the first position information.

Here, if the candidate position CP derived by the derivation unit 20C is located in the blind spot BS of the virtual camera 52 placed at the first virtual viewpoint VP1, the candidate point R of the candidate position CP will not be displayed in the two-dimensional image 40 of the three-dimensional space S viewed from the first virtual viewpoint VP1.

Therefore, the derivation unit 20C changes the display format of the two-dimensional image 40 so that both the first position P1 and the candidate position CP of the second position P2 are visible within the two-dimensional image 40. In other words, the derivation unit 20C changes the display format of the two-dimensional image 40 so that the candidate point R corresponding to the candidate position CP is always visible within the two-dimensional image 40.

In this embodiment, the derivation unit 20C changes the display format by either generating a two-dimensional image 40 by moving the virtual viewpoint VP, or by hiding obstacles 50 that form blind spots BS.

First, we will explain the case where a two-dimensional image 40 is generated by moving the position of the virtual viewpoint VP so that both the first position P1 and the candidate position CP for the second position P2 are visible within the two-dimensional image 40.

Figure 7 is an explanatory diagram of an example of generating a two-dimensional image 40 in which the position of the virtual viewpoint VP has been moved. Figure 7 also shows a schematic diagram of a three-dimensional space S represented by three-dimensional information 30.

For example, consider a situation in which, before the derivation unit 20C acquires the second position P2, the display control unit 20D is displaying a two-dimensional image 40 (corresponding to the first two-dimensional image) on the display unit 14A, with the first virtual viewpoint VP1 as the virtual viewpoint VP. In other words, consider a situation in which the virtual viewpoint VP of the two-dimensional image 40 used by the user when specifying the first measurement point Q1 is the first virtual viewpoint VP1.

Then, if the candidate point R corresponding to the derived candidate position CP is not included in the two-dimensional image 40 (first two-dimensional image), the derivation unit 20C generates a second two-dimensional image, which is a two-dimensional image 40 with the second virtual viewpoint VP2 as the virtual viewpoint VP.

In detail, the derivation unit 20C sets the second virtual viewpoint VP2 and the direction in which the three-dimensional space S is viewed from the second virtual viewpoint VP2 so that both the first position P1 and the candidate position CP are within the field of view of the virtual camera 52 and so that both the first position P1 and the candidate position CP are outside the blind spot BS of the virtual camera 52.

Specifically, the derivation unit 20C places the second virtual viewpoint VP2 within a range E of 90 degrees left and right from the center of the vector representing the direction in which the virtual camera 52 placed at the first virtual viewpoint VP1 views the three-dimensional space S, and at a position where both the first position P1 and the candidate position CP are outside the blind spot BS of the virtual camera 52.

Furthermore, if the three-dimensional object OB located at the first position P1 can be identified, the derivation unit 20C may place the second virtual viewpoint VP2 at a position where the entire view of the three-dimensional object OB falls within the angle of view of the virtual camera 52.

Furthermore, the derivation unit 20C may set the direction in which the three-dimensional space S is viewed from the second virtual viewpoint VP2 to a direction corresponding to the midpoint between the first position P1 and the candidate position CP.

The derivation unit 20C then generates, as a second two-dimensional image, a two-dimensional image 40 of the three-dimensional space S represented by the three-dimensional information 30 viewed from a second virtual viewpoint VP2, which is the virtual viewpoint VP after viewpoint conversion. Note that known methods may be used to move the virtual viewpoint VP in the three-dimensional space S and to generate a two-dimensional image 40 of the three-dimensional space S viewed from the virtual viewpoint VP after the movement. Therefore, the second two-dimensional image, which is a two-dimensional image 40 of the three-dimensional space S viewed from the second virtual viewpoint VP2, includes both the first position P1 and the candidate position CP.

The display control unit 20D displays the two-dimensional image 40 generated by the derivation unit 20C on the display unit 14A. In other words, when the candidate position CP is located in the blind spot BS of the virtual camera 52, the display control unit 20D displays on the display unit 14A a two-dimensional image 40 in which the virtual viewpoint VP is converted from the first virtual viewpoint VP1 to the second virtual viewpoint VP2 so that the candidate position CP is outside the blind spot BS of the virtual camera 52.

The display control unit 20D may generate a first 2D image and a second 2D image and display them on the display unit 14A. In this case, the display control unit 20D acquires camera environment information, such as the position, rotation, and angle of view of the virtual camera 52 of the first virtual viewpoint VP1, from the derivation unit 20C. Then, the display control unit 20D displays a first 2D image of the three-dimensional space S viewed from the position of the virtual camera 52 represented by the camera environment information on the display unit 14A. The display control unit 20D may then change the camera environment information of the first virtual viewpoint VP1 to the camera environment information of the second virtual viewpoint VP2, thereby displaying the second 2D image on the display unit 14A.

As a result, a two-dimensional image 40 is displayed on the display unit 14A, in which the first position P1 and the candidate position CP are visually displayed.

The derivation unit 20C calculates a position Q1' on the two-dimensional image 40 corresponding to the first position P1 and a position CP' on the two-dimensional image corresponding to the candidate position CP in the two-dimensional image 40 of the three-dimensional space S viewed from the second virtual viewpoint VP2.

The display control unit 20D displays first identification information indicating the first position P1 at position Q1' on the two-dimensional image 40, and displays second identification information indicating the candidate position CP at position CP' on the two-dimensional image 40. The first identification information and second identification information correspond to the circular symbols shown in FIG. 3. The display control unit displays third identification information, which is a line segment (dashed line in FIG. 3) connecting positions Q1' and CP'.

The first identification information and second identification information may be any information that allows the visual recognition of the positions of the first position P1 and candidate position CP in the two-dimensional image 40, respectively, and are not limited to the circular symbols shown in FIG. 3. For example, other examples of the first identification information and second identification information may include symbols (such as arrow or polygonal icons), text information indicating the first position P1 (such as "measurement point 1"), or a combination of symbols and text information. Furthermore, the first identification information and second identification information may be displayed with different shapes, colors, etc.

The third identification information is not limited to the dashed line segment connecting the first identification information and the second identification information in Figure 3. Another example of the third identification information may be an arrow pointing from position Q1' to position CP'.

The display control unit 20D displays the first identification information, second identification information, and third identification information on the two-dimensional image 40 according to the positions Q1' and CP', which are updated in accordance with changes in the two-dimensional image as the virtual viewpoint VP2 moves.

The first identification information, second identification information, and third identification information are displayed in the two-dimensional image 40D, allowing the user to visually confirm the first position P1 and candidate point CP in the two-dimensional image 40D, thereby intuitively confirming and specifying the second position P2.

When a second two-dimensional image is displayed on the display unit 14A instead of the first two-dimensional image, it is preferable that the display control unit 20D displays on the display unit 14A a two-dimensional image 40 in which the virtual viewpoint VP is continuously moved from the first virtual viewpoint VP1 to the second virtual viewpoint VP2.

In other words, the display control unit 20D may sequentially display, on the display unit 14A, multiple two-dimensional images 40 of the three-dimensional space S viewed from each of multiple virtual viewpoints VP between the first virtual viewpoint VP1 and the second virtual viewpoint VP2. The creation of the multiple two-dimensional images 40 may be performed by the derivation unit 20C or the display control unit 20D.

The display control unit 20D may also display first identification information and second identification information that identify the first position P1 and the candidate position CP on each of a plurality of two-dimensional images 40 of the three-dimensional space S viewed from each of a plurality of virtual viewpoints VP between the first virtual viewpoint VP1 and the second virtual viewpoint VP2.

Specifically, for example, the display control unit 20D continuously changes camera environment information such as the position, rotation, and angle of view of the virtual camera 52 of the first virtual viewpoint VP1 at a predetermined interval (e.g., one second) toward the camera environment information of the second virtual viewpoint VP2. Then, the display control unit 20D switches the display to the second two-dimensional image when the camera environment information of the virtual camera 52 of the first virtual viewpoint VP1 matches the camera environment information of the second virtual viewpoint VP2.

The display control unit 20D sequentially displays on the display unit 14A multiple two-dimensional images 40 of the three-dimensional space S viewed from each of multiple virtual viewpoints VP between the first virtual viewpoint VP1 and the second virtual viewpoint VP2, thereby providing the user with an easy-to-understand indication of the movement of the virtual viewpoint VP. Furthermore, sudden changes in the displayed two-dimensional images 40 can be prevented, improving user recognition.

The display control unit 20D may display both the first and second two-dimensional images on the display unit 14A.

Figure 8 is a schematic diagram showing an example of a scene in which both the first two-dimensional image and the second two-dimensional image are displayed on the display unit 14A.

For example, assume that two-dimensional image 40B is the first two-dimensional image. Also, assume that two-dimensional image 40C is the second two-dimensional image.

In this case, for example, the display control unit 20D may display on the display unit 14A a two-dimensional image in which two-dimensional image 40C is superimposed on two-dimensional image 40B. As shown in FIG. 8, for example, two-dimensional image 40B, which is an example of a first two-dimensional image, includes first position P1, but candidate position CP is hidden because it is in the blind spot BS of obstacle 50. On the other hand, two-dimensional image 40C, which is an example of a second two-dimensional image, displays both first position P1 and candidate position CP so that they can be seen. This allows the user to always see candidate position CP.

Furthermore, by displaying on the display unit 14A a two-dimensional image in which two-dimensional image 40C is superimposed on two-dimensional image 40B, the display control unit 20D allows the user to check both the two-dimensional images 40 before and after the viewpoint conversion. This makes it easier for the user to check the candidate position CP. It also reduces confusion caused by the two-dimensional image 40 suddenly being switched and displayed.

The display control unit 20D may superimpose first identification information that identifies the first position P1 and second identification information that identifies the second position P2 on the two-dimensional image 40. For example, as shown in FIG. 8, the display control unit 20D may superimpose icon images that represent the first position P1 and the second position P2 or the candidate position CP on the two-dimensional image 40. In FIG. 8, the icon images are shown as circles as an example.

Note that the first identification information that identifies the first position P1 and the second identification information that identifies the second position P2 need only be information that can identify the first position P1, the second position P2, or the candidate position CP, and is not limited to icon images. The identification information may be, for example, a text image.

The above describes the case where there is one second position P2 or candidate position CP, but the derivation unit 20C may calculate multiple second positions P2 or candidate positions CP, and the display control unit 20D may display the two-dimensional image 40 on the display unit 14A so that the two-dimensional image 40 includes positions corresponding to the first position P1 and all of the multiple candidate positions CP (or multiple second positions P2).

Next, we will explain the case where the derivation unit 20C hides the obstacle 50 that forms the blind spot BS so that both the first position P1 and the candidate position CP are visible in the two-dimensional image 40. Hiding the obstacle 50 means displaying at least a portion of the obstacle 50 transparently or semi-transparently.

In detail, the derivation unit 20C generates a two-dimensional image 40 in which obstacles 50 that exist between the virtual viewpoint VP and the candidate position CP and that obstruct the display of the candidate position CP are made transparent or semi-transparent. The display control unit 20D then displays the two-dimensional image 40 generated by the derivation unit 20C on the display unit 14A. Note that the generation of the two-dimensional image 40 may also be performed by the display control unit 20D.

Figure 9 is an explanatory diagram of an example of generating a two-dimensional image 40 in which the obstacle 50 is not displayed.

The derivation unit 20C derives a straight line L3 connecting the candidate position CP in the three-dimensional space S and the virtual viewpoint VP of the virtual camera 52. The derivation unit 20C then makes the three-dimensional information of an object that intersects with the derived straight line L3 in the three-dimensional space S represented by the three-dimensional information 30 transparent or semi-transparent. Figure 9 shows an example in which the three-dimensional information representing an obstacle 50 that intersects with the straight line L3 is made transparent or semi-transparent. The degree of semi-transparency may be such that the candidate position CP is visibly displayed when a two-dimensional image 40 of the three-dimensional space S viewed from the virtual viewpoint VP is displayed on the display unit 14A.

By displaying a two-dimensional image 40 on the display unit 14A in which the obstacle 50 is transparent or semi-transparent, movement of the virtual viewpoint VP is reduced and the candidate position CP can be displayed in a way that is easily visible to the user.

The derivation unit 20C may hide or make transparent the entire object that intersects the line L3 in the three-dimensional space S, or may hide or make transparent only a portion of the object. When hiding a portion of an object, the derivation unit 20C may hide a portion of the obstacle 50 so that the candidate position CP, which is located within the blind spot BS formed by the obstacle 50 located at a position that intersects the line L3, is outside the blind spot BS of the virtual camera 52. By hiding a portion of the obstacle 50, it is possible to prevent areas unintended by the user from being hidden.

Through these processes, the display control unit 20D displays on the display unit 14A a two-dimensional image 40 in which obstacles 50 that exist between the virtual viewpoint VP and the candidate position CP and that obstruct the display of the candidate position CP are made transparent or semi-transparent.

As a result, both the first position P1 and the candidate position CP are displayed visibly on the display unit 14A.

In the above, the case where there is one second position P2 or candidate position CP has been described, but the derivation unit 20C may calculate multiple second positions P2 or candidate positions CP. Furthermore, the display control unit 20D may display the two-dimensional image 40 on the display unit 14A so that the two-dimensional image 40 includes positions corresponding to the first position P1 and all of the multiple candidate positions CP (or multiple second positions P2).

Then, when the derivation unit 20C acquires a confirmation instruction for the candidate point R of the displayed candidate position CP through input by the user to the input unit 14B, the derivation unit 20C confirms the candidate position CP as the second position P2.

Returning to Figure 1, the explanation continues. The distance calculation unit 20E calculates the distance between the first position P1 and the second position P2. The distance calculation unit 20E identifies first position information related to the first position P1 in the three-dimensional space S and position information related to the second position P2, thereby identifying the position coordinates of each of the first position P1 and the second position P2 in the three-dimensional space S. The distance calculation unit 20E then calculates the Euclidean distance between the first position P1 and the second position P2 using the identified position coordinates, thereby calculating the distance between the first position P1 and the second position P2.

Furthermore, if the derivation unit 20C further derives position information of a new candidate position CP and the candidate position CP is confirmed as the second position P2, the distance calculation unit 20E may use the first position P1, the second position P2, and one or more newly confirmed other second positions P2 to calculate the area of the region defined by these positions.

Next, we will explain the flow of information processing executed by the information processing device 10 of this embodiment.

Figure 10 is a flowchart showing an example of the flow of information processing executed by the information processing device 10 of this embodiment.

The acquisition unit 20A acquires three-dimensional information 30 from the imaging device 11 (step S100).

The derivation unit 20C determines a first virtual viewpoint VP1, which is a virtual viewpoint serving as the initial position, and a viewing direction for viewing the three-dimensional space S from the first virtual viewpoint VP1 (step S102).

Next, the derivation unit 20C generates a two-dimensional image 40 in which the three-dimensional space S represented by the three-dimensional information 30 acquired in step S100 is viewed in the initial direction from the first virtual viewpoint VP1 determined in step S102 (step S104). The processing of step S104 generates the two-dimensional image 40, which is the first two-dimensional image.

The display control unit 20D displays the two-dimensional image 40 generated in step S104 on the display unit 14A (step S106). Therefore, for example, the two-dimensional image 40A shown in FIG. 2 is displayed on the display unit 14A.

The derivation unit 20C acquires first position information relating to the first position P1 in the three-dimensional space S represented by the three-dimensional information 30 acquired in step S100 through input by the user to the input unit 14B (step S108).

In detail, the user specifies a first measurement point Q1 by operating the input unit 14B while viewing the two-dimensional image 40A displayed on the display unit 14A in step S106. In response to the user's specification, the reception unit 20B outputs information representing the position of the specified first measurement point Q1 in the displayed two-dimensional image 40A to the derivation unit 20C. Upon receiving the information representing the position of the first measurement point Q1 in the two-dimensional image 40A, the derivation unit 20C uses a known coordinate transformation process to derive first position information relating to a first position P1, which is a position corresponding to the first measurement point Q1 in the three-dimensional space S represented by the three-dimensional information 30. Through these processes, the derivation unit 20C acquires the first position information relating to the first position P1.

Next, the derivation unit 20C determines whether the user designates a candidate position CP (step S110). In step S110, the derivation unit 20C makes the determination of step S110 by determining whether position information for the candidate position CP is acquired from the user's input to the input unit 14B. In this embodiment, the derivation unit 20C determines whether the input form by the user to the input unit 14B satisfies a predetermined condition. The predetermined condition is when the input form is a pointing device such as a mouse, or when the input form is a drag operation on a touch panel, and the pointer position PP is clear on the display unit 14A displaying the two-dimensional image 40.

If the answer is affirmative in step S110 (step S110: Yes), proceed to step S112.

In step S112, the derivation unit 20C acquires position information of the candidate position CP from the pointer position PP input by the user to the input unit 14B (step S112). In step S112, the derivation unit 20C identifies the pointer position PP on the two-dimensional image 40 that was specified by the user's operation using the input unit 14B. The derivation unit 20C sets the identified pointer position PP as a candidate point R, and identifies the position in the three-dimensional space S that corresponds to the candidate point R as the candidate position CP. The derivation unit 20C then acquires position information of the candidate position CP in the three-dimensional space S. The process then proceeds to step S116, which will be described later.

On the other hand, if a negative judgment is made in step S110 (step S110: No), proceed to step S114.

In step S114, the derivation unit 20C calculates the position information of the candidate position CP based on the first position information (step S114). The derivation unit 20C calculates the position information of the candidate position CP based on the virtual viewpoint VP of the two-dimensional image 40 displayed on the display unit 14A and the position information of the first position P1. Then, the process proceeds to step S116.

In step S116, the derivation unit 20C determines whether the candidate position CP is within a blind spot BS based on the derived position information of the candidate position CP (step S116). The derivation unit 20C determines whether the candidate position CP is located within a blind spot BS of the virtual camera 52 located at the first virtual viewpoint VP1, which is the virtual viewpoint VP of the two-dimensional image 40 currently displayed on the display unit 14A.

If it is determined that the candidate position CP is located outside the blind spot BS (step S116: No), proceed to step S118.

In step S118, the display control unit 20D displays on the display unit 14A a two-dimensional image 40 of the three-dimensional space S represented by the three-dimensional information 30 viewed from the first virtual viewpoint VP1 used in the determination of step S116 (step S118). By processing step S118, the display unit 14A displays a two-dimensional image 40 in which both the first position P1 and the candidate position CP derived in step S112 or step S114 are visually displayed. Then, the process proceeds to step S124, which will be described later.

On the other hand, if it is determined that the candidate position CP is located within the blind spot BS (step S116: Yes), proceed to step S120.

In step S120, the derivation unit 20C changes the display format of the two-dimensional image 40 so that both the first position P1 and the candidate position CP are visible within the two-dimensional image 40 (step S120). In step S120, the derivation unit 20C changes the display format by either generating a two-dimensional image 40 in which the virtual viewpoint VP is moved to a second virtual viewpoint VP2, or by hiding the obstacle 50 that forms the blind spot BS, so that the candidate point R corresponding to the candidate position CP is always visible within the two-dimensional image 40. The display control unit 20D then displays the two-dimensional image 40, the display format of which was changed in step S120, on the display unit 14A (step S122).

By the processing of steps S120 and S122, a two-dimensional image 40 is displayed on the display unit 14A, in which both the first position P1 and the candidate position CP derived in step S112 or step S114 are visually displayed. Then, the process proceeds to step S124.

In step S124, the derivation unit 20C determines whether a confirmation instruction for the candidate point R of the candidate position CP displayed in step S118 or step S122 has been acquired by a user's input to the input unit 14B (step S124). If a negative judgment is made in step S124 (step S124: No), the process returns to step S110. If a positive judgment is made in step S124 (step S124: Yes), the derivation unit 20C confirms the candidate position CP for which a confirmation instruction has been received as the second position P2 (step S126).

The distance calculation unit 20E calculates the distance between the first position P1 and the second position P2 (step S128). The distance calculation unit 20E calculates the distance between the first position P1 and the second position P2 using the first position information acquired in step S108 and the position information of the second position P2 located on the seabed in step S126.

The display control unit 20D displays the distance between the first position P1 and the second position P2 calculated in step S128 on the display unit 14A (step S130). Then, this routine ends.

As described above, the information processing device 10 of this embodiment displays, on the display unit 14A, a two-dimensional image 40 of the three-dimensional space S represented by the three-dimensional information 30, viewed from the virtual viewpoint VP of the virtual camera 52. The information processing device 10 includes a derivation unit 20C and a display control unit 20D. The derivation unit 20C acquires first position information relating to a first position P1 in the three-dimensional space S through a user's input to the input unit 14B, and acquires position information relating to a second position P2 from the user's input to the input unit 14B or calculates it based on the first position information. The display control unit 20D displays, on the display unit 14A, a two-dimensional image 40 including the first position P1 and the second position P2.

For this reason, in the information processing device 10 of this embodiment, even if the second position P2 is within the blind spot BS of the virtual camera 52 in the three-dimensional space S, the display unit 14A displays a two-dimensional image 40 including the first position P1 and the second position P2. In other words, the information processing device 10 of this embodiment displays a two-dimensional image 40 including a visible position of the measurement target on the display unit 14A.

Therefore, the user can specify the desired second position P2 by visually viewing the displayed two-dimensional image 40, without having to perform viewpoint manipulation to change the viewpoint so that the desired point is displayed within the two-dimensional image 40.

Therefore, the information processing device 10 of this embodiment makes it possible to easily specify the position of the measurement target within the three-dimensional space S represented by the three-dimensional information 30.

Furthermore, with the information processing device 10 of this embodiment, the user can complete measurement of the distance or area between desired points with minimal or no viewpoint manipulation of the virtual viewpoint VP.

(Variation 1)
In the above embodiment, an example has been described in which the measurement system 1 includes the imaging device 11 and the information processing device 10. However, the information processing device 10 may include at least the derivation unit 20C and the display control unit 20D, and a device including the information processing device 10 and the distance calculation unit 20E may be configured as the measurement device.

(Variation 2)
In the above embodiment, the information processing device 10 has been described as having the acquisition unit 20A, the reception unit 20B, the derivation unit 20C, and the distance calculation unit 20E. However, at least some of the functions included in the information processing device 10 may be installed in a server device or other external information processing device connected to a network or the like.

Figure 11 is a schematic diagram showing an example of a measurement system 1B of this modified example.

The measurement system 1B includes an imaging device 11, a storage unit 17, an information processing device 10A, and a server device 10B. The imaging device 11 and the storage unit 17 are connected to each other so that they can communicate with each other. The storage unit 17, the information processing device 10A, and the server device 10B are connected to each other so that they can communicate with each other via a network NW.

The storage unit 17 stores the three-dimensional information 30 obtained by the imaging device 11. The information processing device 10A includes a communication unit 12, a UI unit 14, and a control unit 21. The communication unit 12, UI unit 14, and control unit 21 are connected so that they can communicate with each other. The control unit 21 includes a reception unit 20B and a display control unit 20D. The communication unit 12, UI unit 14, reception unit 20B, and display control unit 20D are the same as in the above embodiment.

The server device 10B includes a communication unit 13 and a control unit 23. The communication unit 13 communicates with the information processing device 10A and the memory unit 17 via the network NW. The control unit 23 includes an acquisition unit 20A, a derivation unit 20C, and a distance calculation unit 20E. The acquisition unit 20A, derivation unit 20C, and distance calculation unit 20E are the same as in the above embodiment.

As shown in FIG. 11, at least some of the functions of the information processing device 10 may be installed in the server device 10B.

Next, we will explain the hardware configurations of the information processing device 10, information processing device 10A, and server device 10B in the above embodiment and modified example.

Figure 12 is a hardware configuration diagram of an example of the information processing device 10, information processing device 10A, and server device 10B according to the above embodiment and modified example.

The information processing device 10, information processing device 10A, and server device 10B in the above-described embodiment and modified examples include a control device such as a CPU 90A, storage devices such as a ROM (Read Only Memory) 90B and a RAM (Random Access Memory) 90C, a HDD (Hard Disk Drive), an I/F 90D that connects to a network for communication, and a bus 90E that connects the various components.

The programs executed by the information processing device 10, information processing device 10A, and server device 10B in the above-described embodiment and modified examples are provided pre-installed in ROM 90B or the like.

The programs executed by information processing device 10, information processing device 10A, and server device 10B in the above embodiments and variations thereof may be configured to be provided as a computer program product by being recorded in an installable or executable format on a computer-readable recording medium such as a CD-ROM (Compact Disk Read Only Memory), a flexible disk (FD), a CD-R (Compact Disk Recordable), or a DVD (Digital Versatile Disk).

Furthermore, the programs executed by the information processing device 10, information processing device 10A, and server device 10B in the above embodiments and modified examples may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Furthermore, the programs executed by the information processing device 10, information processing device 10A, and server device 10B in the above embodiments and modified examples may be provided or distributed via a network such as the Internet.

The programs executed by the information processing device 10, information processing device 10A, and server device 10B in the above-described embodiments and modified examples can cause a computer to function as each of the information processing devices 10, information processing device 10A, and server device 10B in the above-described embodiments and modified examples. In this computer, the CPU 90A can read the programs from a computer-readable storage medium onto the main storage device and execute them.

Furthermore, the information processing device 10, information processing device 10A, and server device 10B in the above-described embodiment and modified examples may be realized as virtual machines operating on a cloud system.

Although the above describes embodiments and modifications of the present invention, these embodiments and modifications are presented as examples and are not intended to limit the scope of the invention. These novel embodiments and modifications may be embodied in a variety of other forms, and various omissions, substitutions, and modifications may be made without departing from the spirit of the invention. These embodiments and modifications are included within the scope and spirit of the invention, as well as within the scope of the invention and its equivalents as set forth in the claims.

10, 10A Information processing device 20C Derivation unit 20D Display control unit 20E Distance calculation unit

JP 2017-026488 A

Claims (14)

An information processing device that displays, on a display unit, a two-dimensional image obtained by viewing a three-dimensional space represented by three-dimensional information from a virtual viewpoint of a virtual camera,
a derivation unit that acquires first position information related to a first position in the three-dimensional space by a user's input to an input unit, and calculates position information related to a second position based on the first position information;
a display control unit that displays the two-dimensional image including the first position and the second position on the display unit;
An information processing device comprising: The second position is
The information processing device according to claim 1 , wherein the position is determined based on an input specifying position information calculated by the derivation unit based on the first position information with respect to the two-dimensional image displayed on the display unit. The second position is
a position determined based on the virtual viewpoint and the first position,
a position within a cylindrical region of a predetermined radius with a line segment connecting the virtual viewpoint and the first position as its central axis;
3. The information processing device according to claim 1. The information processing device of claim 3, wherein the derivation unit calculates the predetermined radius based on the three-dimensional information. the three-dimensional information includes information related to a three-dimensional object;
the first position is a position on the three-dimensional object;
the second position is a position on the three-dimensional object that is the farthest from the first position;
5. The information processing device according to claim 1. the three-dimensional information includes information related to a three-dimensional object;
the first position is a position on the three-dimensional object;
6. The information processing device according to claim 1, wherein the second position is a position on the three-dimensional object that is a vertex of the three-dimensional object that is farthest from the first position. The display control unit
before the derivation unit acquires the second position, a first two-dimensional image, which is the two-dimensional image with a first virtual viewpoint as the virtual viewpoint, is displayed on the display unit;
When the second position is not included in the first two-dimensional image, a second two-dimensional image is displayed on the display unit, the second two-dimensional image being the two-dimensional image with a second virtual viewpoint as the virtual viewpoint;
the second two-dimensional image includes the first position and the second position;
7. The information processing device according to claim 1. The display control unit
displaying the first two-dimensional image and the second two-dimensional image on the display unit;
The information processing device according to claim 7 . The display control unit
a plurality of two-dimensional images of the three-dimensional space viewed from each of a plurality of virtual viewpoints between the first virtual viewpoint and the second virtual viewpoint are sequentially displayed on the display unit;
The information processing device according to claim 7 . The display control unit
displaying the two-dimensional image in such a manner that an obstacle that exists between the virtual viewpoint and the second position and that obstructs the display of the second position is made transparent or semi-transparent;
7. The information processing device according to claim 1. The display control unit
identification information for identifying the first position and identification information for identifying the second position are displayed superimposed on the two-dimensional image;
The information processing device according to any one of claims 1 to 10. An information processing device according to any one of claims 1 to 11;
a distance calculation unit that calculates a distance between the first position and the second position;
A measuring device comprising: An information processing method for displaying, on a display unit, a two-dimensional image obtained by viewing a three-dimensional space represented by three-dimensional information from a virtual viewpoint of a virtual camera, the method comprising:
acquiring first position information relating to a first position in the three-dimensional space by a user's input to an input unit, and calculating position information relating to a second position based on the first position information;
displaying the two-dimensional image including the first position and the second position on the display unit;
An information processing method including: An information processing program executed by an information processing device that displays, on a display unit, a two-dimensional image of a three-dimensional space represented by three-dimensional information viewed from a virtual viewpoint of a virtual camera,
acquiring first position information relating to a first position in the three-dimensional space by a user's input to an input unit, and calculating position information relating to a second position based on the first position information;
displaying the two-dimensional image including the first position and the second position on the display unit;
An information processing program including: