JP7586351B2 - X-ray imaging system and device display method - Google Patents

Description

The present invention relates to an X-ray imaging system and a device display method, and in particular to an X-ray imaging system and a device display method that display a three-dimensional model of a device.

Conventionally, a technique for generating a three-dimensional curve of a device is known. Such a technique is disclosed, for example, in Charlotte Delmas, Marie-Odile Berger, Erwan Kerrien, Cyril Riddell, Yves Trousset, et al., "Three-dimensional curvilinear device reconstruction from two fluoroscopic views." SPIE, Medical Imaging 2015: Image-Guided Procedures, Robotic Interventions, and Modeling, Feb 2015, San Diego, CA, United States. pp.94150F, 10.1117/12.2081885. hal-01139284 (hereinafter referred to as Non-Patent Document 1).

The above-mentioned Non-Patent Document 1 discloses a technique for generating a three-dimensional model of a curved device such as a guidewire or catheter inserted into a blood vessel of a subject. Specifically, the above-mentioned Non-Patent Document 1 discloses a technique for reconstructing a three-dimensional curve representing the device from two fluoroscopic images generated by X-ray photography from two different imaging angles.

Charlotte Delmas, Marie-Odile Berger, Erwan Kerrien, Cyril Riddell, Yves Trousset, et al., "Three-dimensional curvilinear device reconstruction from two fluoroscopic views." SPIE, Medical Imaging 2015: Image-Guided Procedures, Robotic Interventions, and Modeling , Feb 2015, San Diego, CA, United States. pp.94150F, 10.1117/12.2081885. hal-01139284

However, as in the above-mentioned non-patent document 1, when a three-dimensional curve (three-dimensional model) is generated by restoring the three-dimensional shape of a device placed inside the body of a subject based on two X-ray images generated by X-ray photography at two different shooting angles, it may not be possible to accurately generate a three-dimensional model of the device. Specifically, since the X-ray image generated by X-ray photography is a two-dimensional projection image, the shape in the depth direction along the X-ray irradiation axis in the X-ray image cannot be determined. Therefore, even if two X-ray images are generated by X-ray photography at two different shooting angles, the exact shape of the device cannot be determined in a plane perpendicular to both of the projection planes of the two X-ray images. For this reason, in a device having a linear structure, in a portion where the device is arranged in one X-ray image so as to extend along a direction parallel to the X-ray irradiation axis for generating the other X-ray image, the shape of the generated three-dimensional model of the device is considered to be inaccurate and may differ from the actual shape of the device. In this case, when performing a procedure such as moving a device inside the body of a subject while visually viewing an inaccurate three-dimensional model that has a different shape from the actual device, it is considered that the accuracy of the procedure will decrease. Therefore, when generating a three-dimensional model of a device to be placed inside the body of a subject based on X-ray images generated by X-ray shooting at two different shooting angles, it is desired to prevent a user from recognizing a shape different from the actual device.

This invention has been made to solve the above-mentioned problems, and one object of the invention is to provide an X-ray imaging system and a device display method that can prevent a user from recognizing a shape that differs from the actual device when generating a three-dimensional model of a device to be placed inside a subject's body based on X-ray images generated by X-ray imaging at two different imaging angles.

In order to achieve the above object, an X-ray imaging system according to a first aspect of the present invention includes an X-ray irradiation unit that irradiates X-rays onto a subject having a device with a linear structure placed inside the subject's body, and an X-ray detection unit that detects X-rays that have passed through the subject, an imaging unit that performs X-ray imaging on the subject at each of a first imaging angle and a second imaging angle that are imaging angles in two different directions, and a three-dimensional modeling unit that generates a three-dimensional model of the device placed inside the subject's body based on a first X-ray image generated by X-ray imaging at the first imaging angle and a second X-ray image generated by X-ray imaging at the second imaging angle. the device includes a model generation unit, a determination unit that determines portions of a three-dimensional model of the device generated by the three-dimensional model generation unit, the portions having an inaccurate shape by identifying, from a device having a linear structure, parallel portions extending in a direction parallel to an epipolar line that indicates the irradiation axis of the X-rays at the shooting angle for generating the other X-ray image, in one of the first X-ray image and the second X-ray image; and a display unit that displays the three-dimensional model of the device generated by the three-dimensional model generation unit and an indication based on the determination result of the portions having an inaccurate shape in the three-dimensional model by the determination unit.

A device display method according to a second aspect of the present invention includes the steps of: performing X-ray imaging at a first imaging angle and a second imaging angle, which are two different imaging angles, by detecting X-rays that have passed through a subject in whose body a device having a linear structure is placed; generating a three-dimensional model of the device placed inside the subject's body based on a first X-ray image generated by the X-ray imaging at the first imaging angle and a second X-ray image generated by the X-ray imaging at the second imaging angle; determining, from the device having a linear structure, a parallel portion in one of the first X-ray image and the second X-ray image that extends along a direction parallel to an epipolar line indicating the irradiation axis of the X-rays at the imaging angle for generating the other X-ray image, thereby determining a portion in which the shape of the generated three-dimensional model of the device is inaccurate; and displaying the generated three-dimensional model of the device and an indication based on the determination result of the portion in which the shape of the three-dimensional model is inaccurate.

In the X-ray imaging system in the first aspect and the device display method in the second aspect, a parallel portion extending in a direction parallel to an epipolar line indicating the irradiation axis of X-rays at an imaging angle for generating the other X-ray image in one of the first X-ray image and the second X-ray image is identified from among devices having a linear structure, and a portion having an inaccurate shape in a three-dimensional model of the generated device is determined. Then, a three-dimensional model of the generated device and a display based on the determination result of the portion having an inaccurate shape in the three-dimensional model are displayed. Here, in a device having a linear structure arranged inside the body of a subject, a parallel portion extending in a direction parallel to an epipolar line indicating the irradiation axis of X-rays at an imaging angle for generating the other X-ray image in one of the first X-ray image and the second X-ray image becomes an inaccurate portion when a three-dimensional model is generated based on the first X-ray image and the second X-ray image. In consideration of this point, in the present invention, a three-dimensional model of the generated device and a display based on the determination result of the portion having an inaccurate shape in the three-dimensional model are displayed. This allows the user to recognize inaccurate parts in the displayed 3D model that may have a shape different from the actual shape of the device by visually checking the display based on the displayed determination result. As a result, when a 3D model of a device to be placed inside the subject's body is generated based on X-ray images generated by X-ray shooting at two different shooting angles, it is possible to prevent the user from recognizing a shape different from the actual device.

1 is a schematic diagram showing an overall configuration of an X-ray imaging system according to an embodiment; FIG. 1 is a block diagram for explaining a configuration of an X-ray imaging system. FIG. 2 is a block diagram for explaining a functional configuration of a control unit. 4A and 4B are diagrams for explaining X-ray photography of a device by a first imaging unit and a second imaging unit. FIG. 13 is a diagram for explaining generation of a three-dimensional model of a device. FIG. 13 is a diagram for explaining a parallel portion extending along a direction parallel to an epipolar line. FIG. 4 is a diagram showing an example of a display on a display unit. 11 is a diagram (flowchart) for explaining a device display method by the X-ray imaging system. FIG. 13 is a diagram showing an example of a display of a three-dimensional model according to a modified example.

Below, one embodiment of the present invention is described with reference to the drawings.

(Overall configuration of X-ray imaging system)
An X-ray imaging system 100 according to one embodiment of the present invention will be described with reference to FIGS.

As shown in FIG. 1, the X-ray imaging system 100 of this embodiment irradiates X-rays to a subject 101 having a medical device 80 inserted therein. The X-ray imaging system 100 then performs X-ray imaging by detecting the X-rays that have passed through the subject 101. The X-ray imaging system 100 generates an image for confirming the state of the subject 101's body when performing percutaneous coronary intervention (PCI), for example. Percutaneous coronary intervention is a treatment that uses a device 80 to eliminate vascular stenosis and blockage for diseases such as angina pectoris and myocardial infarction, which are caused by stenosis and blockage of the coronary arteries of the heart.

The device 80 is, for example, a guide wire placed in a blood vessel in the body of the subject 101. The device 80 is inserted into the blood vessel to place a catheter or a stent in the body of the subject 101. The device 80 has a flexible linear structure so that it can be inserted into a blood vessel of the human body. The X-ray imaging system 100 of this embodiment generates a three-dimensional model 81 of the device 80 placed in the body of the subject 101 based on two X-ray images 11 and 12 (see FIG. 4) acquired by performing X-ray imaging. The X-ray imaging system 100 is configured to display the generated three-dimensional model 81.

Configuration of X-ray imaging system
As shown in FIGS. 1 and 2, an X-ray imaging system 100 according to this embodiment includes a tabletop 1, an imaging unit 2, a moving mechanism 3, a display unit 4, an operation unit 5, a control unit 6, and a storage unit 7.

The tabletop 1 is a bed on which the subject 101 irradiated with X-rays lies. With the subject 101 lying on the tabletop 1, the device 80 is inserted into the subject 101 and X-ray photography of the subject 101 is performed. The tabletop 1 is also configured to be movable by a tabletop moving unit 31 (described later) of the moving mechanism 3 under the control of the control unit 6.

The imaging unit 2 includes a first imaging unit 21 and a second imaging unit 22. The first imaging unit 21 has an X-ray irradiation unit 21a, an X-ray detection unit 21b, and a support unit 21c. The second imaging unit 22 has an X-ray irradiation unit 22a, an X-ray detection unit 22b, and a support unit 22c. The X-ray imaging system 100 is a biplane type imaging device having two imaging mechanisms, the first imaging unit 21 and the second imaging unit 22.

Each of the X-ray irradiator 21a of the first imaging unit 21 and the X-ray irradiator 22a of the second imaging unit 22 irradiates (radiates) X-rays to the subject 101 in whose body a device 80 is placed. Each of the X-ray irradiators 21a and 22a includes an X-ray tube that irradiates X-rays by applying a voltage. Each of the X-ray irradiators 21a and 22a is configured such that the intensity of the irradiated X-rays is controlled by controlling the voltage applied to the X-ray tube by the control unit 6. Each of the X-ray irradiators 21a and 22a irradiates the device 80 in the subject 101 with X-rays one or more times.

Each of the X-ray detection unit 21b of the first imaging unit 21 and the X-ray detection unit 22b of the second imaging unit 22 detects X-rays that have passed through the subject 101. Then, the X-ray detection units 21b and 22b output detection signals to the control unit 6 based on the detected X-rays. The X-ray detection units 21b and 22b include, for example, an FPD (Flat Panel Detector). Specifically, in the first imaging unit 21, X-rays irradiated from the X-ray irradiation unit 21a are detected by the X-ray detection unit 21b. Also, in the second imaging unit 22, X-rays irradiated from the X-ray irradiation unit 22a are detected by the X-ray detection unit 22b.

The support unit 21c is a C-arm that supports the X-ray irradiation unit 21a and the X-ray detection unit 21b of the first imaging unit 21. The support unit 21c also supports the X-ray irradiation unit 21a and the X-ray detection unit 21b so that the X-ray irradiation unit 21a and the X-ray detection unit 21b face each other across the top board 1 on which the subject 101 lies. Similarly, the support unit 22c is a C-arm that supports the X-ray irradiation unit 22a and the X-ray detection unit 22b of the second imaging unit 22. The support unit 22c also supports the X-ray irradiation unit 22a and the X-ray detection unit 22b so that the X-ray irradiation unit 22a and the X-ray detection unit 22b face each other across the top board 1 on which the subject 101 lies. The support units 21c and 22c are configured to be able to change the angle (imaging angle) by the imaging unit movement unit 32 of the movement mechanism 3, which will be described later. The support section 21c is configured to be able to change the distance between the X-ray irradiator 21a and the X-ray detector 21b. The support section 22c is configured to be able to change the distance between the X-ray irradiator 22a and the X-ray detector 22b.

The movement mechanism 3 includes a tabletop movement unit 31 and an imaging unit movement unit 32. The tabletop movement unit 31 changes the position and angle of the tabletop 1. The tabletop movement unit 31 moves the tabletop 1 based on a control signal from the control unit 6. The tabletop movement unit 31 includes, as a drive mechanism, for example, a servo motor or a stepping motor. In addition, the tabletop movement unit 31 outputs a signal indicating the current position and angle of the tabletop 1 to the control unit 6.

The imaging unit moving unit 32 moves each of the first imaging unit 21 and the second imaging unit 22. Specifically, the imaging unit moving unit 32 changes the position and angle of the X-ray irradiator 21a and the X-ray detector 21b relative to the subject 101 by moving the support unit 21c. The imaging unit moving unit 32 also changes the position and angle of the X-ray irradiator 22a and the X-ray detector 22b relative to the subject 101 by moving the support unit 22c. That is, the imaging unit moving unit 32 moves the first imaging unit 21 (X-ray irradiator 21a and X-ray detector 21b) and the second imaging unit 22 (X-ray irradiator 22a and X-ray detector 22b) to perform X-ray imaging from various positions and angles with respect to the subject 101. The imaging unit moving unit 32 includes, for example, a servo motor or a stepping motor as a driving mechanism. In addition, the imaging unit moving unit 32 outputs to the control unit 6 a signal indicating the current position and angle of the first imaging unit 21 (X-ray irradiation unit 21a and X-ray detection unit 21b) and a signal indicating the current position and angle of the second imaging unit 22 (X-ray irradiation unit 22a and X-ray detection unit 22b).

In this embodiment, the imaging unit 2 performs X-ray imaging of the subject 101 at each of the first imaging angle and the second imaging angle, which are imaging angles in two different directions, by moving each of the two imaging mechanisms, the first imaging unit 21 and the second imaging unit 22, by the imaging unit moving unit 32. Specifically, the first imaging unit 21 performs X-ray imaging at the first imaging angle. Also, the second imaging unit 22 performs X-ray imaging at the second imaging angle. The imaging angle represents the spatial angle of the irradiation axis of the X-ray irradiated to the subject 101. That is, the first imaging angle is the angle in the direction in which the X-ray irradiator 21a and the X-ray detector 21b face each other. In other words, the first imaging angle is a direction perpendicular to the detection surface of the X-ray detector 21b, and means the spatial angular direction of the irradiation axis of the X-ray irradiated from the X-ray irradiator 21a to the detection surface of the X-ray detector 21b. Similarly, the second imaging angle is an angle in the direction in which the X-ray irradiator 22a and the X-ray detector 22b face each other, is a direction perpendicular to the detection surface of the X-ray detector 22b, and means a spatial angular direction of the irradiation axis of the X-ray irradiated from the X-ray irradiator 22a to the detection surface of the X-ray detector 22b. For example, the first imaging unit 21 is configured to be able to change the imaging angle so that the X-ray detector 21b moves in the LAO (left anterior oblique) direction and the RAO (right anterior oblique) direction, which are the left-right direction of the subject 101, and the CRA (cranial) direction and the CAU (caudal) direction, which are the up-down direction (head side and leg side) of the subject 101, based on a state in which the X-ray irradiator 21a and the X-ray detector 21b are arranged along the vertical direction. Similarly, the second imaging unit 22 is configured to be able to change the imaging angle so that the X-ray detection unit 22b moves in the LAO direction, the RAO direction, and the CRA direction, and the CAU direction.

The display unit 4 is, for example, a display device such as a liquid crystal display. The display unit 4 displays images (still images and moving images) generated by the control unit 6. Details of the display on the display unit 4 will be described later.

The operation unit 5 accepts input operations for operating the X-ray imaging system 100. The operation unit 5 then outputs an operation signal to the control unit 6 based on the accepted input operations. The operation unit 5 accepts, for example, an operation to move the tabletop 1 and the imaging unit 2 by the moving mechanism 3. The operation unit 5 also accepts an operation to irradiate X-rays to perform X-ray imaging on the subject 101. The operation unit 5 also accepts input operations for executing control by the control unit 6. The operation unit 5 includes, for example, a touch panel.

The control unit 6 is a computer including a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory). The control unit 6 controls each part of the X-ray imaging system 100 by the CPU executing a predetermined control program. The control unit 6 is also configured to control the operation of each part of the X-ray imaging system 100 based on the input operation received by the operation unit 5.

3, the control unit 6 includes, as its functional components, an X-ray irradiation control unit 61, an X-ray image generating unit 62, a three-dimensional model generating unit 63, a determination unit 64, and a display control unit 65. That is, the control unit 6 functions as the X-ray irradiation control unit 61, the X-ray image generating unit 62, the three-dimensional model generating unit 63, the determination unit 64, and the display control unit 65 by the CPU executing a predetermined control program. Details of the control by the control unit 6 will be described later.

The storage unit 7 is configured, for example, with a storage device such as a hard disk drive. The storage unit 7 is configured to store image data, shooting conditions, and various setting values. The storage unit 7 also stores programs and parameters for causing the control unit 6 to function. The storage unit 7 also stores a trained model for detecting the device 80 in order to generate a three-dimensional model 81 of the device 80, which will be described later.

(Control of X-ray imaging system by control unit)
The X-ray irradiation control unit 61 of the control unit 6 controls the first imaging unit 21 (X-ray irradiator 21a and X-ray detector 21b) and the second imaging unit 22 (X-ray irradiator 22a and X-ray detector 22b) to perform X-ray imaging. Specifically, the X-ray irradiation control unit 61 controls the voltage applied to the X-ray tube in each of the X-ray irradiators 21a and 22a to control the irradiation of X-rays by the X-ray irradiators 21a and 22a. The X-ray irradiation control unit 61 also controls the movement of the moving mechanism 3. Specifically, the X-ray irradiation control unit 61 controls the operation of the moving mechanism 3 to change the position and angle of the tabletop 1 and to change the imaging angles (first imaging angle and second imaging angle) of X-ray imaging by the first imaging unit 21 and the second imaging unit 22. For example, the X-ray irradiation control unit 61 controls the moving mechanism 3 based on an input operation to the operation unit 5, thereby changing the imaging angles of the first imaging unit 21 and the second imaging unit 22 for performing X-ray imaging.

As shown in FIG. 4, the X-ray image generating unit 62 of the control unit 6 generates an X-ray image 11 by performing X-ray photography at a first imaging angle by the first imaging unit 21 (X-ray irradiation unit 21a and X-ray detection unit 21b), and generates an X-ray image 12 by performing X-ray photography at a second imaging angle by the second imaging unit 22 (X-ray irradiation unit 22a and X-ray detection unit 22b). Specifically, the X-ray image generating unit 62 generates an X-ray image 11 based on the X-rays detected by the X-ray detection unit 21b of the first imaging unit 21. The X-ray image generating unit 62 generates an X-ray image 12 based on the X-rays detected by the X-ray detection unit 22b of the second imaging unit 22. That is, the X-ray image 11 is an image projected onto a projection plane parallel to the detection plane of the X-ray detection unit 21b, and the X-ray image 12 is an image projected onto a projection plane parallel to the detection plane of the X-ray detection unit 22b. The X-ray image 11 is an example of a "first X-ray image" in the claims, and the X-ray image 12 is an example of a "second X-ray image" in the claims.

<Generating 3D models of devices>
5, the three-dimensional model generating unit 63 of the control unit 6 generates a three-dimensional model 81 of the device 80 placed inside the body of the subject 101 based on the X-ray image 11 and the X-ray image 12. Specifically, the three-dimensional model generating unit 63 detects the device 80 in each of the X-ray image 11 and the X-ray image 12 based on a trained model generated by machine learning. This trained model is generated in advance by machine learning using deep learning that trains the process of detecting the position (shape) of the device 80 from the input X-ray images 11 and 12.

Then, the three-dimensional model generating unit 63 detects the position (shape) of the device 80 in each of the X-ray images 11 and 12 from each of the X-ray images 11 and 12 using the learned model, thereby acquiring shape information (coordinates) of the device 80 included in the X-ray images 11 and 12. In addition, the three-dimensional model generating unit 63 acquires the imaging angles (first imaging angle and second imaging angle) of the first imaging unit 21 and the second imaging unit 22 based on the output from the imaging unit moving unit 32 in order to generate a three-dimensional model 81. Then, the three-dimensional model generating unit 63 generates a three-dimensional model 81 of the device 80 using epipolar geometry based on the shape of the device 80 included in each of the X-ray images 11 and 12 and the imaging angles of the first imaging unit 21 and the second imaging unit 22. That is, the three-dimensional model generation unit 63 is configured to reconstruct a three-dimensional model 81 that estimates the shape of the device 80 in three-dimensional space from the device 80 contained in the X-ray image 11 and the X-ray image 12, which are projection images from two different directions.

Here, as shown in FIG. 5, the shape of the generated three-dimensional model 81 may differ from the shape of the actual device 80. For example, the three-dimensional model 81 shown by the solid line in FIG. 5 differs in shape from the actual device 80 shown by the dotted line in part 80a. When the three-dimensional model 81 is generated using two imaging angles, the shape of the parallel part of the device 80 having a linear structure that extends along a direction parallel to the epipolar line becomes inaccurate. The epipolar line is a line that indicates the irradiation axis of the X-rays in one of the X-ray images 11 and 12 at the imaging angle for generating the other X-ray image.

6, in the X-ray image 11, the direction of the irradiation axis extending from the X-ray irradiation unit 22a of the second imaging unit 22 toward the X-ray detection unit 22b (direction D2) is the direction of the epipolar line in the X-ray image 11. Also, in the X-ray image 12, the direction of the irradiation axis extending from the X-ray irradiation unit 21a of the first imaging unit 21 toward the X-ray detection unit 21b (direction D1) is the direction of the epipolar line in the X-ray image 12. In other words, the direction of the epipolar line is perpendicular to the detection surfaces of the X-ray detection units 21b and 22b (the projection surfaces of the X-ray images 11 and 12).

For example, the portion 80a of the device 80 shown in FIG. 6 is a portion that extends along the direction D2 in the X-ray image 11 and a portion that extends along the direction D1 in the X-ray image 12. That is, the portion 80a is a portion (parallel portion) of the device 80 placed inside the body of the subject 101 that extends along a plane parallel to both the direction D1 and the direction D2. In other words, the portion 80a is a portion (parallel portion) that extends along a plane perpendicular to both the projection planes of the X-ray image 11 and the X-ray image 12. As illustrated by three dotted lines in FIG. 6, the portion 80a, which is a parallel portion, cannot uniquely estimate the three-dimensional shape of the actual device 80 from the X-ray image 11 and the X-ray image 12, which are projection images. In other words, portion 80a extending along a direction parallel to direction D2 in X-ray image 11 and portion 80a extending along a direction parallel to direction D1 in X-ray image 12 are photographed to have the same shape even though their shapes in the depth direction are different in each image, and therefore, in principle, when reconstructing three-dimensional model 81, these portions will have an inaccurate shape.

<Determination of parallel parts>
Therefore, in this embodiment, the determination unit 64 of the control unit 6 identifies a portion 80a of the device 80, which is a parallel portion extending along a direction parallel to the epipolar line, and thereby determines a portion of the three-dimensional model 81 generated by the three-dimensional model generation unit 63 that has an inaccurate shape.

Specifically, the determination unit 64 acquires the direction of the epipolar line in each of the X-ray images 11 and 12 based on a signal from the imaging unit movement unit 32. Then, based on the shape of the device 80 in the X-ray images 11 and 12 detected by the three-dimensional model generation unit 63, the determination unit 64 compares the extension direction of the device 80 in each of the X-ray images 11 and 12 with the direction of the epipolar line. Then, the determination unit 64 identifies a parallel portion that is a portion extending along a direction parallel to the epipolar line from the shape of the device 80 detected in each of the X-ray images 11 and 12. The determination unit 64 determines that the portion of the device 80 identified as the parallel portion is a portion whose shape in the three-dimensional model 81 is inaccurate.

For example, the determination unit 64 identifies, as a parallel portion, a portion of the device 80 in each of the X-ray images 11 and 12 in which the angle difference with the epipolar line is smaller than a predetermined determination threshold. In the determination by the determination unit 64, a portion in both the X-ray images 11 and 12 in which the angle difference with the epipolar line is smaller than a predetermined determination threshold may be identified as a parallel portion, or a portion in at least one of the X-ray images 11 and 12 in which the angle difference with the epipolar line is smaller than a predetermined determination threshold may be identified as a parallel portion.

<Display control of the display unit>
7, the display control unit 65 of the control unit 6 controls the display of the display unit 4. Specifically, the display control unit 65 causes the display unit 4 to display the three-dimensional model 81 generated by the three-dimensional model generation unit 63. Then, the display control unit 65 causes the display unit 4 to display a display based on the determination result of the determining unit 64 of the part in the three-dimensional model 81 having an inaccurate shape. Specifically, the display control unit 65 causes the display unit 4 to display, as a display based on the determination result, the part in the three-dimensional model 81 having an inaccurate shape determined by the determining unit 64 so as to be identifiable by coloring it.

For example, the display control unit 65 divides the three-dimensional model 81 into a plurality of elements, and displays each of the divided elements in a plurality of colors according to the angle difference with the epipolar line. For example, the display control unit 65 displays the three-dimensional model 81 displayed on the display unit 4 in a plurality of colors, such as blue, green, yellow, orange, and red, in order of the part (element) with the largest angle difference with the epipolar line. The part (element) displayed in blue in the three-dimensional model 81 is a part that has a large angle difference with the epipolar line and is estimated to have an accurate shape in the reconstruction of the three-dimensional model 81. On the other hand, as in the portion 80a of FIG. 7, the part (element) displayed in red in the three-dimensional model 81 is a part that has a small angle difference with the epipolar line and is determined to be a parallel part extending along a direction parallel to the epipolar line (a part with an inaccurate shape in the three-dimensional model 81). That is, the parallel portion determined by the determination unit 64 to have an angle difference with the epipolar line smaller than a predetermined determination threshold value is displayed in red color in the three-dimensional model 81. In Fig. 7, differences in color coding are indicated by different hatching.

In addition, in this embodiment, the display control unit 65 of the control unit 6 is configured to display suggested information 41 on the display unit 4 when the determination unit 64 determines that a part (parallel part) in the three-dimensional model 81 has an inaccurate shape. The suggested information 41 is a display that suggests taking an X-ray from an imaging angle different from the first imaging angle and the second imaging angle so that the shape in the three-dimensional model 81 can be accurately determined. In other words, the suggested information 41 is a display that suggests changing either the first imaging angle or the second imaging angle in order to accurately estimate the shape of the parallel part in the three-dimensional model 81 whose shape is estimated to be inaccurate.

Specifically, the proposed information 41 is a display proposing X-ray photography at an imaging angle that intersects both a direction parallel to the first irradiation axis (direction D1, see FIG. 6) at the first imaging angle and a direction parallel to the second irradiation axis (direction D2, see FIG. 6) at the second imaging angle. That is, the proposed information 41 is a display including a recommended movement direction for moving the imaging unit 2 so as to perform X-ray photography at an imaging angle that intersects both directions D1 and D2. The recommended movement direction is information indicating which angle direction is recommended to move from the current imaging angles (first imaging angle and second imaging angle) of the first imaging unit 21 and the second imaging unit 22 of the imaging unit 2. For example, in the example shown in FIG. 7, the proposed information 41 is displayed on the display unit 4 as text information proposing to move from the current imaging angle of the imaging unit 2 (first imaging unit 21 and second imaging unit 22) toward the CAU direction. Furthermore, the proposal information 41 may illustrate the recommended movement direction with an arrow, etc. Furthermore, the proposal information 41 may include information proposing which of the shooting angles of the first shooting unit 21 (first shooting angle) and the second shooting unit 22 (second shooting angle) should be changed.

As described above, in this embodiment, the display unit 4 is configured to display the three-dimensional model 81 and the coloring of the three-dimensional model 81, which is a display based on the judgment result, as well as to display the proposed information 41, under the control of the control unit 6 (display control unit 65). The display control unit 65 of the control unit 6 is configured to display the X-ray image 11 and the X-ray image 12 on the display unit 4 in addition to the three-dimensional model 81. In the X-ray imaging system 100 according to this embodiment, the control unit 6 generates 15 X-ray images 11 and 12 per second by irradiating X-rays 15 times per second. The display control unit 65 of the control unit 6 displays the X-ray images 11 and 12 on the display unit 4 as a moving image at 15 FPS (frames per second). Similarly, the control unit 6 generates the three-dimensional model 81 based on the generated X-ray images 11 and 12 so that it is updated 15 times per second, and displays it on the display unit 4.

(How to display devices)
Next, a device display method using the X-ray imaging system 100 according to this embodiment will be described with reference to Fig. 8. The control processes in steps 301 to 305 are executed by the control unit 6.

First, in step 301, X-ray imaging is performed. Specifically, the first imaging unit 21 and the second imaging unit 22 each detect X-rays that have passed through the subject 101 in whose body the device 80 is placed, and perform X-ray imaging at each of the first imaging angle and the second imaging angle, which are imaging angles in two different directions.

Next, in step 302, X-ray image 11 and X-ray image 12 are generated. Specifically, X-ray image 11 is generated based on X-rays detected by X-ray detection unit 21b of first imaging unit 21. Also, X-ray image 12 is generated based on X-rays detected by X-ray detection unit 22b of second imaging unit 22.

Next, in step 303, a three-dimensional model 81 is generated. Specifically, the three-dimensional model 81 of the device 80 placed inside the body of the subject 101 is generated based on the X-ray image 11 generated by X-ray photography at the first imaging angle by the first imaging unit 21 and the X-ray image 12 generated by X-ray photography at the second imaging angle by the second imaging unit 22.

Next, in step 304, parts of the three-dimensional model 81 that are geometrically inaccurate are determined. Specifically, parts of the device 80 that are geometrically inaccurate are determined in the three-dimensional model 81 of the generated device 80 by identifying parallel parts of the device 80 that extend along a direction parallel to the epipolar line.

Next, in step 305, the generated three-dimensional model 81 of the device 80 and a display based on the determination result of the parts of the three-dimensional model 81 whose shape is inaccurate are displayed. Specifically, the three-dimensional model 81 is displayed on the display unit 4. Then, as a display based on the determination result, the parts of the displayed three-dimensional model 81 whose shape is determined to be inaccurate (parallel parts) are displayed in color so that they can be identified.

(Effects of this embodiment)
In this embodiment, the following effects can be obtained.

In the X-ray imaging system 100 of this embodiment, as described above, in one of the X-ray images 11 (first X-ray image) and 12 (second X-ray image), a parallel portion extending along a direction parallel to the epipolar line indicating the irradiation axis of the X-ray at the imaging angle for generating the other X-ray image is identified from among the linearly structured device 80, and a portion having an inaccurate shape in the generated three-dimensional model 81 of the device 80 is determined. Then, the generated three-dimensional model 81 of the device 80 and a display based on the determination result of the portion having an inaccurate shape in the three-dimensional model 81 are displayed. Here, in the linearly structured device 80 placed inside the body of the subject 101, a parallel portion extending along a direction parallel to the epipolar line indicating the irradiation axis of the X-ray at the imaging angle for generating the other X-ray image in one of the X-ray images 11 and 12 becomes an inaccurate portion when the three-dimensional model 81 is generated based on the X-ray images 11 and 12. In consideration of this point, in the present embodiment, a generated three-dimensional model 81 of the device 80 and a display based on a determination result of a part having an inaccurate shape in the three-dimensional model 81 are displayed. In this way, by visually checking the display based on the displayed determination result, it is possible to recognize an inaccurate part in the displayed three-dimensional model 81 that may have a shape different from the shape of the actual device 80. As a result, when the three-dimensional model 81 of the device 80 to be placed inside the body of the subject 101 is generated based on the X-ray image 11 and the X-ray image 12 generated by X-ray photography at two different imaging angles, it is possible to prevent the user from recognizing a shape different from the actual device 80.

In addition, in the above embodiment, further advantages can be obtained by configuring as follows:

That is, in this embodiment, as described above, the determination unit 64 (control unit 6) is configured to determine the part of the three-dimensional model 81 having an inaccurate shape by identifying the parallel part of the device 80 including the guidewire placed in the blood vessel of the subject 101, and the display unit 4 is configured to display the three-dimensional model 81 of the device 80 including the guidewire and a display based on the determination result of the part of the three-dimensional model 81 having an inaccurate shape by the determination unit 64. With this configuration, when generating the three-dimensional model 81 of the device 80 including the guidewire placed in the blood vessel, the user can visually recognize the three-dimensional model 81 displayed on the display unit 4 and the display based on the determination result of the part of the three-dimensional model 81 having an inaccurate shape, thereby suppressing recognition of a shape different from the device 80 actually placed in the blood vessel. Therefore, when the user performs a procedure such as moving the device 80 in the blood vessel while visually checking the displayed three-dimensional model 81, it is possible to suppress a decrease in the accuracy of the procedure.

In addition, in this embodiment, as described above, the display unit 4 is configured to display parts of the three-dimensional model 81 that are determined by the determination unit 64 (control unit 6) in a manner that makes them identifiable by coloring them. With this configuration, the parts of the three-dimensional model 81 that are determined inaccurate in shape are displayed in color, so that the user can intuitively and easily recognize the parts of the three-dimensional model 81 that are inaccurate in shape by visually checking the display unit 4.

In addition, in this embodiment, as described above, the display unit 4 is configured to display the proposed information 41 proposing X-ray photography from an imaging angle different from the first imaging angle and the second imaging angle so that the shape of the three-dimensional model 81 can be determined when the determination unit 64 (control unit 6) determines that the shape of the three-dimensional model 81 is inaccurate. With this configuration, when the generated three-dimensional model 81 has an inaccurate part in shape, the proposed information 41 is displayed, so that when changing the imaging angle to eliminate the inaccurate part, the user can easily refer to the displayed proposed information 41 by visually checking the display unit 4. As a result, since the proposed information 41 can be easily referred to, the imaging angle can be easily changed to eliminate the inaccurate part.

In addition, in this embodiment, as described above, the proposed information 41 includes a recommended movement direction for moving the imaging unit 2 so as to perform X-ray imaging at an imaging angle in a direction intersecting both a direction parallel to the first irradiation axis (D1 direction) at the first imaging angle and a direction parallel to the second irradiation axis (D2 direction) at the second imaging angle. With this configuration, the user can recognize the proposed recommended movement direction by visually checking the proposed information 41, and can easily recognize in which direction the imaging angle should be changed specifically. Therefore, in order to eliminate an inaccurate part in the three-dimensional model 81, the user can easily determine in which angle direction the imaging angle should be moved. As a result, the user can more easily change the imaging angle to eliminate the inaccurate part.

In addition, in this embodiment, as described above, the imaging unit 2 includes a first imaging unit 21 that performs X-ray imaging at a first imaging angle and a second imaging unit 22 that performs X-ray imaging at a second imaging angle, and the determination unit 64 (control unit 6) is configured to identify parallel parts extending along the epipolar line from the device 80 having a linear structure, and determine parts having an inaccurate shape in the three-dimensional model 81 of the device 80 generated by the three-dimensional model generation unit 63 (control unit 6) based on the X-ray image 11 (first X-ray image) generated by X-ray imaging by the first imaging unit 21 and the X-ray image 12 (second X-ray image) generated by X-ray imaging by the second imaging unit 22. With this configuration, the first imaging unit 21 and the second imaging unit 22 can simultaneously obtain the X-ray image 11 and the X-ray image 12 by X-ray imaging at imaging angles in two different directions. Therefore, unlike the case where the X-ray images 11 and 12 are acquired by performing X-ray photography at two different imaging angles while moving one imaging unit 2, the three-dimensional model 81 can be generated in real time based on the X-ray images 11 and 12 while reducing the effort of moving the imaging unit 2 one by one to generate the three-dimensional model 81. As a result, the three-dimensional model 81 generated in real time can be visually confirmed and a procedure can be performed while reducing the effort of moving the imaging unit 2, and a display based on the determination result by the determination unit 64 is displayed, so that the user can recognize parts of the three-dimensional model 81 generated in real time that have inaccurate shapes.

(Effects of the image generating method according to this embodiment)
The device display method of this embodiment can provide the following effects.

In the device display method of the present embodiment, by configuring as described above, a parallel portion extending along a direction parallel to an epipolar line indicating the irradiation axis of X-rays at an imaging angle for generating the other X-ray image in one of the X-ray images 11 (first X-ray image) and 12 (second X-ray image) from the device 80 having a linear structure is identified, and a portion having an inaccurate shape in the three-dimensional model 81 of the generated device 80 is determined. Then, the three-dimensional model 81 of the generated device 80 and a display based on the determination result of the portion having an inaccurate shape in the three-dimensional model 81 are displayed. Here, in the device 80 having a linear structure arranged inside the body of the subject 101, a parallel portion extending along a direction parallel to an epipolar line indicating the irradiation axis of X-rays at an imaging angle for generating the other X-ray image in one of the X-ray images 11 and 12 becomes an inaccurate portion when the three-dimensional model 81 is generated based on the X-ray images 11 and 12. In consideration of this point, in the present embodiment, a three-dimensional model 81 of the generated device 80 and a display based on a determination result of a part having an inaccurate shape in the three-dimensional model 81 are displayed. In this way, by visually checking the display based on the displayed determination result, it is possible to recognize an inaccurate part in the displayed three-dimensional model 81 that may have a shape different from the shape of the actual device 80. As a result, it is possible to provide a device display method that can suppress a user from recognizing a shape different from the actual device 80 when generating a three-dimensional model 81 of the device 80 placed inside the body of the subject 101 based on the X-ray image 11 and the X-ray image 12 generated by X-ray photography at two different imaging angles.

[Modification]
It should be noted that the embodiments disclosed herein are illustrative and not restrictive in all respects. The scope of the present invention is indicated by the claims, not by the description of the embodiments above, and further includes all modifications (variations) within the meaning and scope of the claims.

For example, in the above embodiment, an example of generating a three-dimensional model 81 of a device 80 including a guidewire placed in a blood vessel of a subject 101 is shown, but the present invention is not limited to this. In the present invention, the device that generates the three-dimensional model may be a medical catheter. For example, the device that generates the three-dimensional model may be a balloon catheter for placing a stent in a subject, or a high-speed rotational coronary atherectomy that cuts a stenosed portion in a blood vessel. The device may also be the stent itself.

In the above embodiment, an example was shown in which parts of a three-dimensional model with inaccurate shapes are displayed in a identifiable manner by coloring them with multiple colors, but the present invention is not limited to this. For example, as in the modified three-dimensional model 281 shown in FIG. 9, a part whose angle difference with the epipolar line is smaller than a predetermined judgment threshold may be determined to be a parallel part, and only the part determined to be a parallel part may be colored with one color, thereby displaying the inaccurate part in the three-dimensional model 281 in an identifiable manner. Note that in FIG. 9, the difference in color is represented by the difference in hatching. Also, when coloring with multiple colors, the colors may be differentiated by a gradation that gradually changes the hue.

In addition, in the above embodiment, an example has been shown in which, when a part with an inaccurate shape is determined in the three-dimensional model 81, the display unit 4 displays the proposal information 41 including a recommended movement direction that proposes moving from the first imaging angle or the second imaging angle to a recommended angular direction, but the present invention is not limited to this. For example, the proposal information may indicate a specific imaging angle (such as LAO 30 degrees or RAO 45 degrees) instead of the angular direction of the recommended movement direction.

In addition, in the above embodiment, an example of a biplane type imaging unit 2 having two imaging mechanisms, the first imaging unit 21 and the second imaging unit 22, is shown, but the present invention is not limited to this. For example, a first X-ray image and a second X-ray image may be generated by performing X-ray imaging at two different imaging angles while changing the imaging angle using one imaging unit (one set of X-ray irradiation unit and X-ray detection unit).

[Aspects]
It will be appreciated by those skilled in the art that the exemplary embodiments described above are examples of the following aspects.

(Item 1)
an imaging unit including an X-ray irradiator that irradiates X-rays onto a subject inside of which a device having a linear structure is placed, and an X-ray detector that detects the X-rays that have passed through the subject, and that performs X-ray imaging of the subject at each of a first imaging angle and a second imaging angle that are imaging angles in two directions different from each other;
a three-dimensional model generation unit that generates a three-dimensional model of the device disposed inside the subject based on a first X-ray image generated by X-ray photography at the first imaging angle and a second X-ray image generated by X-ray photography at the second imaging angle;
a determination unit that determines a portion having an inaccurate shape in the three-dimensional model of the device generated by the three-dimensional model generation unit by identifying, from the device having a linear structure, a parallel portion extending along a direction parallel to an epipolar line indicating an irradiation axis of X-rays at the imaging angle for generating the other X-ray image in one of the first X-ray image and the second X-ray image;
and a display unit that displays the three-dimensional model of the device generated by the three-dimensional model generation unit and a display based on a determination result of a part of the three-dimensional model having an inaccurate shape by the determination unit.

(Item 2)
the determination unit is configured to determine a portion having an inaccurate shape in the three-dimensional model by identifying the parallel portion of the device, including a guidewire or a catheter, that is disposed in a blood vessel of the subject;
2. The X-ray imaging system according to claim 1, wherein the display unit is configured to display the three-dimensional model of the device including a guidewire or a catheter, and a display based on a result of the determination unit's determination of parts of the three-dimensional model having inaccurate shapes.

(Item 3)
3. The X-ray imaging system according to claim 1, wherein the display unit is configured to display a portion of the three-dimensional model having an inaccurate shape determined by the determination unit in a identifiable manner by coloring the portion.

(Item 4)
The X-ray imaging system according to any one of items 1 to 3, wherein the display unit is configured to display, when the determination unit determines that a portion of the three-dimensional model has an inaccurate shape, suggestion information that proposes X-ray imaging from an imaging angle different from the first imaging angle and the second imaging angle so that the shape of the three-dimensional model can be determined.

(Item 5)
5. The X-ray imaging system according to claim 4, wherein the proposed information includes a recommended movement direction for moving the imaging unit so as to perform X-ray imaging at an imaging angle that intersects both a direction parallel to a first irradiation axis at the first imaging angle and a direction parallel to a second irradiation axis at the second imaging angle.

(Item 6)
the imaging unit includes a first imaging unit that performs X-ray imaging at the first imaging angle and a second imaging unit that performs X-ray imaging at the second imaging angle;
The X-ray imaging system according to any one of items 1 to 5, wherein the determination unit is configured to determine portions of the device having an inaccurate shape in the three-dimensional model of the device generated by the three-dimensional model generation unit based on the first X-ray image generated by X-ray imaging with the first imaging unit and the second X-ray image generated by X-ray imaging with the second imaging unit, by identifying the parallel portions extending along the epipolar line from among the device having a linear structure.

(Item 7)
a step of detecting X-rays transmitted through a subject placed inside a body of a subject using a device having a linear structure, thereby performing X-ray imaging at each of a first imaging angle and a second imaging angle which are two different imaging angles;
generating a three-dimensional model of the device disposed inside the subject's body based on a first X-ray image generated by X-ray photography at the first imaging angle and a second X-ray image generated by X-ray photography at the second imaging angle;
determining a portion of the device having a linear structure, in which a parallel portion extends along a direction parallel to an epipolar line indicating an X-ray irradiation axis at the imaging angle for generating the other X-ray image, in one of the first X-ray image and the second X-ray image, where the parallel portion extends along a direction parallel to an epipolar line indicating an X-ray irradiation axis at the imaging angle for generating the other X-ray image;
A method for displaying a device, comprising: displaying the generated three-dimensional model of the device and an indication based on a determination result of a portion of the three-dimensional model having an inaccurate shape.

2 imaging unit 4 display unit 11 X-ray image (first X-ray image)
12 X-ray image (second X-ray image)
21 First imaging unit 21a, 22a X-ray irradiation unit 21b, 22b X-ray detection unit 22 Second imaging unit 41 Proposal information 63 Three-dimensional model generation unit 64 Determination unit 80 Device 81, 281 Three-dimensional model 100 X-ray imaging system 101 Subject

Claims (7)

an imaging unit including an X-ray irradiator that irradiates X-rays onto a subject inside of which a device having a linear structure is placed, and an X-ray detector that detects the X-rays that have passed through the subject, and that performs X-ray imaging of the subject at each of a first imaging angle and a second imaging angle that are imaging angles in two directions different from each other;
a three-dimensional model generation unit that generates a three-dimensional model of the device disposed inside the subject based on a first X-ray image generated by X-ray photography at the first imaging angle and a second X-ray image generated by X-ray photography at the second imaging angle;
a determination unit that determines a portion having an inaccurate shape in the three-dimensional model of the device generated by the three-dimensional model generation unit by identifying, from the device having a linear structure, a parallel portion extending along a direction parallel to an epipolar line indicating an irradiation axis of X-rays at the imaging angle for generating the other X-ray image in one of the first X-ray image and the second X-ray image;
and a display unit that displays the three-dimensional model of the device generated by the three-dimensional model generation unit and a display based on a determination result of a part of the three-dimensional model having an inaccurate shape by the determination unit. the determination unit is configured to determine a portion having an inaccurate shape in the three-dimensional model by identifying the parallel portion of the device, including a guidewire or a catheter, that is disposed in a blood vessel of the subject;
2. The X-ray imaging system according to claim 1, wherein the display unit is configured to display the three-dimensional model of the device including a guidewire or a catheter, and a display based on a result of the determination unit's determination of parts of the three-dimensional model having inaccurate shapes. The X-ray imaging system of claim 1, wherein the display unit is configured to identifiably display parts of the three-dimensional model whose shape is inaccurate as determined by the determination unit by coloring them. The X-ray imaging system of claim 1, wherein the display unit is configured to display suggestion information proposing X-ray imaging from an imaging angle different from the first imaging angle and the second imaging angle when the determination unit determines that a portion of the three-dimensional model has an inaccurate shape so that the shape of the three-dimensional model can be determined. The X-ray imaging system of claim 4, wherein the proposed information includes a recommended movement direction for moving the imaging unit so as to perform X-ray imaging at the imaging angle in a direction intersecting both a direction parallel to the first irradiation axis at the first imaging angle and a direction parallel to the second irradiation axis at the second imaging angle. the imaging unit includes a first imaging unit that performs X-ray imaging at the first imaging angle and a second imaging unit that performs X-ray imaging at the second imaging angle;
2. The X-ray imaging system according to claim 1, wherein the determination unit is configured to determine portions of the device having a linear structure, the portions having an inaccurate shape in the three-dimensional model of the device generated by the three-dimensional model generation unit based on the first X-ray image generated by X-ray imaging with the first imaging unit and the second X-ray image generated by X-ray imaging with the second imaging unit, by identifying the parallel portions extending along the epipolar line from among the device having a linear structure. a step of detecting X-rays transmitted through a subject placed inside a body of a subject using a device having a linear structure, thereby performing X-ray imaging at each of a first imaging angle and a second imaging angle which are two different imaging angles;
generating a three-dimensional model of the device disposed inside the subject's body based on a first X-ray image generated by X-ray photography at the first imaging angle and a second X-ray image generated by X-ray photography at the second imaging angle;
determining a portion of the device having a linear structure, in which a parallel portion extends along a direction parallel to an epipolar line indicating an X-ray irradiation axis at the imaging angle for generating the other X-ray image, in one of the first X-ray image and the second X-ray image, where the parallel portion extends along a direction parallel to an epipolar line indicating an X-ray irradiation axis at the imaging angle for generating the other X-ray image;
A method for displaying a device, comprising: displaying the generated three-dimensional model of the device and an indication based on a determination result of a portion of the three-dimensional model having an inaccurate shape.

Images