JP7708635B2 - Medical image processing device, medical image processing method, and medical image processing program - Google Patents

Description

The embodiments disclosed in this specification and the drawings relate to a medical image processing device, a medical image processing method, and a medical image processing program.

Conventionally, when treating or planning heart valves, various morphological information about the heart valve or the heart is confirmed. Heart valves usually have two to three leaflets, and the morphological information of each must be confirmed individually or by comparison. A technique for confirming such morphological information of a heart valve is known that identifies and displays any cross section based on the valve shape.

JP 2019-202142 A

One of the problems that the embodiments disclosed in this specification and the drawings aim to solve is to facilitate identification of displayed valve leaflets. However, the problems that the embodiments disclosed in this specification and the drawings aim to solve are not limited to the above problem. Problems that correspond to the effects of each configuration shown in the embodiments described below can also be positioned as other problems.

The medical image processing device according to the embodiment includes an acquisition unit, an extraction unit, an identification unit, a display control unit, and a reception unit. The acquisition unit acquires volume data including at least a cardiac valve. The extraction unit extracts an area corresponding to the cardiac valve included in the volume data. The identification unit identifies each of a plurality of valve leaflets included in the cardiac valve. The display control unit displays each of the plurality of valve leaflets in a different display form. The reception unit receives settings of a display cross section related to the cardiac valve. The display control unit sets information related to the display cross section to a display form corresponding to the display cross section position set by the reception unit among the plurality of valve leaflets.

FIG. 1 is a diagram showing an example of the configuration of a medical image processing apparatus according to the first embodiment. FIG. 2 is a flowchart showing the processing procedure of the processing performed by each processing function of the processing circuitry of the medical image processing apparatus according to the first embodiment. FIG. 3A is a diagram showing an example of a display screen according to the first embodiment. FIG. 3B is a diagram illustrating an example of a display screen according to the first embodiment. FIG. 4A is a diagram illustrating an example of a reference line according to the first embodiment. FIG. 4B is a diagram for explaining an example of an operation on a reference line according to the first embodiment. FIG. 4C is a diagram for explaining an example of an operation on a reference line according to the first embodiment. FIG. 4D is a diagram for explaining an example of an operation on a reference line according to the first embodiment. FIG. 4E is a diagram for explaining an example of an operation on a reference line according to the first embodiment. FIG. 5 is a diagram showing an example of an overlaid display of images according to the first embodiment. FIG. 6A is a diagram showing an example of a setting screen for display conditions according to the first embodiment. FIG. 6B is a diagram showing an example of a setting screen for display conditions according to the first embodiment. FIG. 7A is a diagram illustrating an example of a graph according to the first embodiment. FIG. 7B is a diagram illustrating an example of a graph according to the first embodiment. FIG. 7C is a diagram illustrating an example of a graph according to the first embodiment. FIG. 8 is a diagram for explaining an example of the process of specifying the initial cross-section position according to the first embodiment. FIG. 9 is a diagram for explaining an example of the leaflet determination process according to the first embodiment. FIG. 10A is a diagram for explaining an example of a process for determining a range of a determination target according to the first embodiment. FIG. 10B is a diagram for explaining an example of a process for determining a range of a determination target according to the first embodiment. FIG. 11A is a diagram illustrating an example of a display process according to the first embodiment. FIG. 11B is a diagram illustrating an example of a display process according to the first embodiment. FIG. 11C is a diagram illustrating an example of a display process according to the first embodiment. FIG. 11D is a diagram illustrating an example of a display process according to the first embodiment. FIG. 12 is a diagram for explaining an example of a display process of a cross-sectional image according to the first embodiment. FIG. 13A is a diagram showing an example of volume data according to Modification Example 1. As shown in FIG. FIG. 13B is a diagram showing an example of volume data according to Modification Example 1. As shown in FIG. FIG. 14 is a diagram for explaining an example of the calcification identifying process according to the first modification. FIG. 15A is a diagram illustrating an example of movement of a reference line according to Modification 2. FIG. 15B is a diagram illustrating an example of movement of a reference line according to Modification Example 2. FIG. 16A is a diagram showing an example of a change in a display area according to Modification Example 3. In FIG. FIG. 16B is a diagram showing an example of a change in the display area according to Modification Example 3. In FIG.

Below, embodiments of a medical image processing device, method, and program are described in detail with reference to the drawings. Note that the medical image processing device, method, and program according to the present application are not limited to the embodiments shown below. In addition, in the following description, similar components are given common reference numerals, and duplicate descriptions are omitted.

(First embodiment)
Fig. 1 is a diagram showing an example of the configuration of a medical image processing apparatus according to the first embodiment. For example, as shown in Fig. 1, a medical image processing apparatus 3 according to this embodiment is communicably connected to a medical image diagnostic apparatus 1 and a medical image storage apparatus 2 via a network. Note that various other devices and systems may also be connected to the network shown in Fig. 1.

The medical image diagnostic device 1 captures an image of a subject to generate a medical image. The medical image diagnostic device 1 then transmits the generated medical image to various devices on a network. For example, the medical image diagnostic device 1 is an X-ray diagnostic device, an X-ray CT (Computed Tomography) device, an MRI (Magnetic Resonance Imaging) device, an ultrasound diagnostic device, a SPECT (Single Photon Emission Computed Tomography) device, a PET (Positron Emission computed Tomography) device, etc.

The medical image storage device 2 stores various medical images related to subjects. Specifically, the medical image storage device 2 receives medical images from the medical image diagnostic device 1 via a network, and stores the medical images in a memory circuit within the device. For example, the medical image storage device 2 is realized by a computer device such as a server or a workstation. Also, for example, the medical image storage device 2 is realized by a PACS (Picture Archiving and Communication System) or the like, and stores medical images in a format that complies with DICOM (Digital Imaging and Communications in Medicine).

The medical image processing device 3 performs various information processing related to the subject. Specifically, the medical image processing device 3 receives medical images from the medical image diagnostic device 1 or the medical image storage device 2 via the network, and performs various information processing using the medical images. For example, the medical image processing device 3 is realized by computer equipment such as a server or a workstation.

For example, the medical image processing device 3 includes a communication interface 31, an input interface 32, a display 33, a memory circuit 34, and a processing circuit 35.

The communication interface 31 controls the transmission and communication of various data sent and received between the medical image processing device 3 and other devices connected via a network. Specifically, the communication interface 31 is connected to the processing circuitry 35, and transmits data received from other devices to the processing circuitry 35, or transmits data transmitted from the processing circuitry 35 to other devices. For example, the communication interface 31 is realized by a network card, a network adapter, a NIC (Network Interface Controller), etc.

The input interface 32 accepts input operations of various instructions and various information from the user. Specifically, the input interface 32 is connected to the processing circuit 35, converts the input operations received from the user into electrical signals, and transmits them to the processing circuit 35. For example, the input interface 32 is realized by a trackball, a switch button, a mouse, a keyboard, a touchpad that performs input operations by touching the operation surface, a touch screen that integrates a display screen and a touchpad, a non-contact input interface that uses an optical sensor, and a voice input interface. Note that in this specification, the input interface 32 is not limited to only those that have physical operating parts such as a mouse and a keyboard. For example, an electrical signal processing circuit that receives an electrical signal corresponding to an input operation from an external input device provided separately from the device and transmits this electrical signal to a control circuit is also included as an example of the input interface 32.

The display 33 displays various information and data. Specifically, the display 33 is connected to the processing circuit 35, and displays various information and data received from the processing circuit 35. For example, the display 33 is realized by a liquid crystal display, a CRT (Cathode Ray Tube) display, a touch panel, etc.

The memory circuitry 34 stores various data and programs. Specifically, the memory circuitry 34 is connected to the processing circuitry 35, and stores data received from the processing circuitry 35, or reads out stored data and transmits it to the processing circuitry 35. For example, the memory circuitry 34 is realized by a semiconductor memory element such as a random access memory (RAM) or a flash memory, a hard disk, an optical disk, etc.

The processing circuitry 35 controls the entire medical image processing device 3. For example, the processing circuitry 35 performs various processes in response to input operations received from a user via the input interface 32. For example, the processing circuitry 35 receives data transmitted from another device via the communication interface 31 and stores the received data in the memory circuitry 34. Also, for example, the processing circuitry 35 transmits the data received from the memory circuitry 34 to the communication interface 31, thereby transmitting the data to another device. Also, for example, the processing circuitry 35 displays the data received from the memory circuitry 34 on the display 33.

A configuration example of the medical image processing device 3 according to this embodiment has been described above. For example, the medical image processing device 3 according to this embodiment is installed in a medical facility such as a hospital or clinic, and supports various diagnoses and the formulation of treatment plans performed by users such as doctors. For example, the medical image processing device 3 executes various processes to facilitate the identification of displayed valve leaflets when displaying morphological information of heart valves.

As described above, there is a known technique for displaying morphological information about heart valves, which identifies and displays an arbitrary cross section based on the valve shape. However, because the structures of the leaflets of heart valves are very similar, it can be difficult to recognize which leaflet is displayed when observing each leaflet, and in such cases there is a risk of misidentifying the leaflets.

The medical image processing device 3 according to this embodiment is configured to associate display information related to a displayed cross section of a heart valve with the valve leaflets included in the displayed cross section, thereby making it easier to identify the valve leaflets in the displayed cross section. Specifically, the medical image processing device 3 displays information related to the displayed cross section in a display form that corresponds to the display form of the valve leaflets corresponding to the position of the displayed cross section. Below, the medical image processing device 3 having such a configuration will be described in detail.

For example, as shown in FIG. 1, in this embodiment, the processing circuitry 35 of the medical image processing device 3 executes a control function 351, an image acquisition function 352, an extraction function 353, a specification function 354, and a judgment function 355. Here, the control function 351 is an example of a display control unit and a reception unit. The image acquisition function 352 is an example of an acquisition unit. The extraction function 353 is an example of an extraction unit. The specification function 354 is an example of a specification unit. The judgment function 355 is an example of a judgment unit.

The control function 351 generates various GUIs (Graphical User Interfaces) and various display information in response to operations via the input interface 32, and controls the display 33 to display them. For example, the control function 351 causes the display 33 to display a GUI for displaying morphological information about the heart valve, various analytical information about the heart valve, and the like. The control function 351 also generates various display images about the heart valve based on medical images acquired by the image acquisition function 352, and causes the display 33 to display them.

For example, the control function 351 displays each of the multiple valve leaflets in a different display form. The control function 351 also accepts settings for display cross sections related to the heart valve, and displays information about the display cross section in a display form that corresponds to the display form of the leaflet that corresponds to the position of the set display cross section among the multiple valve leaflets. The processing by the control function 351 will be described in detail later.

The image acquisition function 352 acquires medical images of the subject from the medical image diagnostic device 1 or the medical image storage device 2 via the communication interface 31. Specifically, the image acquisition function 352 acquires volume data including at least the heart valves. The image acquisition function 352 can also acquire multiple volume data obtained by capturing multiple images in three dimensions in the time direction. For example, the image acquisition function 352 acquires CT images, ultrasound images, MRI images, X-ray images, Angio images, PET images, SPECT images, etc. as the above-mentioned volume data. By executing the above-mentioned image acquisition function 352, the processing circuitry 35 receives medical images of the subject from the medical image diagnostic device 1 or the medical image storage device 2 and stores the received medical images in the memory circuitry 34.

The extraction function 353 extracts an area indicating a biological organ of interest (hereinafter referred to as an area of interest) from the volume data acquired by the image acquisition function 352. For example, the extraction function 353 extracts an area corresponding to a heart valve included in the volume data as the area of interest. The processing by the extraction function 353 will be described in detail later.

The identification function 354 identifies each of the multiple valve leaflets contained in the heart valve in the volume data acquired by the image acquisition function 352. The identification function 354 also identifies the position of the display cross section in which each valve leaflet is appropriately displayed. The processing performed by the identification function 354 will be described in detail later.

The determination function 355 determines the type of leaflet included in the display cross section identified by the identification function 354. The processing by the determination function 355 will be described in detail later.

The above-mentioned processing circuitry 35 is realized, for example, by a processor. In this case, each of the above-mentioned processing functions is stored in the memory circuitry 34 in the form of a program executable by a computer. Then, the processing circuitry 35 realizes the function corresponding to each program by reading and executing each program stored in the memory circuitry 34. In other words, when each program is read, the processing circuitry 35 has each of the processing functions shown in FIG. 1.

The processing circuitry 35 may be configured by combining multiple independent processors, and each processor may execute a program to realize each processing function. Each processing function of the processing circuitry 35 may be appropriately distributed or integrated into a single or multiple processing circuits. Each processing function of the processing circuitry 35 may be realized by a combination of hardware and software such as circuits. Here, an example is described in which a program corresponding to each processing function is stored in a single storage circuitry 34, but the embodiment is not limited to this. For example, a configuration may be used in which a program corresponding to each processing function is distributed and stored in multiple storage circuits, and the processing circuitry 35 reads out and executes each program from each storage circuit.

Next, the processing procedure by the medical image processing device 3 will be explained using FIG. 2, and then each process will be explained in detail. FIG. 2 is a flowchart showing the processing procedure of the processing performed by each processing function of the processing circuitry 35 of the medical image processing device 3 according to the first embodiment.

For example, as shown in FIG. 2, in this embodiment, the image acquisition function 352 acquires medical images (volume data) of the subject from the medical image diagnostic device 1 or the medical image storage device 2 (step S101). For example, the image acquisition function 352 acquires volume data including morphological information of the anatomical structure of the heart valve in response to an operation for acquiring volume data via the input interface 32. This process is realized, for example, by the processing circuitry 35 calling up and executing a program corresponding to the image acquisition function 352 from the memory circuitry 34.

Next, the extraction function 353 extracts a region of interest (heart valve) contained in the medical image from the acquired volume data (step S102). This process is realized, for example, by the processing circuitry 35 retrieving a program corresponding to the extraction function 353 from the memory circuitry 34 and executing it.

Then, the identification function 354 identifies each valve leaflet based on the extraction result of the heart valve, and identifies the initial cross-sectional position, which is the position of the cross-section at which the identified valve leaflet can be easily observed (step S103). For example, this is realized by the processing circuitry 35 retrieving a program corresponding to the identification function 354 from the memory circuitry 34 and executing it.

Next, the determination function 355 determines the identified cross-sectional position (step S104). Specifically, the determination function 355 determines the leaflet that is easy to observe at the cross-sectional position. This process is realized, for example, by the processing circuitry 35 retrieving a program corresponding to the determination function 355 from the memory circuitry 34 and executing it.

Then, the control function 351 sets the display form (step S105) and displays the cross-sectional image on the display 33 in the set display form (step S106). Specifically, the control function 351 sets the display form of the valve leaflets and the display form of the information related to the cross-sectional image, and displays them on the display 33. Here, the control function 351 performs display control that associates the display form of the valve leaflets with the display form of the information related to the cross-sectional image. This process is realized, for example, by the processing circuitry 35 calling up a program corresponding to the control function 351 from the storage circuitry 34 and executing it.

Then, the control function 351 determines whether the cross-sectional position has been changed (step S107). If the cross-sectional position has been changed (step S107, Yes), the process returns to step S104, and the cross-sectional position determination process is executed. On the other hand, if the cross-sectional position has not been changed (step S107, No), the control function 351 determines whether the interpretation has been completed (step S108). If the interpretation has not been completed in the determination of step S108 (step S108, No), the determination process of step S107 is continued. On the other hand, if the interpretation has been completed in the determination of step S108 (step S108, Yes), the medical image processing device 3 ends the process. This process is realized, for example, by the processing circuitry 35 calling up a program corresponding to the control function 351 from the storage circuitry 34 and executing it.

The details of each process executed by the medical image processing device 3 are described below. Note that the following describes the process when the aortic valve is the observation target as an example. Note that the target of the process described in this embodiment is not limited to this, and all heart valves can be the target.

(Medical image acquisition processing)
2, the image acquisition function 352 acquires volume data including three-dimensional morphological information of the cardiac valve (aortic valve) to be observed in response to an operation for acquiring volume data via the input interface 32. For example, the image acquisition function 352 acquires a CT image of the aortic valve captured in three dimensions.

As described above, the medical image acquisition process in step S101 may be started by a user instruction via the input interface 32, or may be started automatically. In such a case, for example, the image acquisition function 352 monitors the medical image storage device 2 and automatically acquires volume data every time new volume data is stored.

Here, the image acquisition function 352 may determine the newly stored volume data based on a preset acquisition condition, and execute the acquisition process if the volume data satisfies the acquisition condition. For example, an acquisition condition that can determine the state of the volume data is stored in the memory circuitry 34, and the image acquisition function 352 determines the newly stored volume data based on the acquisition condition stored in the memory circuitry 34.

As an example, the memory circuitry 34 stores, as an acquisition condition, "acquire volume data captured using an imaging protocol targeting a cardiac valve," "acquire enlarged and reconstructed medical images," or a combination of these. The image acquisition function 352 acquires volume data that satisfies the above acquisition conditions.

As described above, after the volume data is acquired by the image acquisition function 352, a display image based on the volume data may be displayed. Here, an example of a display screen displayed by the display 33 will be described. FIG. 3A is a diagram showing an example of a display screen according to the first embodiment. As shown in FIG. 3A, the display screen according to this embodiment includes, for example, an area 201 for displaying thumbnail images, an area 202 for displaying images, an area 203 for displaying a menu bar, and an area 204 for displaying analysis results.

(Thumbnail view)
The area 201 displays a list of thumbnail images based on volume data acquired by the image acquisition function 352 in response to a user instruction. The thumbnail images displayed in a list are thumbnail images of volume data that satisfy conditions designated by the user using an interface (not shown) for designating image conditions.

For example, the user uses an interface (not shown) to specify information about the subject, such as the subject's name, subject ID, date of birth, and weight, as well as information about the image, such as the type of image modality, imaging device name, imaging date, imaging conditions, and reconstruction conditions. The image acquisition function 352 acquires volume data that satisfies the above conditions specified by the user from the medical image diagnostic device 1 or medical image storage device 2. For example, the image acquisition function 352 acquires information about the specified conditions from the image DICOM header, PACS, electronic medical record, RIS, HIS, etc., and compares the acquired conditions with the conditions specified by the user to acquire volume data that satisfies the conditions specified by the user.

The control function 351 generates a thumbnail image from the volume data acquired by the image acquisition function 352, and displays the generated thumbnail image in the area 201. For example, the control function 351 generates a thumbnail image by reducing a two-dimensional image of a representative cross section in the volume data to fit the size of the area 201.

Here, the control function 351 can display various icons in the area 201 in addition to the thumbnail images described above. For example, the control function 351 displays in the area 201 character strings or symbols indicating the acquired volume data, or various figures, images, and schematic images prepared in advance and stored in the memory circuitry 34. Furthermore, the control function 351 can display basic information about the volume data (such as the imaging date, the number of slices, and the reconstruction function) alongside the thumbnail images and icons described above. In such a case, for example, the control function 351 acquires this information from the DICOM header of the image, the PACS, the electronic medical record, the RIS, the HIS, and the like, and displays it in association with the thumbnail images and icons. The basic information to be displayed may be determined in advance, or may be specified by the user.

(Image display)
In the area 202, various images are displayed according to the user's instructions. For example, the user selects a thumbnail image displayed in the area 201 and drags and drops it into the area 202. In response to this operation, the control function 351 generates an image to be displayed from volume data corresponding to the selected thumbnail image, and displays the generated image in the area 202. Here, if an image is already displayed in the area 202 when the drag and drop is performed, the control function 351 displays a warning (not shown) (for example, a warning to prompt the user to save the image) to the user. Then, after the user accepts the operation in response to the warning, the control function 351 removes the image already displayed from the area 202 and displays an image corresponding to the dragged and dropped icon.

When displaying an image in region 202, control function 351 displays the image based on predetermined display conditions. Here, the display conditions include the allocation of images to be displayed in the multiple display regions included in region 202 (for example, which image is to be displayed in which of regions 202a to 202d shown in FIG. 3A), the cross-sectional position, magnification ratio, window level, window width, etc. when displaying a cross-sectional image. That is, when displaying an image in region 202, control function 351 acquires the above-mentioned display conditions, generates an image to be displayed in region 202 based on the acquired display conditions, and displays the generated image in region 202.

The above display conditions are merely examples, and any conditions may be set. Furthermore, the display conditions may be changed arbitrarily by the user. In such a case, for example, the control function 351 displays a GUI for setting the display conditions and accepts the display conditions specified by the user.

For example, in the initial layout of the display screen, as shown in FIG. 3A, four regions, 202a to 202d, are set for region 202. A VR (Volume Rendering) image of the aortic valve generated based on the aortic valve region identified by a method described below is displayed in region 202a, and regions 202b to 202d are set to display each MPR (Multi Planar Reconstruction) cross section set based on the aortic valve axis (an example of a calculation method is described below) identified based on the aortic valve region.

The images displayed in areas 202a to 202d are not limited to the images described above, and can be set to display any cross-sectional direction or type of image specified by the user under any conditions. Also, known types of images such as VR images, SR images, MIP images, and MinIP images can be reconstructed and displayed. The images displayed here can be displayed by changing the observation cross-section, slice advance (browsing), magnification ratio, center position (translation), window level, window width, etc. based on the user's instructions.

The size of areas 202a to 202d can also be changed as desired in response to user instructions. For example, the user performs operations (drag and drop, etc.) on the frame of each area (the dividing line that divides the display area), or specific operations on each area (double-click, click while holding down the Ctrl key, etc.). The control function 351 changes the size of areas 202a to 202d in response to the operation input by the user.

The control function 351 can also display information specified in advance or by the user superimposed on a specific position on each region. For example, as shown in region 202e in FIG. 3A, the control function 351 displays information superimposed on the edge position of each region. Here, the information can specify information related to the patient, such as the subject's name, subject ID, date of birth, and weight, as well as information related to the image, such as the type of image modality, name of the imaging device, imaging date, imaging conditions, and reconstruction conditions. That is, the control function 351 obtains the specified information from the image's DICOM header, PACS, electronic medical record, RIS, HIS, etc., and displays it in regions 202b to 202d.

(Menu bar)
Area 203 is a menu bar on which icons and buttons corresponding to various functions are arranged. A user can select an icon arranged in area 203 using an input interface such as a mouse to activate the corresponding function.

Icon 203a is a button for switching between displaying and hiding area 201, and when this icon is selected, control function 351 switches between displaying and hiding area 201, which displays thumbnail images. For example, when icon 203a is pressed while area 201 is displayed, control function 351 hides area 201, as shown in FIG. 3B. Here, control function 351 enlarges area 202 and/or area 204 to match the size of area 201 that has become hidden. Note that FIG. 3B is a diagram showing an example of a display screen according to the first embodiment.

Icon 203b is a button for changing the number of divisions in area 202. In FIG. 3A, four areas (202a to 202d) with two rows and two columns are set, but the number of rows or columns of the display areas may be changeable. The size of each display area may also be changeable. For example, several patterns of sets of the number of divisions and sizes of the display areas are registered in advance as presets. When icon 203b is pressed, control function 351 displays an interface for selecting a previously registered set, and accepts a selection operation on the interface, thereby setting the display form of area 202. Note that control function 351 can also display an interface for accepting registration of a new preset from the user.

The icons 203c to 203g are a group of buttons for assigning mouse operation systems. For example, when each icon is selected, the control function 351 controls to assign the operation system of left click and drag of the mouse to the operation system corresponding to the selected icon. Specifically, the icon 203c is a button for assigning the browse operation system for continuously displaying images in the slice direction to the operation system of left click and drag of the mouse. The icon 203d is a button for assigning the operation system for changing the gradation of the image (for example, the window level and window width in CT) to the operation system of left click and drag of the mouse. The icon 203e is a button for assigning the operation system for translating the image to the operation system of left click and drag of the mouse. The icon 203f is a button for assigning the operation system for changing the magnification ratio of the image to the operation system of left click and drag of the mouse. The icon 203g is a button for assigning the operation system for rotating the image, etc. to the operation system of left click and drag of the mouse.

The operations to which the above-mentioned functions are assigned are not limited to the left click and drag operation system of the mouse, but the above-mentioned functions may also be assigned to the right click and drag operation system, the mouse wheel click and drag operation system, or the simultaneous right and left click and drag operation system.

It may also be possible to set the speed or amount of slice advance in the browse function, the amount of change in magnification, the amount of translation, the amount of change in gradation, and the amount of rotation in relation to the amount of mouse movement (amount of drag operation). Also, the assignment may be changed according to the mouse operation when selecting the icon. For example, when the icon is selected by left clicking, the operation system corresponding to the icon is assigned to the left click operation system, when the icon is selected by right clicking, the operation system corresponding to the icon is assigned to the right click operation system, when the icon is selected by simultaneous right and left clicking, the operation system corresponding to the icon is assigned to the simultaneous right and left click operation system, and when the icon is selected by mouse wheel clicking, the operation system corresponding to the icon is assigned to the mouse wheel click operation system.

Icons 203h to 203n are icons for drawing and measuring various figures, and when the icon is selected, the control function 351 controls the drawing and measuring functions of various figures to be executable.

When icon 203h is selected, a function can be executed to draw a straight line on the image and measure and display the length of the line. The display form, such as the position, color, thickness, and font of the measurement value of the line, of the start point and end point of the line can be adjusted by the user.

When icon 203i is selected, it becomes possible to execute a function of drawing two straight lines on an image and measuring and displaying the acute angle between the two straight lines. The display form, such as the position, color, thickness, and font of the measurement value of each of the two straight lines, can be adjusted by the user.

When icon 203j is selected, it becomes possible to execute the function of drawing an ellipse on the image and calculating and displaying the perimeter, internal area, and statistics of the pixel values inside the ellipse (average, maximum, minimum, etc.). The display form of the ellipse, such as the center position, major axis, minor axis, color, thickness, and font of the measurement values, can be adjusted by the user.

When icon 203k is selected, a function of drawing an arrow icon on the image can be executed. The display form of the arrow icon, such as the start and end positions, color, thickness, and tip shape, can be adjusted by user operation.

When icon 203l is selected, it becomes possible to execute a function of displaying any character string on the image. The character string to be displayed is specified by the user using a keyboard or the like. The display form, such as the position where the character string is displayed, the font of the character string, and the background color, can be adjusted by user operation.

When icon 203m is selected, it becomes possible to execute a function of drawing a closed curve of any shape on the image, and calculating and displaying the perimeter of the closed curve, the area inside the closed curve, and statistics of the pixel values inside (average, maximum, minimum, etc.). The display form of the closed curve, such as the center position, color, thickness, and font of the measurement values, can be adjusted by user operation. The closed curve can also be set to a predetermined shape (circle, ellipse, rectangle, square, triangle, etc.), and the length of each side of the shape, the angle between two sides, the diameter, major axis, minor axis, etc. can be adjusted, or a freeform shape can be drawn.

When icon 203n is selected, it becomes possible to execute a function of drawing an open curve of any shape on the image, and calculating and displaying the perimeter of the open curve. The display form of the open curve, such as the center position, color, thickness, and font of the measurement value, can be adjusted by user operation. In addition, it is also possible to set three-dimensional figures (spheres, oval spheres, rectangular parallelepipeds, triangular pyramids, etc.), and to control the calculation and display of their surface areas, volumes, etc.

Icon 203o is a checkbox that switches between displaying and hiding reference lines indicating the cross-sectional positions of other images on the image displayed in area 202. When the checkbox is selected, control function 351 switches between displaying and hiding the reference lines.

Figure 4A is a diagram showing an example of a reference line according to the first embodiment. Here, Figure 4A is an enlarged view of area 202b in Figure 3A. For example, the control function 351 displays a vertical reference line and a horizontal reference line shown in Figure 4A by selecting icon 203o. The vertical reference line in Figure 4A indicates the cross-sectional position of the image displayed in area 202c. Also, the horizontal reference line in Figure 4A indicates the cross-sectional position of the image displayed in area 202d.

For example, the control function 351 identifies the position of other images on the image to be displayed in the area 202b based on the cross-sectional position of each image in the volume data, and displays vertical and horizontal reference lines by placing straight lines at the identified positions.

Here, the reference lines according to this embodiment can accept operations from the user. For example, the control function 351 changes the display form of the image by accepting selection operations and drag-and-drop operations on each reference line, the intersections 301 of the reference lines, the circle marks 302, and the triangle marks 303.

For example, the user can move the intersection 301 of the reference lines by operating the mouse to select the intersection 301 of the reference lines or its surroundings and then dragging and dropping it. Here, when the intersection 301 is moved, the cross-sectional position indicated by the reference lines also changes, and the cross-sectional images displayed in each area also change accordingly.

Figure 4B is a diagram for explaining an example of an operation on a reference line according to the first embodiment. Here, Figure 4B is an example of a screen when the intersection of the reference lines shown in area 202b of Figure 3A is moved to the upper left. For example, when the user moves the intersection 301 of the reference lines to the upper left as shown in area 202b of Figure 4B, the control function 351 identifies the position of each reference line after the movement in the volume data based on the position of each reference line with respect to the image displayed in area 202b. Then, the control function 351 generates an image in which the position of each identified reference line is set as the cross-sectional position, and displays the generated images in areas 202c and 202d.

The user can also translate the reference line by operating the mouse to select the reference line and drag and drop it. For example, the user can translate the reference line by selecting a location on the reference line other than the circle mark 302 and dragging and dropping it. Here, when the reference line is translated, the cross-sectional position indicated by the reference line changes, and the cross-sectional images displayed in each display area change accordingly.

Figure 4C is a diagram for explaining an example of an operation on a reference line according to the first embodiment. Here, Figure 4C is an example of a screen when the horizontal reference line shown in region 202b in Figure 3A is translated downward in region 202b. For example, when the user translates the horizontal reference line downward as shown in region 202b in Figure 4C, the control function 351 identifies the position of the horizontal reference line after the movement in the volume data based on the position of the horizontal reference line relative to the image displayed in region 202b. Then, the control function 351 generates an image in which the position of the identified horizontal reference line is the cross-sectional position, and displays the generated image in region 202d.

The user can also rotate the reference line by operating the mouse to select the reference line and dragging and dropping it in a specific direction. For example, a rotation function is assigned to the circle mark 302 displayed on the reference line, and the user rotates the reference line by selecting the circle mark 302 with the mouse and dragging and dropping it. Here, when the reference line is rotated, the cross-sectional position indicated by the reference line changes, and the cross-sectional images displayed in each display area change accordingly.

Figure 4D is a diagram for explaining an example of an operation on a reference line according to the first embodiment. Here, Figure 4D is an example of a screen in which the vertical reference line shown in area 202b in Figure 3A is rotated 45 degrees clockwise. When there are multiple reference lines during rotation, the other reference lines may be controlled to operate in conjunction with an operation on one reference line so that the angle they form is always kept constant, or they may move independently. When the angle formed by multiple reference lines is always kept constant, for example, 180 degrees may be divided equally based on the number of reference lines (perpendicular intersection (90 degree intersection) if there are two references, and 60 degree intersection if there are three reference lines). Figure 4D shows a case in which the angle formed by two reference lines is kept constant (90 degrees).

For example, when the user rotates the vertical reference line 45 degrees clockwise as shown in area 202b in FIG. 4D, the control function 351 identifies the position of each reference line after rotation in the volume data based on the position of each reference line relative to the image displayed in area 202b. The control function 351 then generates images in which the positions of each identified reference line are set as cross-sectional positions, and displays the generated images in areas 202c and 202d, respectively.

Note that in FIG. 4C and FIG. 4D, a case has been described in which a translation is performed by selecting a location on the reference line other than the circle mark 302, and a rotation is performed by selecting the circle mark 302. However, the embodiment is not limited to this, and for example, translation and rotation may be switched by an operation using a keyboard. For example, control may be performed so that when a reference line operation is performed with the mouse while the Ctrl key is held down, rotation is performed, and when a reference line operation is performed with the mouse while the Ctrl key is not held down, translation is performed.

The user can also use the reference line to specify a certain width, create a maximum value projection image or a minimum value projection image in a range based on the width, and display the projection image in the display area corresponding to the reference line. For example, the user can specify the width by selecting and moving the triangular mark 303 with the mouse.

Figure 4E is a diagram for explaining an example of an operation on a reference line according to the first embodiment. Here, Figure 4E shows an example in which a range sandwiched between two triangular marks 303 is specified as a width by moving a triangular mark 303 placed on a vertical reference line left and right. Also, Figure 4E shows an example in which a range sandwiched between two triangular marks 303 is specified as a width by moving a triangular mark 303 placed on a horizontal reference line up and down. Also, Figure 4E shows an example in which a Mip image of a range based on the size of the width specified by the triangular marks 303 is generated.

For example, as shown in the area 202b of FIG. 4E, when the user moves the triangular mark 303 arranged relative to the vertical reference line to the left or right, the control function 351 identifies positions in the volume data corresponding to the vertices of the left and right triangular marks 303 on the reference line side after the movement. Then, the control function 351 sets a plane in the volume data that passes through the position corresponding to the vertex of the left triangular mark 303 and is parallel to the vertical reference line. Similarly, the control function 351 sets a plane in the volume data that passes through the position corresponding to the vertex of the right triangular mark 303 and is parallel to the vertical reference line. Then, the control function 351 identifies voxels included between the left plane and the right plane with the vertical reference line as the center, and generates a Mip image based on the voxel value of the identified voxel. That is, the control function 351 generates a Mip image with the projection direction perpendicular to the vertical reference line (left and right direction) and displays it in the area 202c.

Similarly, when the user moves the triangular mark 303 arranged relative to the horizontal reference line up or down, the control function 351 identifies positions in the volume data corresponding to the vertices of the upper and lower triangular marks 303 on the reference line side after the movement. Then, the control function 351 sets a plane that passes through the position corresponding to the vertex of the upper triangular mark 303 and is parallel to the horizontal reference line in the volume data. Similarly, the control function 351 sets a plane that passes through the position corresponding to the vertex of the lower triangular mark 303 and is parallel to the horizontal reference line in the volume data. Then, the control function 351 identifies voxels included between the upper plane and the lower plane with the horizontal reference line as the center, and generates a Mip image based on the voxel value of the identified voxel. That is, the control function 351 generates a Mip image with the projection direction perpendicular to the horizontal reference line (up and down direction) and displays it in the area 202d.

The display format of the reference line (color, thickness, type (solid line or dotted line, dotted line fineness, etc.)) may be preset or may be user-specified. In addition, when the triangular mark 303 arranged for the vertical reference line is moved, the triangular mark 303 arranged for the horizontal reference line may also be controlled to move by the same amount.

Icon 203p is a checkbox that switches whether or not to display an arbitrary two-dimensional cross-sectional image superimposed on an image showing a three-dimensional area, such as a rendering image such as a VR image or an SR image. When the checkbox is selected, the control function 351 switches between displaying and not displaying the superimposition.

FIG. 5 is a diagram showing an example of superimposed display of images according to the first embodiment. For example, by selecting icon 203p, control function 351 displays a cross-sectional image displayed in area 202b in correspondence with a three-dimensional position for a VR image occupying each leaflet of the aortic valve displayed in area 202a, as shown in FIG. 5. That is, control function 351 specifies the position of the cross-sectional image relative to the VR image based on the positional relationship between the position of the VR image in the volume data and the position of the cross-sectional image in the volume data. Then, control function 351 places the cross-sectional image at the specified position of the VR image, thereby displaying the superimposed image shown in area 202a in FIG. 5.

When the images are superimposed, the control function 351 displays the VR image located in front of the cross-sectional image in the observation direction, as shown in FIG. 5, but does not display the VR image located behind the cross-sectional image.

In FIG. 5, an example is described in which a cross-sectional image of area 202b is superimposed on a VR image, but the embodiment is not limited to this, and the cross-sectional image to be superimposed may be specified by the user. In such a case, for example, it may be specified using a context menu that is displayed by right-clicking on each area displaying a cross-sectional image. Also, an area that will be the cross-section to be superimposed may be determined in advance (for example, area 202b), and the cross-sectional image displayed in that area may be controlled to be superimposed.

Here, when superimposing a cross-sectional image on a VR image, the control function 351 matches the cross-sectional position and size of the cross-sectional image to the position and magnification of the VR image, but may also match other display conditions (such as window conditions) of the cross-sectional image to the display conditions of the VR image. In addition, the conditions for matching between images may be fixed to predetermined conditions. Note that, when the display conditions of the superimposed cross-sectional images are changed (slice forward, etc.) while the superimposed display is set, the control function 351 also changes the display conditions of the cross-sectional image superimposed on the VR image accordingly. For example, when a slice forward operation is performed on the cross-sectional image displayed in the area 202b, the control function 351 sequentially switches the cross-sectional images superimposed and displayed on the VR image to the cross-sectional images that are sequentially switched by the slice forward operation.

Icon 203q is a button that can return the display state of area 202 and area 204 to a specific display state, and when the button is pressed, control function 351 returns the display state of area 202 and area 204 to the specific display state. Here, the specific state is, for example, the state when the app is launched, the state when the image is first displayed in each area, the state before a specific number of operations, etc.

For example, the control function 351 may record the display conditions and display forms in the specific states described above, and restore them in response to pressing of the icon 203q. Alternatively, the control function 351 may record the display conditions and display forms for a certain period of time, and be able to continuously restore the display state after the previous or next operation (for example, the forward or back button).

Icon 203r is a button for displaying a setting screen for setting the display conditions of an image showing a three-dimensional region, such as a rendering image such as a VR image or an SR image, or an area to be superimposed on a two-dimensional image. For example, when a user selects icon 203r by operating the mouse, a setting screen for setting the display conditions of the area showing each leaflet in the VR image of the aortic valve leaflet displayed in area 202a in FIG. 3A, or the area showing each leaflet in the cross-sectional image displayed in areas 202b to 202d, is displayed. Note that the position information of the area of each leaflet is identified in step S102.

Figures 6A and 6B are diagrams showing an example of a setting screen for display conditions according to the first embodiment. For example, as shown in Figure 6A, the setting screen includes setting items for "Priority," "Color," "Transparency," "VR," "MPR," and "Area Name."

"Priority" sets the display priority of the specified area (specified from the "Area Name" combo box on the right). For example, the higher the area specified on the settings screen, the higher the display priority, and if multiple areas correspond to the same coordinates on the image, the area with the highest priority will be displayed.

"Color" is set to the color that is assigned when the corresponding area (specified from the "Area Name" combo box on the right) is superimposed on the VR image or cross-sectional image. For example, a color sample is displayed in "Color." For example, when the user selects an area that shows the sample color, the control function 351 displays a color map and an RGB value input box, as shown in FIG. 6B. The user can assign any color to the target area by selecting a color from the color map or entering RGB values.

In addition, the "transparency" sets the transparency of the VR image or cross-sectional image of the corresponding area (specified from the "area name" combo box on the right) that is superimposed on the corresponding area. For example, the "transparency" can be specified in 1% increments from 0 to 99% using a slider bar, and at 0%, the image is not transparent at all (i.e., the background image is not visible). Although not shown in Figures 6A and 6B, it is possible to set display conditions such as saturation and brightness, or to set textures instead of colors. In addition, all the transparencies can be set together by selecting the "linked" checkbox at the bottom left of the figure. When linking, all the transparencies can be controlled to be set to the same value, or the transparencies can be controlled to be increased or decreased overall while maintaining the relationship between the transparencies corresponding to each area at the time the "linked" checkbox is selected.

"VR" is a check box for specifying an area to be displayed on a VR image. Also, "MPR" is a check box for specifying an area to be displayed on an MPR image. Although not shown in Figures 6A and 6B, a button may be set that allows the user to simultaneously check or uncheck all of the "VR" or "MPR" check boxes.

The "Area Name" is set to the area to be displayed according to the set priority and display conditions. For example, the user specifies the area using a combo box arranged in the "Area Name" column. Note that it is possible to control so that the same area cannot be set in multiple combo boxes (for example, if a combo box specifies an area that has already been specified in another combo box, control may be performed so that it cannot be specified, or the setting of the existing combo box may be canceled), or it may be possible to specify an area but control so that the setting with the highest priority is used first. Note that in FIG. 6A, it is set so that the calcified area is displayed in white, the LCC (Left Coronary Cusp) of the aortic valve is displayed in green, the RCC (Right Coronary Cusp) is displayed in blue, and the NCC (Non Coronary Cusp) is displayed in yellow on the VR image and MPR image.

The "Close" button hides this setting screen, and the "Reset" button resets the settings to their initial state. The display conditions set on this setting screen can be reflected in each area immediately after each condition is set, or all at once after the "Close" button is selected.

(Display analysis results)
Area 204 includes areas 204a and 204b, and displays the analysis results. For example, based on each area of interest identified in step 102, control function 351 calculates values (measurement values) indicating the characteristics (measurement items) of the area of interest, and displays them in area 204. Area 204a displays the analysis results in a graph, and area 204b displays the analysis results in a list indicating the relationship between various measurement item names and the measurement values.

Here, the measurement items include, for example, morphological characteristics of each valve leaflet (e.g., "area," "effective height," "geometric height," "free margin length," "cusp insertion length," "commissural height," "valve orifice area," etc.), characteristics of the relationship between each valve leaflet (e.g., "coaptation height," "commissural distance"), morphological characteristics of the ascending aorta including the sinus of Valsalva (e.g., characteristics of the cross-sectional contour of the ascending aorta such as "annular ring," "commissure ring," "ST junction ring," "Valsalva ring," "basal ring," etc. ("area," "perimeter," "circularity," "aspect ratio," "major axis diameter," "minor axis diameter," etc.)), and characteristics of the relationship with the ostium of the coronary artery (e.g., "coronary ostium height").

For example, based on the area of each aortic valve leaflet identified in step 102, the control function 351 identifies the coordinates of "Nadir," which is the position closest to the LVOT (Left Ventricular Outflow Tract), for each leaflet (RCC, LCC, NCC). Then, the control function 351 extracts a cross section passing through the "Nadir" of each leaflet (hereinafter, the Nadir cross section), and calculates the "effective height," which is the linear distance from the cross section to the position of "Arantius." Note that "Arantius" is located at the center of the tip of each leaflet.

In addition, the control function 351 can calculate the "geometric height", which is the distance along the surface of the leaflet from "Nadir" to "Arantius", based on the area of each leaflet of the aortic valve identified in step 102, and can extract the commissures of each leaflet and calculate the "commissural distance", which is the straight-line distance between each commissure.

For example, the control function 351 displays a graph of the calculated measurement value in the area 204a. Figures 7A, 7B, and 7C are diagrams showing examples of graphs according to the first embodiment. Here, Figures 7A and 7B show line graphs showing the relationship between the measurement value and the cardiac phase, with the vertical axis representing the measurement value and the horizontal axis representing the cardiac phase. Also, Figure 7C shows a radar chart showing the characteristics of the valve leaflets at any cardiac phase.

For example, the control function 351 calculates the "effective height" and "geometric height" for each cardiac phase based on the area of the aortic valve leaflets identified in the volume data corresponding to each cardiac phase. The control function 351 then generates and displays each graph based on the calculated results.

Here, the control function 351 can generate and display a graph showing multiple measurement items for any one leaflet (LCC in FIG. 7A), as shown in FIG. 7A. The control function 351 can also generate and display multiple graphs to display graphs for all leaflets (LCC, RCC, NCC).

As shown in FIG. 7B, the control function 351 can also display the relationship between a single measurement item (in FIG. 7B, "geometric height") and each leaflet in the same graph. Note that the display of the line graph is not limited to the format of FIG. 7A and FIG. 7B, and it is also possible to combine FIG. 7A and FIG. 7B to display the values of multiple measurement items for each leaflet on a single graph. In such a case, the control function 351 can generate a graph in which lines representing the same leaflet or measurement item are displayed in the same color scheme.

In addition, the control function 351 can generate and display a radar chart showing the characteristics of each leaflet (LCC, RCC, NCC) at any cardiac phase, as shown in FIG. 7C.

The graph form may be controlled to be a form suitable for each measurement item. In such a case, for example, the relationship between the measurement item and the graph form is preset and stored. The control function 351 displays a check box to the left of the list of measurement items displayed in the area 204b, and when the check box is checked, the measurement results of the checked measurement item are displayed in the preset graph form.

The display format, such as the color and thickness of the line graph, may be set by the user, or may be changed to match the display format of each leaflet set on the setting screen shown in FIG. 6A.

As shown in FIG. 7A, the control function 351 can also perform control so that a straight line 601 is displayed at a position on the graph corresponding to the cardiac phase of the image displayed in the area 202, and the image displayed in the area 202 is changed to a corresponding image as the straight line 601 moves left or right (in the cardiac phase direction) on the graph based on the user's operation. In such a case, the control function 351 generates a VR image or a cross-sectional image from the volume data corresponding to the cardiac phase after the movement of the straight line 601, and displays them in the area 202.

In addition, when button 204c is selected, the control function 351 can output a table showing the relationship between the measurement values of various measurement items and the cardiac phase or slice in a file in a format such as CSV to a storage area on a computer specified by the user.

The controller 205 is a controller for cine display. The controller 205 is set with buttons such as a play button, a stop button, a speed increase button, a speed decrease button, a button to go back to the starting image, a button to go to the final image, a button to go to the image of the next cardiac phase, and a button to go to the image of the previous cardiac phase, and controls the execution of functions assigned to the various buttons when the user designates them by an operation such as clicking the mouse. The display order during cine display may be determined based on the selection order of thumbnails, or may be determined based on the order of imaging dates and times obtained from a DICOM header or the order of cardiac phases set based on the R-R interval. In addition, if the user gives some kind of instruction to the image during cine display (slice advance, translation, change of magnification, change of phase, measurement using various measurement functions, etc.), the controller may be hidden.

(Extraction process of noteworthy structures)
As described in step S102 of FIG. 2, the extraction function 353 extracts a region of interest from the volume data. Specifically, the extraction function 353 acquires coordinate information of a pixel indicating the aortic valve in the CT image. Here, the extraction function 353 can extract the region of interest by various methods. For example, the extraction function 353 can extract a region specified on the CT image via the input interface 32 as the region of interest. That is, the extraction function 353 extracts a region manually specified by the user as the region of interest.

Also, for example, the extraction function 353 can extract a region of interest based on an anatomical structure depicted in a CT image using a known region extraction technique. For example, the extraction function 353 extracts a region of interest in a CT image using Otsu's binarization method based on CT values, a region growing method, a snake method, a graph cut method, a mean shift method, or the like.

In addition, the extraction function 353 can extract regions of interest in CT images using a trained model constructed based on training data prepared in advance using machine learning techniques (including deep learning).

Here, if a process such as the graph cut method is performed on the entire image, the calculation cost may become excessively high. Therefore, the extraction function 353 can also target an area (hereinafter referred to as a related area) that is related to the region of interest and is larger than the region of interest but smaller than the entire image for the extraction process. For example, if the region of interest is the aortic valve, the extraction function 353 identifies the heart region, the left ventricle, and the area surrounding the left ventricle as the region of interest. Then, the extraction function 353 applies the above-mentioned extraction process only to the identified related area to extract the region of interest. Note that the related area may be set manually using the input interface 32.

The surrounding area of the region of interest may also be extracted using a similar method. For example, the extraction function 353 may extract not only the aortic valve leaflets, but also the area of the ascending aorta including the sinus of Valsalva, areas indicating cardiac cavities such as the left atrium, left ventricle, right atrium, and right ventricle, and the LVOT area. The extraction function 353 may also identify characteristic points in the aortic valve or the surrounding area based on the extracted area. For example, the extraction function 353 may identify the positions of the "Nadir", "Commissure", "Coronary Artery Ostium", etc.

(Processing for identifying initial cross-section position)
2, the identification function 354 identifies the region of each leaflet (LCC, RCC, NCC) based on the region of the aortic valve or the extraction result of the feature points, and identifies the initial cross-sectional position at which each identified leaflet is easily observed. Specifically, the identification function 354 sets a cross section perpendicular to the Nadir cross section (i.e., a cross section passing through three Nadirs) for each leaflet, and identifies the set cross section as the initial cross-sectional position.

Figure 8 is a diagram for explaining an example of the process of identifying the initial cross-sectional position according to the first embodiment. Note that Figure 8 shows a schematic diagram of identifying the initial cross-sectional position for each leaflet of the aortic valve. For example, as a method of setting a cross-section where the NCC is easily observed, the identification function 354 first identifies the midpoint position 705 of the line segment 704 connecting the Commissure position 701 between the NCC and the RCC and the Commissure position 702 between the NCC and the LCC in Figure 8. Then, the identification function 354 sets a cross-section that passes through the straight line 706 connecting the Commissure position 703 between the RCC and the LCC and the midpoint position 705 as the initial cross-section where the NCC is easily observed.

Similarly, as a method of setting an easily observable cross section of the LCC, the identification function 354 first identifies the midpoint position 707 of the line segment 708 connecting the Commissure position 703 between the LCC and the RCC and the Commissure position 702 between the LCC and the NCC in FIG. 8. The identification function 354 then sets a cross section that passes through a straight line 709 connecting the Commissure position 701 between the NCC and the RCC and the midpoint position 707, and is perpendicular to the Nadir cross section, as the initial cross section of the easily observable cross section of the LCC.

Similarly, as a method of setting a cross section that is easy to observe of the RCC, the identification function 354 first identifies the midpoint position 710 of the line segment 711 that connects the Commissure position 701 between the RCC and the NCC and the Commissure position 703 between the RCC and the LCC in FIG. 8. The identification function 354 then sets a cross section that passes through a straight line 712 that connects the Commissure position 702 between the NCC and the LCC and the midpoint position 710, and is perpendicular to the Nadir cross section, as the initial cross section of the cross section that is easy to observe of the RCC.

In the above-mentioned method, the three initial cross-section positions may not intersect at one point. If they do not intersect at one point, problems may occur in the following steps. Therefore, the method of setting the last cross-section (for example, a cross-section where the RCC is easily observed) may involve identifying the midpoint position 710 of a line segment 711 connecting the Commissure position 701 between the RCC and the NCC and the Commissure position 703 between the RCC and the LCC, and setting the line 712 connecting the intersection point 713 between the previously identified line 706 and line 709 and the midpoint position 710, and setting the cross-section that passes through the line 712 and is perpendicular to the Nadir cross-section as the initial cross-section where the RCC is easily observed. The above-mentioned method is merely an example, and any method may be used to identify the cross-section position.

(Valve Leaflet Determination Processing)
As described in step S104 of Fig. 2, the determination function 355 determines the leaflets that are easy to observe at the set cross-sectional position. Specifically, the determination function 355 determines the type of the leaflets included in the set cross-sectional position. For example, the determination function 355 determines a range in the cross-sectional area to be subjected to type determination based on structural information of a plurality of leaflets, calculates an index using information included in the determined range, and determines the type of the leaflets based on the calculated index.

Figure 9 is a diagram for explaining an example of a leaflet determination process according to the first embodiment. Here, Figure 9 shows a schematic diagram of determining the type of leaflet included in the cross section set in Figure 8. For example, the determination function 355 determines the type of leaflet that is easy to observe for each of a cross section passing through a straight line 706 and perpendicular to the Nadir cross section, a cross section passing through a straight line 709 and perpendicular to the Nadir cross section, and a cross section passing through a straight line 712 and perpendicular to the Nadir cross section.

Here, the determination function 355 determines the range for each cross section that is the subject of type determination. For example, the determination function 355 determines range 715 as the range for type determination in a cross section that passes through the straight line 706 and is perpendicular to the Nadir cross section. Range 715 is the range on the side of the cross section that passes through the straight line 706 and is perpendicular to the Nadir cross section that does not include the Commissure position 703, with the intersection point 713 as the boundary (i.e., the range that passes through range 715 and is perpendicular to the Nadir cross section). Similarly, the determination function 355 determines range 714 as the range for type determination in a cross section that passes through the straight line 709 and is perpendicular to the Nadir cross section, and determines range 716 as the range for type determination in a cross section that passes through the straight line 709 and is perpendicular to the Nadir cross section.

In this way, for each of the three cross sections set as the initial cross sections, the determination function 355 determines the range that does not include the Commissure position out of the two ranges with the intersection point 713 as the boundary as the range to be subjected to type determination.

Then, the determination function 355 calculates an index for determining the leaflets for each of the three cross sections, using the determined range as the target. For example, the determination function 355 calculates the size (number of pixels) of the area of each leaflet in the determined range, and determines the type of leaflet based on the calculation.

For example, the determination function 355 calculates the number of pixels corresponding to each leaflet region for the region corresponding to range 715 in the cross-sectional image indicated by line 706, and determines the leaflet indicated by the region with the largest number of pixels as the leaflet that is easiest to observe in the cross-sectional image indicated by line 706. Here, at the initial cross-sectional position shown in FIG. 9, the leaflet that is easiest to observe in the cross-sectional image indicated by line 706 is determined to be NCC.

Similarly, the determination function 355 calculates the number of pixels corresponding to each leaflet region for the region corresponding to range 714 in the cross-sectional image indicated by line 709, and determines which leaflets are easy to observe. The determination function 355 also calculates the number of pixels corresponding to each leaflet region for the region corresponding to range 716 in the cross-sectional image indicated by line 712, and determines which leaflets are easy to observe.

In the above example, the case where the range for calculating the number of pixels is determined based on the Commissure position has been described. However, the embodiment is not limited to this, and the range for calculating the number of pixels may be determined based on other positions. In FIG. 9, the range is determined based on the initial cross-section position, so one of the two ranges with the intersection point 713 as the boundary includes the Commissure position, but if the position of the cross-section is moved (for example, if the three cross-sections are rotated), the Commissure position may no longer be included in the cross-section.

Therefore, the determination function 355 may determine the range using, for example, a line segment connecting the Commissure positions, or the midpoint of the line segment. As one example, the determination function 355 may determine the range based on the angle of a straight line showing the cross section relative to a line segment connecting the Commissure positions, or the distance between the midpoint of the line segment and the straight line showing the cross section.

10A and 10B are diagrams for explaining an example of a process for determining a range of a determination target according to the first embodiment. Here, FIG. 10A and FIG. 10B show a state in which a section perpendicular to the Nadir section through a straight line 706 in FIG. 9, a section perpendicular to the Nadir section through a straight line 709, and a section perpendicular to the Nadir section through a straight line 712 are rotated clockwise around an intersection point 713. FIG. 10A also shows a case in which the type of leaflet that is easy to observe is determined using the angle of a straight line showing the section with respect to a line segment connecting the Commissure positions. FIG. 10B also shows a case in which the type of leaflet that is easy to observe is determined using the distance between the midpoint of a line segment connecting the Commissure positions and the straight line showing the section.

For example, the determination function 355 extracts line segments (line segments 704 and 711) that connect the Commissure positions that intersect with the line 706 shown in FIG. 10A and are perpendicular to the Nadir section. The determination function 355 then calculates the angle of the line 706 with respect to each of the extracted line segments, and determines the range of the two ranges with the intersection point 713 as the range for which the number of pixels is to be calculated, the range with the angle closer to 90 degrees. For example, the determination function 355 calculates the angles "θ1" and "θ2" shown in FIG. 10A, and determines the range 717 on the "θ1" side, the angle closer to 90 degrees, as the range to be determined for the type. The determination function 355 similarly determines the ranges to be determined for the type for the section that passes through the line 709 and is perpendicular to the Nadir section, and the section that passes through the line 712 and is perpendicular to the Nadir section.

For example, the determination function 355 extracts line segments (line segments 708 and 711) that connect the Commissure positions that intersect with the line 712 in the cross section that passes through the line 712 shown in FIG. 10B and is perpendicular to the Nadir cross section. The determination function 355 then calculates the distance between the midpoint of each extracted line segment and the line 712, and determines the range with the shorter distance to the midpoint of the two ranges with the intersection point 713 as the range for which the number of pixels is to be calculated. For example, the determination function 355 calculates the distances "D1" and "D2" shown in FIG. 10B, and determines the range 718 on the "D2" side with the shorter distance as the range for which the type is to be determined. The determination function 355 similarly determines the cross section that passes through the line 706 and is perpendicular to the Nadir cross section and the cross section that passes through the line 709 and is perpendicular to the Nadir cross section as the range for which the type is to be determined.

(Display mode setting process)
As described in step S105 of FIG. 2, the control function 351 sets the display form of the leaflets and the display form of the information on the displayed cross section. Specifically, the control function 351 determines the display form of the information on the displayed cross section based on the display form of each of the plurality of leaflets and the determination result of the type of the leaflets included in the cross section image. For example, the control function 351 refers to the display form (color, etc.) of each leaflet set in the setting screen shown in FIG. 6A, and determines the display form of the information on the cross section image in which the corresponding leaflet is easily observed to match the display form set in the setting screen. As an example, the control function 351 sets the display form so that the display form (color, etc.) of the LCC set in the setting screen shown in FIG. 6A matches the display form of the information on the cross section image in which the LCC is easily observed.

Here, the control function 351 targets, as information related to the cross-sectional image, for example, reference information indicating the position of the cross-section and the display area of the cross-sectional image. For example, the control function 351 sets the display form of the reference line indicating the position of the cross-section and the display area frame of the cross-sectional image so that they are the same display form as the valve leaflets. This makes it easier to understand the relationship between the reference line and the display area and the valve leaflets. More specifically, for example, if the LCC area is set to be displayed in red, the color of the reference line indicating the cross-section determined to be an LCC-easy-to-observe cross-section and the color of the frame of the display area displaying that cross-section are also set to red.

When multiple cross sections are determined to be cross sections that make it easy to observe the same valve leaflet, the reference lines indicating the multiple cross sections and the frames of the display areas that display the cross sections may be set to the same display form, but in order to distinguish between them, different display forms may be set based on the size of the index (in this embodiment, the number of pixels) used in the determination in step S104. In other words, by changing the values of some systems of brightness, transparency, and RGB values based on the size of the index, it is possible to set the same system but different display forms. In other words, when it is determined in the type determination result that the same valve leaflet is included in multiple cross sections, the control function 351 determines the display form of the information related to the cross sections based on the type determination result and the indexes calculated from each cross section.

Any other display form may be set based on the judgment in step 104. For example, the index (in this embodiment, the number of pixels) used for the judgment in step 104 may be compared for each cross section, and the size of the display area may be changed based on the size of the index.

(Display processing of cross-sectional images)
2, the control function 351 displays the cross-sectional image based on the display form set in step S105. For example, the control function 351 displays the reference line and the display area frame of the cross section in a display form associated with the display form of the valve leaflets.

11A is a diagram showing an example of display processing according to the first embodiment. For example, as shown in FIG. 11A, the control function 351 expresses the color of the reference line 810a indicating the cross section where the NCC is easily observed and the color of the frame 811a of the area 801 displaying the cross section in the same color (e.g., green) as the color that represents the NCC area 812a in the VR image. Similarly, the control function 351 expresses the color of the reference line 810b indicating the cross section where the RCC is easily observed and the color of the frame 811b of the area 802 displaying the cross section in the same color (e.g., orange) as the color that represents the RCC area 812b in the VR image. Similarly, the control function 351 expresses the color of the reference line 810c indicating the cross section where the LCC is easily observed and the color of the frame 811c of the area 803 displaying the cross section in the same color (e.g., light blue) as the color that represents the LCC area 812c in the VR image. Furthermore, the control function 351 can also display text indicating an easily observable cross section in each area by overlaying it, as shown in area 804. The control function 351 can also display the text in a similar color.

As described above, the reference line according to this embodiment can accept various operations. Below, we will explain the operations performed on the reference line on the display screen that differ from those explained in step S101. Since this is merely an explanation of the operation system, we will not explain the change in display form (steps S107 and after) caused by changing the cross-sectional position by operating the reference line.

FIGS. 11B to 11D are diagrams showing an example of the display process according to the first embodiment. Here, FIG. 11B shows an example of a screen when the intersection of the three reference lines (810a, 810b, 810c) is moved to the upper left of the screen. When the intersection of the reference lines (810a, 810b, 810c) is moved in this way, the cross-sectional images displayed in areas 801 to 803 change, as described in FIG. 4B.

Here, since this screen is a screen for observing the region of interest (aortic valve), the movement range of the intersection may be controlled to be limited. Specifically, the control function 351 controls the setting range of the display cross section related to the heart valve based on the extraction result of the region corresponding to the heart valve. For example, the control function 351 determines the control range based on the region or feature points indicating the valve leaflets and surrounding structures extracted in step S102. As an example, the control function 351 identifies the region indicating the sinus of Valsalva indicated by the curve 805 based on the extraction result in step S102, and controls so that the intersection cannot move outside the identified region.

Figure 11C shows an example of a case where the reference line 810b, which indicates a cross section where the RCC can be easily observed, is translated downward. Here, the control function 351 accepts a setting to move the position of at least one of the multiple displayed cross sections relating to the heart valve, and in displaying the reference lines indicating the positions of the multiple cross sections, the reference line indicating the position of the cross section before the cross section whose position has been moved and the reference line indicating the position after the cross section have been moved are displayed in similar display forms.

For example, in order to maintain the display of the intersection position of the three reference lines, the control function 351 does not move the original reference line 810b, but displays a reference line 810d in a similar display form (same color, etc.) as the original reference line 810b at the moved position. This control is similar when the region 802 is sliced (browsed). In other words, the control function 351 displays both the reference line 810b indicating the rotation and the reference line 810d indicating the cross-sectional position.

Figure 11D is an example of a screen when the reference line 810b, which indicates a cross section where the RCC can be easily observed, is moved 45 degrees clockwise from the state of Figure 11C. Here, the control function 351 accepts a rotation operation of the displayed cross section whose position has been moved by the reference line 810b indicating the position before the movement, and displays the reference line 810b indicating the position before the movement and the reference line 810d indicating the position after the movement in a position according to the rotation operation while maintaining the positional relationship between the reference line 810b indicating the position before the movement and the reference line 810d indicating the position after the movement. In other words, when a rotation operation is performed after a parallel translation, the control function 351 rotates while maintaining the relationship between the two reference lines (parallel and maintaining the distance between the lines).

As shown in FIG. 11D, when a parallel translation is applied to the reference line 810b, both the reference line 810b indicating the rotation and the reference line 810d indicating the cross-sectional position are displayed. The display form of both of these reference lines may be adjusted to the display form set in step S105, or only one of them may be adjusted.

Here, we will provide additional information regarding the display conditions for each cross-sectional image in areas 801 to 803. It is preferable to display the cross-sectional images in areas 801 to 803 in a way that makes them easy to compare. Therefore, the control function 351 adjusts the display positions of multiple cross sections related to the heart valve based on the morphological information of the heart valve. For example, when displaying multiple cross-sectional images, the control function 351 applies a certain amount of rotation to the cross-sectional images.

Figure 12 is a diagram for explaining an example of the display process of the cross-sectional image according to the first embodiment. For example, the control function 351 identifies the position of "Nadir" for each leaflet based on the region of interest extracted in step S102. Then, the control function 351 controls the display so that the position of "Nadir" for each region is in the same direction and position, as shown by the straight line 900 in Figure 12. Note that the control function 351 can also control so that the leaflet determined to be easy to observe is displayed on the left side as viewed from the front.

(Cross-section position change process)
2, the control function 351 can accept a movement of the position of the cross-sectional image displayed in step S106. Here, when a movement of the position of the cross-sectional image is accepted, the process returns to step S104, and the cross-sectional position is determined.

(End Determination Process)
2, the control function 351 determines whether the user ends the image interpretation. For example, the control function 351 determines whether the image interpretation is ended by the user designating an image interpretation end button (not shown) or the like.

(Variation 1)
In the above-described embodiment, in steps S104 and S105, each cross section is determined based on the area of the valve leaflet, and the display form such as the color of the reference line is determined. However, the embodiment is not limited to this, and the display form may be determined using other information. Specifically, the control function 351 determines the display form of the information related to the cross section based on the display form and type determination result of each of the multiple valve leaflets, as well as the properties of the valve leaflets.

For example, the control function 351 further changes the display form for the easily observable leaflets identified in step S104 based on the characteristics of the leaflets. As one example, if the cross section shows calcification in an easily observable leaflet, the control function 351 may change the thickness of the reference line. In addition to the thickness, the control function 351 can also change the display form to surround the reference line itself with a different color, or can change the form of the reference line (solid line, dotted line, dotted line fineness, etc.). This allows the user to grasp the characteristics of the leaflets, such as the presence or absence of calcification in the cross section, by observing the reference line.

Here, an example of the process of identifying calcification in the valve leaflets by the control function 351 will be described with reference to Figs. 13A, 13B, and 14. Figs. 13A and 13B are diagrams showing an example of volume data according to Modification 1. Fig. 14 is a diagram for explaining an example of the process of identifying calcification according to Modification 1.

As shown in Figures 13A and 13B, in this modified example, for the sake of simplicity, it is assumed that the image acquisition function 352 has acquired an image data set of volume data that includes eight slice images S1 to S8 arranged in the cranio-caudal direction, and each slice image includes 10 pixels in the X direction and 10 pixels in the Y direction.

The control function 351 identifies calcified regions in the eight slice images S1 to S8. Here, the identification of calcified regions is appropriately performed using a known method. For example, as shown in FIG. 14, the control function 351 identifies calcified regions (pixels marked in black in FIG. 14) in each of the eight slice images S1 to S8, thereby identifying calcified regions in the entire volume data.

Then, the control function 351 identifies the position of the cross-sectional image in the volume data, and determines whether or not each cross-sectional image contains a calcified region. For example, the control function 351 determines that the cross-section indicated by the reference line L1 in FIG. 14 contains calcification (pixels colored black) at corresponding positions in the slice image S2 and the slice image S8. In this case, the control function 351 changes the display form of the reference line L1. For example, the control function 351 changes the thickness of the reference line L1, or frames the reference line in red, as shown in FIG. 14.

On the other hand, the cross section indicated by reference line L2 and the cross section indicated by reference line L3 in FIG. 14 do not include calcification, so the control function 351 does not change the display form for either reference line.

The additional display form determined in this modified example may or may not be reflected in step S106. In other words, only the color of the frame of the display area may be reflected, and changes in the display form due to the presence or absence of calcification (thickness, frame color, solid or dotted line, etc.) may not be reflected. In addition, the properties of the leaflets are not limited to calcification, and the positions of abnormal forms such as holes or tears in the leaflets may be detected, and the display form may be controlled to be changed if the abnormal form is present in each cross section.

(Variation 2)
In the above embodiment, the reference line is translated in the slice direction. However, the embodiment is not limited to this, and the reference line may be moved in any direction. Figures 15A and 15B are diagrams showing an example of the movement of the reference line according to the second modification.

The control function 351 accepts movement of the cross section to a position based on the position of the tip of the junction in the heart valve. For example, the control function 351 acquires the position of the tip of the junction of each leaflet based on the extraction result of the region of interest (heart valve) in step S102. As an example, the control function 351 acquires the positions 1001 to 1005 of the tips of the junction of the leaflet as shown in FIG. 15A. Note that, for convenience of explanation, FIG. 15A shows a case where five positions of the tips of the junction are acquired, but the embodiment is not limited to this, and any number can be acquired. Also, for convenience of explanation, FIG. 15A shows the positions of the tips of the junction on a plane in the figure, but in reality, the positions of the tips of the junction are different positions in the depth direction of the figure.

As shown in FIG. 15A, when the position of the tip of the joint is obtained, the control function 351 identifies the line segment connecting each tip and identifies the identified line segment or the line segment perpendicularly projected onto the observation section (the section shown in FIG. 15A). For example, the control function 351 identifies the line segment 1006 obtained by perpendicularly projecting the line segment connecting the positions 1001 and 1002 onto the observation section. Similarly, the control function 351 identifies each of the line segments obtained by perpendicularly projecting the line segments connecting the respective positions onto the observation section. Note that the line segment connecting the tips may be a straight line connecting the two tips at the closest positions, or may be a curved line connecting the positions of the multiple tips using a known numerical analysis technique such as spline interpolation.

Then, the control function 351 moves the reference line in a direction perpendicular to each of the identified line segments. For example, the control function 351 moves the reference line L4 to the position of the line segment L5 perpendicular to the line segment 1006. Similarly, the control function 351 moves the reference line L4 in a direction perpendicular to each of the identified line segments.

The control function 351 also accepts movement of the cross section to a position based on the position of the leaflets included in the cross section. For example, the control function 351 acquires the position of each leaflet included in the cross section image. As an example, the control function 351 acquires a curve 1007 indicating the position of the junction side of one leaflet in the cross section image, as shown in FIG. 15B. The control function 351 then moves the reference line L4 in a vertical direction at each position of the acquired curve 1007. For example, the control function 351 moves the reference line L4 along the curve 1007 to the position of the line segment L6 or line segment L7 that intersects at right angles at each position.

(Variation 3)
The size of the display area of the image described in the above embodiment may be set by an operation on the reference line or a display state of the VR image. Figures 16A and 16B are diagrams showing an example of a change in the display area according to Modification 3.

For example, the control function 351 sets the size of the display area of the cross-sectional image in response to an operation on the reference lines 810a to 810c shown in FIG. 16A. As an example, the user performs an operation such as rotation or translation on the reference line 810b indicating the cross-section where the RCC is easily observed. In such a case, the control function 351 sets the area 802 displaying the cross-sectional image where the RCC is easily observed to a size larger than the other display areas (areas 801 and 803) as shown in FIG. 16A, and displays the cross-sectional image where the RCC is easily observed in the set area 802.

Also, for example, the control function 351 sets the size of the display area of the cross-sectional image according to the display state of the VR image displayed in the area 813 in FIG. 16B. As an example, the control function 351 determines the area displayed on the front side of the screen in the VR image displayed in the area 813, and sets the display area displaying the cross-section in which the determined area is easy to observe to a size larger than the other display areas. For example, as shown in FIG. 16B, if the area displayed on the front side of the screen in the VR image in the area 813 is the RCC area 812b, the control function 351 sets the area 802 displaying the cross-sectional image in which the RCC is easy to observe to a size larger than the other display areas (areas 801 and 803), and displays the cross-sectional image in which the RCC is easy to observe in the set area 802.

The display state of the VR image may be not only the front side of the screen, but also the size of the displayed leaflet region. In such a case, for example, the control function 351 calculates the size of each leaflet region (NCC region 812a, RCC region 812b, LCC region 812c) in the VR image displayed in region 813. Then, the control function 351 sets the region for displaying the cross-sectional image of the leaflet with the largest calculated size to a size larger than the other display regions.

(Variation 4)
In the above-described embodiment, a case where a normal tricuspid aortic valve is the target has been described. However, the embodiment is not limited to this, and may be, for example, a case where a quadricuspid aortic valve is the target. In such a case, the medical image processing device 3 displays the display form of the valve leaflets and the display form of the information on the display cross section (reference lines and display area frames) in association with each other for each of the four valves.

As described above, according to the first embodiment, the image acquisition function 352 acquires volume data including at least a cardiac valve. The extraction function 353 extracts an area corresponding to a cardiac valve included in the volume data. The identification function 354 identifies each of the multiple valve leaflets included in the cardiac valve. The control function 351 displays each of the multiple valve leaflets in a different display form. The control function 351 accepts the setting of a display cross section related to the cardiac valve. The control function 351 sets information regarding the display cross section to a display form that corresponds to the display form of the valve leaflet corresponding to the position of the set display cross section among the multiple valve leaflets. Therefore, the medical image processing device 3 according to the first embodiment can display the display form of the valve leaflet in association with information regarding the display cross section that makes it easy to observe the valve leaflet, making it possible to easily identify the displayed valve leaflet. As a result, the medical image processing device 3 makes it possible to suppress the occurrence of mistakes in identifying the valve leaflet.

Furthermore, according to the first embodiment, the determination function 355 determines the type of valve leaflets included in the displayed cross section. The control function 351 determines the display form of the information on the displayed cross section based on the display form of each of the multiple valve leaflets and the type determination result. Therefore, the medical image processing device 3 according to the first embodiment can determine which valve leaflets are easy to observe on the displayed cross section, and enables accurate correspondence between the display form of the valve leaflets and information on the displayed cross section on which the valve leaflets are easy to observe.

Furthermore, according to the first embodiment, the determination function 355 determines the range in the displayed cross section to be used for type determination based on structural information of multiple leaflets, calculates an index using information included in the determined range, and determines the type of leaflet based on the calculated index. Therefore, the medical image processing device 3 according to the first embodiment makes it possible to accurately determine the leaflet that is easy to observe in the displayed cross section.

Furthermore, according to the first embodiment, the determination function 355 determines the ranges in the multiple display cross sections based on the structural information of the multiple leaflets, calculates an index for each determined range, and determines the type of each leaflet included in the multiple display cross sections based on the calculated indexes. When it is determined in the type determination result that the same leaflet is included in the multiple display cross sections, the control function 351 determines the display form of the information on the display cross sections based on the type determination result and the indexes calculated from each display cross section. Therefore, the medical image processing device 3 according to the first embodiment makes it possible to more accurately determine which leaflets are easy to observe.

Furthermore, according to the first embodiment, the control function 351 displays a reference line indicating the position of the display cross section and a display area frame of the display cross section in a display form associated with the display form of the leaflet corresponding to the position of the set display cross section among the multiple leaflets. Therefore, the medical image processing device 3 according to the first embodiment makes it possible to easily understand the relationship between the reference line or display area and the leaflets.

Furthermore, according to the first embodiment, the control function 351 controls the setting range of the display cross section related to the heart valve based on the extraction result of the area corresponding to the heart valve. Therefore, the medical image processing device 3 according to the first embodiment enables the setting of an appropriate display cross section.

Furthermore, according to the first embodiment, the control function 351 accepts a setting to move the position of at least one of the multiple display cross sections related to the heart valve. Furthermore, in displaying the reference information indicating the positions of the multiple display cross sections, the control function 351 displays the reference information indicating the position of the moved display cross section before the movement and the reference information indicating the position after the movement in a similar display form. Therefore, the medical image processing device 3 according to the first embodiment makes it possible to realize an easy-to-operate reference line.

Furthermore, according to the first embodiment, the control function 351 accepts a rotation operation of the displayed cross section whose position has been moved, using reference information indicating the position before the movement. Furthermore, the control function 351 displays the reference information indicating the position before the movement and the reference information indicating the position after the movement at a position according to the rotation operation, while maintaining the positional relationship between the reference information indicating the position before the movement and the reference information indicating the position after the movement. Therefore, the medical image processing apparatus 3 according to the first embodiment makes it possible to accurately grasp the positional relationship between the observation cross section and the rotation position.

Furthermore, according to the first embodiment, the control function 351 adjusts the display positions of multiple display cross sections related to the heart valve based on the morphological information of the heart valve. Therefore, the medical image processing device 3 according to the first embodiment makes it possible to display cross-sectional images that are easy to observe.

Furthermore, according to the first embodiment, the control function 351 determines the display form of the information regarding the displayed cross section based on the display form and type determination result of each of the multiple leaflets, as well as the properties of the leaflets. Therefore, the medical image processing device 3 according to the first embodiment makes it possible to grasp abnormalities related to the properties of the leaflets at a glance.

Furthermore, according to the first embodiment, the control function 351 accepts movement of the display cross section to a position based on the position of the tip of the joint in the heart valve. Furthermore, the control function 351 accepts movement of the display cross section to a position based on the position of the valve leaflets included in the display cross section. Therefore, the medical image processing device 3 according to the first embodiment makes it possible to display various display cross sections.

Other Embodiments
In the above embodiment, an example is described in which the image of the valve leaflet is displayed on the display 33 of the medical image processing device 3, but the embodiment is not limited to this. For example, the image of the valve leaflet may be displayed on the display of another device connected to the network.

In the above-mentioned embodiment, an example was described in which the control unit, image acquisition unit, extraction unit, identification unit, and judgment unit in this specification are realized by the control function, image acquisition function, extraction function, identification function, and judgment function of the processing circuit, respectively, but the embodiment is not limited to this. For example, in addition to being realized by the control function, image acquisition function, extraction function, identification function, and judgment function described in the embodiment, the control unit, image acquisition unit, extraction unit, identification unit, and judgment unit in this specification may also be realized by hardware only, software only, or a combination of hardware and software.

The term "processor" used in the description of the above embodiment means, for example, a circuit such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an Application Specific Integrated Circuit (ASIC), or a programmable logic device (for example, a Simple Programmable Logic Device (SPLD), a Complex Programmable Logic Device (CPLD), and a Field Programmable Gate Array (FPGA)). Here, instead of storing a program in a memory circuit, the program may be directly built into the processor circuit. In this case, the processor realizes its function by reading and executing the program built into the circuit. In addition, each processor in this embodiment is not limited to being configured as a single circuit for each processor, but may be configured as a single processor by combining multiple independent circuits to realize its function.

Here, the medical image processing program executed by the processor is provided by being pre-installed in a ROM (Read Only Memory) or a storage circuit, etc. Note that the medical image processing program may be provided by being recorded in a computer-readable non-transient storage medium such as a CD (Compact Disk)-ROM, FD (Flexible Disk), CD-R (Recordable), or DVD (Digital Versatile Disk) in a format that can be installed or executed by these devices. The medical image processing program may also be provided or distributed by being stored on a computer connected to a network such as the Internet and downloaded via the network. For example, the medical image processing program is composed of modules including each of the above-mentioned processing functions. As actual hardware, the CPU reads out the medical image processing program from a storage medium such as a ROM and executes it, so that each module is loaded onto the main storage device and generated on the main storage device.

In addition, in the above-mentioned embodiment and modified examples, each component of each device shown in the figures is a functional concept, and does not necessarily have to be physically configured as shown in the figures. In other words, the specific form of distribution or integration of each device is not limited to that shown in the figures, and all or part of it can be functionally or physically distributed or integrated in any unit depending on various loads, usage conditions, etc. Furthermore, each processing function performed by each device can be realized in whole or in any part by a CPU and a program analyzed and executed by the CPU, or can be realized as hardware using wired logic.

Furthermore, among the processes described in the above-mentioned embodiments and variations, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using known methods. In addition, the information including the processing procedures, control procedures, specific names, various data, and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified.

At least one of the embodiments described above makes it easy to identify the displayed leaflets.

Although several embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are within the scope of the invention and its equivalents as set forth in the claims, as well as the scope and spirit of the invention.

3 Medical image processing device 35 Processing circuit 351 Control function 352 Image acquisition function 353 Extraction function 354 Identification function 355 Judgment function

Claims (22)

an acquisition unit for acquiring volume data including at least a cardiac valve;
an extraction unit that extracts a region corresponding to the cardiac valve included in the volume data;
An identification unit that identifies each of a plurality of leaflets included in the cardiac valve;
A display control unit that displays each of the plurality of leaflets in a different display form;
a reception unit that receives settings of a display cross section related to the heart valve;
a determination unit that determines a type of leaflet included in the display cross section received by the reception unit;
Equipped with
The display control unit determines a display form of the information regarding the display cross section based on a display form of each of the plurality of leaflets and a result of the determination of the type .
Medical imaging equipment. 2. The medical image processing device according to claim 1, wherein the determination unit determines a range in the displayed cross section to be used for determining the type based on structural information of the plurality of leaflets, calculates an index using information included in the determined range, and determines the type of the leaflet based on the calculated index . the determination unit determines the ranges in the plurality of display cross sections based on the structural information of the plurality of leaflets, calculates the indexes for each of the determined ranges, and determines the types of the leaflets included in the plurality of display cross sections based on the calculated indexes;
3. The medical image processing device according to claim 2, wherein when the type determination result determines that the same valve leaflet is included in multiple displayed sections, the display control unit determines a display form of information regarding the displayed sections based on the type determination result and each index calculated from each displayed section. The medical image processing device according to any one of claims 1 to 3, wherein the display control unit displays reference information indicating the position of the display cross section and a display area of the display cross section in a display form associated with a display form of a leaflet corresponding to the position of the display cross section set by the reception unit among the plurality of leaflets . The medical image processing device according to claim 1 , wherein the receiving unit controls a setting range of a display cross section relating to the cardiac valve based on a result of extraction of a region corresponding to the cardiac valve. the receiving unit receives a setting to move a position of at least one of a plurality of display cross sections related to the heart valve;
The medical image processing device according to any one of claims 1 to 5, wherein, in displaying the reference information indicating the positions of the plurality of display cross sections, the display control unit displays the reference information indicating the position of the display cross section before the movement and the reference information indicating the position after the movement in a similar display form. the reception unit receives a rotation operation of the display cross section whose position has been moved, using reference information indicating the position before the movement;
7. The medical image processing device according to claim 6, wherein the display control unit displays the reference information indicating the position before the movement and the reference information indicating the position after the movement at a position corresponding to the rotation operation while maintaining a positional relationship between the reference information indicating the position before the movement and the reference information indicating the position after the movement . The medical image processing apparatus according to claim 1 , wherein the display control unit adjusts display positions of a plurality of display cross sections relating to the cardiac valve based on morphological information of the cardiac valve. The medical image processing device according to any one of claims 1 to 8, wherein the display control unit determines the display form of the information regarding the displayed cross section based on the display form of each of the plurality of leaflets and the determination result of the type, as well as the characteristics of the leaflets. The medical image processing apparatus according to claim 1 , wherein the receiving unit receives a command to move the displayed cross section to a position based on a position of a tip of a joint in the heart valve. The medical image processing apparatus according to claim 1 , wherein the receiving unit receives a command to move the displayed cross section to a position based on a position of a valve leaflet included in the displayed cross section. 1. A processor-implemented medical image processing method, comprising:
acquiring volumetric data including at least a cardiac valve;
extracting a region corresponding to the cardiac valve included in the volume data;
identifying each of a plurality of leaflets included in the cardiac valve;
displaying each of the plurality of leaflets in a different display format;
receiving a setting of a display cross section relating to the heart valve;
determining a type of leaflet included in the received display cross section;
determining a display form of information regarding the display cross section based on a display form of each of the plurality of leaflets and a result of the determination of the type;
A medical image processing method comprising : an acquisition function for acquiring volume data including at least a heart valve;
an extraction function for extracting a region corresponding to the cardiac valve included in the volume data;
An identification function for identifying each of a plurality of leaflets included in the cardiac valve;
A display control function for displaying each of the plurality of leaflets in a different display form;
a reception function for receiving settings of a display cross section relating to the heart valve;
a determination function for determining a type of leaflet included in the display cross section received by the reception function;
on the computer,
The display control function determines a display form of information regarding the display cross section based on the display form of each of the plurality of leaflets and the determination result of the type . an acquisition unit for acquiring volume data including at least a cardiac valve;
an extraction unit that extracts a region corresponding to the cardiac valve included in the volume data;
An identification unit that identifies each of a plurality of leaflets included in the cardiac valve;
A display control unit that displays each of the plurality of leaflets in a different display form;
a reception unit that receives settings of a display cross section related to the heart valve;
Equipped with
The display control unit displays reference information indicating the position of the display cross section and a display area of the display cross section in a display form associated with a display form of a leaflet corresponding to the position of the display cross section set by the reception unit among the plurality of leaflets.
Medical imaging equipment. an acquisition unit for acquiring volume data including at least a cardiac valve;
an extraction unit that extracts a region corresponding to the cardiac valve included in the volume data;
An identification unit that identifies each of a plurality of leaflets included in the cardiac valve;
A display control unit that displays each of the plurality of leaflets in a different display form;
a receiving unit that receives a setting for moving a position of at least one of a plurality of display cross sections relating to the cardiac valve;
Equipped with
the display control unit, in displaying the reference information indicating the positions of the plurality of display cross sections, displays the reference information indicating the position before the movement of the display cross section whose position has been moved and the reference information indicating the position after the movement in a similar display form, respectively.
Medical imaging equipment. an acquisition unit for acquiring volume data including at least a cardiac valve;
an extraction unit that extracts a region corresponding to the cardiac valve included in the volume data;
An identification unit that identifies each of a plurality of leaflets included in the cardiac valve;
A display control unit that displays each of the plurality of leaflets in a different display form;
a reception unit that receives settings of a display cross section related to the heart valve;
Equipped with
The display control unit adjusts display positions of the plurality of display cross sections related to the cardiac valve set by the reception unit based on the morphological information of the cardiac valve.
Medical imaging equipment. 1. A processor-implemented medical image processing method, comprising:
acquiring volumetric data including at least a cardiac valve;
extracting a region corresponding to the cardiac valve included in the volume data;
identifying each of a plurality of leaflets included in the cardiac valve;
displaying each of the plurality of leaflets in a different display format;
receiving a setting of a display cross section relating to the heart valve;
displaying reference information indicating the position of the display cross section and a display area of the display cross section in a display form associated with a display form of a leaflet corresponding to the position of the display cross section set among the plurality of leaflets;
A medical image processing method comprising: 1. A processor-implemented medical image processing method, comprising:
acquiring volumetric data including at least a cardiac valve;
extracting a region corresponding to the cardiac valve included in the volume data;
identifying each of a plurality of leaflets included in the cardiac valve;
displaying each of the plurality of leaflets in a different display format;
receiving a setting for moving a position of at least one of a plurality of display slices relating to the cardiac valve;
a step of displaying, in a display of reference information indicating positions of the plurality of display cross sections, reference information indicating a position before the movement of the display cross section whose position has been moved and reference information indicating a position after the movement in a similar display form;
A medical image processing method comprising: 1. A processor-implemented medical image processing method, comprising:
acquiring volumetric data including at least a cardiac valve;
extracting a region corresponding to the cardiac valve included in the volume data;
identifying each of a plurality of leaflets included in the cardiac valve;
displaying each of the plurality of leaflets in a different display format;
receiving a setting of a display cross section relating to the heart valve;
adjusting display positions of the set display cross sections related to the cardiac valve based on the morphological information of the cardiac valve;
A medical image processing method comprising: an acquisition function for acquiring volume data including at least a heart valve;
an extraction function for extracting a region corresponding to the cardiac valve included in the volume data;
An identification function for identifying each of a plurality of leaflets included in the cardiac valve;
A display control function for displaying each of the plurality of leaflets in a different display form;
a reception function for receiving settings of a display cross section relating to the cardiac valve;
on the computer,
The display control function is a medical image processing program that displays reference information indicating the position of the display cross section and a display area of the display cross section in a display form that corresponds to the display form of a leaflet that corresponds to the position of the display cross section set by the reception function among the multiple leaflets. an acquisition function for acquiring volume data including at least a heart valve;
an extraction function for extracting a region corresponding to the cardiac valve included in the volume data;
An identification function for identifying each of a plurality of leaflets included in the cardiac valve;
A display control function for displaying each of the plurality of leaflets in a different display form;
a receiving function for receiving a setting for moving a position of at least one of a plurality of display cross sections relating to the cardiac valve;
Run the following on your computer:
The display control function is a medical image processing program that, when displaying reference information indicating the positions of the multiple displayed cross sections, displays reference information indicating the position of the moved displayed cross section before the movement and reference information indicating the position after the movement in similar display forms. an acquisition function for acquiring volume data including at least a heart valve;
an extraction function for extracting a region corresponding to the cardiac valve included in the volume data;
An identification function for identifying each of a plurality of leaflets included in the cardiac valve;
A display control function for displaying each of the plurality of leaflets in a different display form;
a reception function for receiving settings of a display cross section relating to the cardiac valve;
Run the following on your computer:
The display control function adjusts display positions of a plurality of display cross sections relating to the cardiac valve, which are set by the reception function, based on morphological information of the cardiac valve.